OpenStreetMap Forum

The Free Wiki World Map

You are not logged in.

Announcement

A fix has been applied to the login system for the forums - if you have trouble logging in please contact support@openstreetmap.org with both your forum username and your OpenStreetMap username so we can make sure your accounts are properly linked.

#26 2017-03-04 00:41:20

DrishT
Member
Registered: 2017-01-14
Posts: 27

Re: "AI-Based Country Scale Road Import"

Hello all,

Thank you again for all of this feedback. We joined this forum to work with you directly and support high-quality data with your expertise. We are always willing to share and answer any questions you have. My sole focus at Facebook is on OSM, so you will reach me and my team directly if you talk to us through these discussion posts or email us at osm@fb.com (please email if you need a quicker response). Our goal is your goal: to efficiently and accurately create maps powered by our community.

We look forward to continuing to share our derived data so the community can work together to create high-quality maps. We will be sharing sample data with the community next week via the import list. Following that, we'll share the area we hope to start mapping in Thailand through this forum.

We are currently using the HOT Tasking Manager and iD tool and are working to share our editions/changes to these tools as well. Stay tuned.

Have a lovely weekend wherever you are!
Drish

Last edited by DrishT (2017-03-04 00:42:48)

Offline

#27 2017-03-04 09:06:09

AlaskaDave
Member
From: Homer, Alaska ; Chiang Mai
Registered: 2013-09-21
Posts: 257
Website

Re: "AI-Based Country Scale Road Import"

Thanks to Kevin for clarifying the intentions and philosophy of DigitalGlobe. We are indebted to the people who produce the satellite imagery we use every day and without which we would be lost.

Drish, I wonder if the iD editor is the best one to use for this project. I am not all that familiar with it but the few times I have used it, I found it very limiting. The learning curve is steeper but I think using JOSM will save time and prevent problems in the end. It is very customizable. One can create and use "presets" for example, that make data entry more uniform. In an effort like yours, where many people are adding similar data, you don't want some of them to tag an object one way while some others choose a different tagging scheme. Also, tagging an object with a preset requires only a couple of keystrokes no matter how many tags are involved.

It's your project and your decision might need to weigh other important factors. Many of the most prolific contributors to OSM use Potlatch or iD so you are in fine company in that regard. Still, I feel that you might as well be using the most powerful and flexible OSM editor available and that is, unquestionably, JOSM.

Best Regards,

Dave

Offline

#28 2017-03-04 18:01:59

rorym
Member
Registered: 2008-04-23
Posts: 23

Re: "AI-Based Country Scale Road Import"

AlaskaDave wrote:

Drish, I wonder if the iD editor is the best one to use for this project. I am not all that familiar with it but the few times I have used it, I found it very limiting. The learning curve is steeper but I think using JOSM will save time and prevent problems in the end. It is very customizable. One can create and use "presets" for example, that make data entry more uniform. In an effort like yours, where many people are adding similar data, you don't want some of them to tag an object one way while some others choose a different tagging scheme. Also, tagging an object with a preset requires only a couple of keystrokes no matter how many tags are involved.

Another feature of JOSM is the inbuild validator which can detect common mistakes, like crossing ways, not connected way, bad tagging, etc. The Find and To-do plugin can help find and fix errors like this. In computer generated imports like this, you want to be able to automatically detect and fix problems to ensure a high quality of data.

Offline

#29 2017-03-04 18:49:53

Bernhard Hiller
Member
Registered: 2011-05-10
Posts: 619

Re: "AI-Based Country Scale Road Import"

The White Spots on the map of Thailand are still enormously big. Last December in Isaan, I was in places with nothing at all mapped within 10 km from my location. And let's take a look at the contributors to the map of Thailand: most data were contributed by farangs.
We need some organisations helping us to complete some tasks. Roads, rivers, landuse can be seen on images. Place names, road numbers, shops, hotels, etc. can't be detected there: we need people on site who will do that. And I hope they'll more likely be willing to contribute when they see that the map is good (which it isn't - also Google maps actually isn't good as it does not differentiate between roads and tracks).
Although the examples shown by Stephan and Dave are very terrible, let's try to integrate the FB team into the OSM community. With a slower approach, they could get feedback on small change sets, without causing too much disturbances with bad data. Also try to map connected items instead of singular segments. As I see from Dave's comment, it could be helpful if the FB team tried to work in places where good and recent imagery is available also to us (instead of only to them).

Offline

#30 2017-03-05 07:50:50

DrishT
Member
Registered: 2017-01-14
Posts: 27

Re: "AI-Based Country Scale Road Import"

Hello Dave and Rorym,

Thank you for the advice on tooling. As mentioned in our wiki page we do use both JOSM and iD. We totally agree that JOSM is superior for what we are trying to do and are all familiar with both tools. In fact, we started primarily with JOSM for the exact reasons you mentioned smile In order to create a better internal process, our engineers have made changes to the iD editor to mirror the cool features JOSM has.

For example here are a few changes we have made to iD that help us be more efficient.

* The ability to load/save locally generated .osm files much smoother and faster.
* Customizing colors and highlighting specific point, lines or polygons. For example, we highlight roads so editors can quickly tell the difference between our edits versus what is currently in OSM.
* Added conflict validation to iD so edits cannot be saved unless all conflicts are resolved.
* Batch tagging for multiple selected roads
* Automated suggestion for road connections. So we can easily connect roads in neighbouring tiles when working in the bounding box of a task from the tasking manager.

We are happy to share these and are working on ways to do this in a thoughtful way.

Best,
Drish

Offline

#31 2017-03-06 06:53:09

PlaneMad
Member
Registered: 2008-09-16
Posts: 20

Re: "AI-Based Country Scale Road Import"

* The ability to load/save locally generated .osm files much smoother and faster.
* Customizing colors and highlighting specific point, lines or polygons. For example, we highlight roads so editors can quickly tell the difference between our edits versus what is currently in OSM.
* Added conflict validation to iD so edits cannot be saved unless all conflicts are resolved.
* Batch tagging for multiple selected roads
* Automated suggestion for road connections. So we can easily connect roads in neighbouring tiles when working in the bounding box of a task from the tasking manager.

These sound great and make powerful editing features that were only available in JOSM more accessible to mappers. Looking forward to a publicly usable fork.

Offline

#32 2017-03-18 10:56:02

stephankn
Member
Registered: 2010-05-04
Posts: 385

Re: "AI-Based Country Scale Road Import"

Again FB shows that they are not interested in cooperation with the community. Instead of presenting their ideas and working together with the community on improving the tooling they speak about creating their own toys and a fork.

Besides that:
When will the existing mess be cleaned up? Does Facebook expect the community to clean up after them?
I'm tempted to immediately revert uploads done by RVR007 and others. Removal of rice-field tracks are no big loss.

Duplicates for example in those changesets:
https://www.openstreetmap.org/changeset/46112980
https://www.openstreetmap.org/changeset/46117993

The number of duplicate geometry skyrocketed from non-existent to 10.000 Duplicated in 2017 thanks to technically bad executed Facebook imports.

duplicate_geometry_thailand.png

Offline

#33 2017-03-18 11:22:32

SomeoneElse
Member
Registered: 2010-10-13
Posts: 584

Re: "AI-Based Country Scale Road Import"

stephankn wrote:

When will the existing mess be cleaned up? Does Facebook expect the community to clean up after them?
I'm tempted to immediately revert uploads done by RVR007 and others. Removal of rice-field tracks are no big loss.

Given that it's likely that most of these errors were introduced by Facebook, it would be good to see a response by e.g. Drishtie about this here.

Further up this thread I said:

SomeoneElse wrote:

If any of the edits by any of the above mappers look problematical and you'd like help with reverting them, then you can contact the Data Working Group by email on data@osmfoundation.org .

and that still stands.  There are a number of tools around that can "revert everything that hasn't been since touched by another mapper (e.g. to correct it)".  In this case it's not quite as straightforward as it might be because non-import accounts used for other mapping activities have been used for the import.

Although I'd be normally very happy for a person who made a buggy import to revert it themselves, given the lack of technical competence demonstrated by Facebook so far I'd personally be somewhat concerned if they did the revert on their own without anyone else checking it.  If Drishtie or someone else at Facebook wants to do the revert, then it's best that they say so here before doing so.  If someone from this forum wants to do it, great - but probably best to say so here first to avoid duplication.  If not, the DWG can clean up the data (which as I understand it was all imported by mistake), and if I do it I'll say so here first.

Best Regards,
Andy (DWG)

Offline

#34 2017-03-18 11:52:07

rorym
Member
Registered: 2008-04-23
Posts: 23

Re: "AI-Based Country Scale Road Import"

For those not on the imports mailing list, Facebook have posted to it about this import: https://lists.openstreetmap.org/piperma … 04840.html

Offline

#35 2017-03-19 04:30:00

DrishT
Member
Registered: 2017-01-14
Posts: 27

Re: "AI-Based Country Scale Road Import"

Hello Andy and Stephan,

Thank-you for your feedback. We are looking into duplicate geometry and will make sure to fix that example you shared and any others we find.

Best,
Drishtie Patel

Offline

#36 2017-03-20 17:48:20

mvexel
Member
From: Salt Lake City, UT
Registered: 2009-06-02
Posts: 41
Website

Re: "AI-Based Country Scale Road Import"

stephankn wrote:

The number of duplicate geometry skyrocketed from non-existent to 10.000 Duplicated in 2017 thanks to technically bad executed Facebook imports.

https://s5.postimg.org/w728zc1rb/duplic … ailand.png

Hey Stephan -- That is useful info. I am wondering how you created this chart? (Could be useful for other mappers).

Also agreed that this should be cleaned up. If you can supply the data (as OSM or GeoJSON) I can make a MapRoulette challenge from it.

Perhaps as a community we can figure out how to help FB do better?

Offline

#37 2017-03-20 20:33:39

DenisCarriere
Member
Registered: 2016-11-17
Posts: 6

Re: "AI-Based Country Scale Road Import"

@Stephan I'd also be interested in seeing where you came up with those numbers.

From what I can tell from OSM Inspector, Thailand has barely any issues with duplicate ways (only about 10 ways have issues out of the entire country - in my opinion that's pretty darn good!).
https://tools.geofabrik.de/osmi/

Out of those 10 ways that have issues only 1 of them is from Facebook and another is from yourself @stephank, you don't validate before pushing your edits?
http://www.openstreetmap.org/way/474870936

Stephane, next time you try to use a fancy outdated graph to threaten a revert, please try to actually look at the OSM data before you speak.

bye

Offline

#38 2017-03-20 21:02:36

DenisCarriere
Member
Registered: 2016-11-17
Posts: 6

Re: "AI-Based Country Scale Road Import"

@Stephan: Have you also considered the cause of the increase duplicate nodes from PokemonGo vandalism.

I was able to find this changeset which was flagged as "Overlapping Highways" (clearly not a Facebook edit).
https://www.openstreetmap.org/changeset/45552933

Offline

#39 2017-03-20 23:30:02

stephankn
Member
Registered: 2010-05-04
Posts: 385

Re: "AI-Based Country Scale Road Import"

@Denis: way 474870936 was one of the tagging fixes for Balthus. It fixes only tags. If there had been duplicates before they had not been touched by this. Was not the scope of fixing the tags.

The statistic comes from Osmose checks.

Check for yourself.
http://osmose.openstreetmap.fr/en/error … 30&class=5

I opened the first five nodes on the list:
http://www.openstreetmap.org/node/4689300837
http://www.openstreetmap.org/node/4689300837
http://www.openstreetmap.org/node/4691069588
http://www.openstreetmap.org/node/4691069588
http://www.openstreetmap.org/node/4691071904

Don't try to blame pokemon users. These mentioned changes all have "#nsroadimport #thailand" listed in the comment. This clearly points towards Facebook.

Again: It is Facebook doing imports. So it's Facebook to prove they are executing this in a technically correct way. We have enough examples indicating this is not the case!

And don't try to switch topic. Yes, there are other bad edits in the database. But here we are discussion about those caused by Facebook.

Offline

#40 2017-03-21 00:02:57

stephankn
Member
Registered: 2010-05-04
Posts: 385

Re: "AI-Based Country Scale Road Import"

Facebook, you still did not tell use what user names you are using for the import. Why? What do you want to hide?

Could it be that this user also belongs to you? His patterns match what you are doing...

http://www.openstreetmap.org/user/Tanavut

Offline

#41 2017-03-21 00:16:24

stephankn
Member
Registered: 2010-05-04
Posts: 385

Re: "AI-Based Country Scale Road Import"

Maybe you also forgot to mention that this user is also editing for you?

http://www.openstreetmap.org/user/konhin121

What else?

Offline

#42 2017-03-21 05:59:03

DrishT
Member
Registered: 2017-01-14
Posts: 27

Re: "AI-Based Country Scale Road Import"

Hello Stephan,

Thank you for your feedback. Based on our analysis, we found that out of the 5919 duplicate geometries shown in the Osmose tool, Facebook mappers added 35.  We appreciate your time in helping us flag this issue. We have taken steps to correct this in our workflow to ensure that we prevent this in the future.

As noted on our wiki, these are our mappers with links to their profiles and names. All of our edits go through one of these accounts. 

Jaclyn (https://www.openstreetmap.org/user/VLD001) - VLD001
Annie (https://www.openstreetmap.org/user/VLD002) - VLD002
Alexandra (https://www.openstreetmap.org/user/VLD003) - VLD003
Sheffield (https://www.openstreetmap.org/user/VLD004) - VLD004
Mei (https://www.openstreetmap.org/user/VLD005) - VLD005
Joseph (https://www.openstreetmap.org/user/VLD006) - VLD006
Kurt (https://www.openstreetmap.org/user/VLD007) - VLD007
Pablo https://www.openstreetmap.org/user/VLD008) VLD008
Goerge (https://www.openstreetmap.org/user/RVR001) - RVR001
Jeff (https://www.openstreetmap.org/user/RVR002) - RVR002
David (https://www.openstreetmap.org/user/RVR003) - RVR003
Yunzhi (https://www.openstreetmap.org/user/RVR004) - RVR004
Adrian (https://www.openstreetmap.org/user/RVR005) - RVR005
Stefani (https://www.openstreetmap.org/user/RVR006) - RVR006
Mohamed (https://www.openstreetmap.org/user/RVR007) - RVR007

The following changeset comments are also built into the tools so any edit we make by default will contain #nsroadimport #thailand.

Thank-you.

Best,
Drishtie

Offline

#43 2017-03-21 08:54:01

SomeoneElse
Member
Registered: 2010-10-13
Posts: 584

Re: "AI-Based Country Scale Road Import"

Thanks Drishtie.

During the initial tidy-up there were people using other accounts for changesets with "#nsroadimport #thailand" tags but I guess that this was just accidental in the hurry to fix the data.

With regard to the "new mappers" behind e.g. https://www.openstreetmap.org/way/472343677/history and https://www.openstreetmap.org/way/472279193/history , it just looks like a common-or-garden geography class project to me.  It ticks all the usual boxes - uploads only buildings, first time users using JOSM, very few changesets each, all changesets done within a few days (i.e. when they move on to the next part of their geography course).

Maybe someone in this forum knows a geography class somewhere who it might be, maybe not.  It can be useful to find out as these classes tend to run periodically with new students each time.

Best Regards,

Andy

Offline

#44 2017-03-21 12:23:21

stephankn
Member
Registered: 2010-05-04
Posts: 385

Re: "AI-Based Country Scale Road Import"

Hello Drishtie,

Thanks for providing the list of user accounts used for your edits. I changed your wiki page so that a search for the username actually finds it. I missed it because the search did not catch it due to the name not mentioned.

So you are saying that Facebook is not involved in the recent addition of over 20.000 building footprints in the south of Thailand, right? It simply matched from a timeline and geographical point of view the areas you said you are working on.

But ok, let's assume it is a different party. In that case the source of the duplicates are indeed not related you your edits. They come from these building footprints which uploads duplicate geometry and also point different outlines on top of each others.
https://www.openstreetmap.org/#map=19/7.68799/100.37925

As Andy asked already: Does anyone knows details about who is responsible to setting up tasks for mappers to draw building outlines on a large scale?

Stephan

Offline

#45 2017-03-22 00:09:14

DrishT
Member
Registered: 2017-01-14
Posts: 27

Re: "AI-Based Country Scale Road Import"

Thank you for the clarification Andy.

Stephan, I can confirm that Facebook was not involved in the recent addition of over 20.000 building footprints in the south of Thailand. We are only working on roads. smile

Best,
Drishtie

Offline

#46 2017-03-22 14:55:32

DenisCarriere
Member
Registered: 2016-11-17
Posts: 6

Re: "AI-Based Country Scale Road Import"

Lying with graphs/data is very easy to do, this graph created by osmose[0] would better represent if someone has done a "poor import" by showing an increase of overlapping highways (usually caused by importing data twice or overlapping existing OSM data with imported data). By eliminating the "no-road" data you don't see any spikes that would indicate a poor import in Thailand. graph.png?country=thailand&item=1070

[0] http://osmose.openstreetmap.fr/en/error … &item=1070

* To preview the graph, at the very top click on "graph"

Offline

#47 2017-03-22 18:04:42

LogicalViolinist
Member
Registered: 2016-11-18
Posts: 10

Re: "AI-Based Country Scale Road Import"

stephankn wrote:

@Denis: way 474870936 was one of the tagging fixes for Balthus. It fixes only tags. If there had been duplicates before they had not been touched by this. Was not the scope of fixing the tags.

The statistic comes from Osmose checks.

Check for yourself.
http://osmose.openstreetmap.fr/en/error … 30&class=5

I opened the first five nodes on the list:
http://www.openstreetmap.org/node/4689300837
http://www.openstreetmap.org/node/4689300837
http://www.openstreetmap.org/node/4691069588
http://www.openstreetmap.org/node/4691069588
http://www.openstreetmap.org/node/4691071904

Don't try to blame pokemon users. These mentioned changes all have "#nsroadimport #thailand" listed in the comment. This clearly points towards Facebook.

Again: It is Facebook doing imports. So it's Facebook to prove they are executing this in a technically correct way. We have enough examples indicating this is not the case!

And don't try to switch topic. Yes, there are other bad edits in the database. But here we are discussion about those caused by Facebook.

It may not all be facebook(there might be some):
I have the same spike in Canada in Jan/Feb:
http://osmose.openstreetmap.fr/en/error … &item=1230
http://osmose.openstreetmap.fr/en/error … &item=1230
http://osmose.openstreetmap.fr/en/error … &item=1230
http://osmose.openstreetmap.fr/en/error … &item=1230
http://osmose.openstreetmap.fr/en/error … &item=1230
http://osmose.openstreetmap.fr/en/error … &item=1230
http://osmose.openstreetmap.fr/en/error … &item=1230
http://osmose.openstreetmap.fr/en/error … &item=1230
http://osmose.openstreetmap.fr/en/error … &item=1230
http://osmose.openstreetmap.fr/en/error … &item=1230
http://osmose.openstreetmap.fr/en/error … &item=1230
http://osmose.openstreetmap.fr/en/error … &item=1230
http://osmose.openstreetmap.fr/en/error … &item=1230
Most likely explanation is that they added a new method of calculating "Duplicate geometry", my guessing is that they added "Duplicated node without tag"
Cause I doubt anyone is extensively importing CanVEC(most of the issues seem to be from that) in all provinces/territories, so blindly(without fixing errors).

That graphic does not tell 100% the truth if you look into the spike as they changed the metric in which it's evaluated mid-way(very flawed way of doing it) and would explain the massive spike...

Last edited by LogicalViolinist (2017-03-22 18:16:02)

Offline

#48 2017-03-22 18:52:58

Bernhard Hiller
Member
Registered: 2011-05-10
Posts: 619

Re: "AI-Based Country Scale Road Import"

SomeoneElse wrote:

it just looks like a common-or-garden geography class project to me.  It ticks all the usual boxes - uploads only buildings, first time users using JOSM

Strange. A few years ago I found such strange edits in the Ijen volcano area in Indonesia. Also overlapping buildings created by users editing at the same time.

Offline

#49 2017-04-01 08:50:50

stephankn
Member
Registered: 2010-05-04
Posts: 385

Re: "AI-Based Country Scale Road Import"

Drishtie said it was not Facebook, so let's believe it.
It most likely is some sort of mapping party/tasking manager related stuff with a very defined area activity and a very limited focus on what to map. I don't see any of the users involved in the duplicate buildings mapping somewhere else.  If we are not getting response we might simply clean up the duplicates and move on with daily business. Most of the building footprints mapped there are fine. Just some don't match buildings/overlap or are not rectangular at all.

Offline

#50 2017-04-01 08:54:10

stephankn
Member
Registered: 2010-05-04
Posts: 385

Re: "AI-Based Country Scale Road Import"

Hello Drishtie and Facebook team,

We had a mapper meeting in Chiang Mai yesterday and spoke about how to deal with your AI-assisted mapping.

We acknowledge that this might speed up completing the road network. As you are keen to try this new technology we think it is better you do it in a controlled environment here in Thailand where mappers are around to check your contributions instead of having it done somewhere in Africa.

Saying that, we had been quite unhappy about your way of interacting with the community.
We ask you to interact with the OSM community in a much more open way than done previously.
We understand that it might be difficult for you to achieve this in a large company with Facebook not being used to interact with Open Source communities. Still we are confident that you will find a way to bring the interaction with the community to a satisfying level.

To proceed with your assisted import we are fine to run another test-drive together with your team.
We have set up some conditions that we believe makes it easier for all of us to run such a test.

1) We ask you to define an area in Thailand where you plan to import. This area has to be communicated to us and agreed on beforehand.
We also ask that you choose an area having high-resolution Bing imagery available. As we don't get access to the DigitalGlobe imagery you are using we want to have a way to verify the accuracy of your work.

2) We ask you to restrict the import to higher road classes. So don't import agricultural and rice-field tracks.

3) We also ask you to only import roads where your algorithm has a high confidence level of the geometry and classification.

After we (Facebook and the community) agreed on a test area you do the import. You will give us feedback on the progress and we will review your edits and give feedback regarding the quality.
In the case of a positive outcome we will jointly discuss on how to scale this up.

We believe the process outlined above is a feasible approach considering the wishes and concerns of all involved parties. Still it only reflects a fraction of the mappers active in Thailand.
We publicly post it and explicitly ask for others not been part of the meeting for comments or suggestions on the proceeding.

Offline

Board footer

Powered by FluxBB