"AI-Based Country Scale Road Import"

Hello Stephan,

Thank you for your feedback. Based on our analysis, we found that out of the 5919 duplicate geometries shown in the Osmose tool, Facebook mappers added 35. We appreciate your time in helping us flag this issue. We have taken steps to correct this in our workflow to ensure that we prevent this in the future.

As noted on our wiki, these are our mappers with links to their profiles and names. All of our edits go through one of these accounts.

Jaclyn (https://www.openstreetmap.org/user/VLD001) - VLD001
Annie (https://www.openstreetmap.org/user/VLD002) - VLD002
Alexandra (https://www.openstreetmap.org/user/VLD003) - VLD003
Sheffield (https://www.openstreetmap.org/user/VLD004) - VLD004
Mei (https://www.openstreetmap.org/user/VLD005) - VLD005
Joseph (https://www.openstreetmap.org/user/VLD006) - VLD006
Kurt (https://www.openstreetmap.org/user/VLD007) - VLD007
Pablo https://www.openstreetmap.org/user/VLD008)) VLD008
Goerge (https://www.openstreetmap.org/user/RVR001) - RVR001
Jeff (https://www.openstreetmap.org/user/RVR002) - RVR002
David (https://www.openstreetmap.org/user/RVR003) - RVR003
Yunzhi (https://www.openstreetmap.org/user/RVR004) - RVR004
Adrian (https://www.openstreetmap.org/user/RVR005) - RVR005
Stefani (https://www.openstreetmap.org/user/RVR006) - RVR006
Mohamed (https://www.openstreetmap.org/user/RVR007) - RVR007

The following changeset comments are also built into the tools so any edit we make by default will contain #nsroadimport #thailand.

Thank-you.

Best,
Drishtie

Thanks Drishtie.

During the initial tidy-up there were people using other accounts for changesets with “#nsroadimport #thailand” tags but I guess that this was just accidental in the hurry to fix the data.

With regard to the “new mappers” behind e.g. https://www.openstreetmap.org/way/472343677/history and https://www.openstreetmap.org/way/472279193/history , it just looks like a common-or-garden geography class project to me. It ticks all the usual boxes - uploads only buildings, first time users using JOSM, very few changesets each, all changesets done within a few days (i.e. when they move on to the next part of their geography course).

Maybe someone in this forum knows a geography class somewhere who it might be, maybe not. It can be useful to find out as these classes tend to run periodically with new students each time.

Best Regards,

Andy

Hello Drishtie,

Thanks for providing the list of user accounts used for your edits. I changed your wiki page so that a search for the username actually finds it. I missed it because the search did not catch it due to the name not mentioned.

So you are saying that Facebook is not involved in the recent addition of over 20.000 building footprints in the south of Thailand, right? It simply matched from a timeline and geographical point of view the areas you said you are working on.

But ok, let’s assume it is a different party. In that case the source of the duplicates are indeed not related you your edits. They come from these building footprints which uploads duplicate geometry and also point different outlines on top of each others.
https://www.openstreetmap.org/#map=19/7.68799/100.37925

As Andy asked already: Does anyone knows details about who is responsible to setting up tasks for mappers to draw building outlines on a large scale?

Stephan

Thank you for the clarification Andy.

Stephan, I can confirm that Facebook was not involved in the recent addition of over 20.000 building footprints in the south of Thailand. We are only working on roads. :slight_smile:

Best,
Drishtie

Lying with graphs/data is very easy to do, this graph created by osmose[0] would better represent if someone has done a “poor import” by showing an increase of overlapping highways (usually caused by importing data twice or overlapping existing OSM data with imported data). By eliminating the “no-road” data you don’t see any spikes that would indicate a poor import in Thailand.

[0] http://osmose.openstreetmap.fr/en/errors/?country=thailand&item=1070

  • To preview the graph, at the very top click on “graph”

It may not all be facebook(there might be some):
I have the same spike in Canada in Jan/Feb:
http://osmose.openstreetmap.fr/en/errors/graph.png?country=canada_alberta&item=1230
http://osmose.openstreetmap.fr/en/errors/graph.png?country=canada_british_columbia&item=1230
http://osmose.openstreetmap.fr/en/errors/graph.png?country=canada_manitoba&item=1230
http://osmose.openstreetmap.fr/en/errors/graph.png?country=canada_new_brunswick&item=1230
http://osmose.openstreetmap.fr/en/errors/graph.png?country=canada_newfoundland_and_labrador&item=1230
http://osmose.openstreetmap.fr/en/errors/graph.png?country=canada_northwest_territories&item=1230
http://osmose.openstreetmap.fr/en/errors/graph.png?country=canada_nova_scotia&item=1230
http://osmose.openstreetmap.fr/en/errors/graph.png?country=canada_quebec&item=1230
http://osmose.openstreetmap.fr/en/errors/graph.png?country=canada_ontario&item=1230
http://osmose.openstreetmap.fr/en/errors/graph.png?country=canada_nunavut&item=1230
http://osmose.openstreetmap.fr/en/errors/graph.png?country=canada_prince_edward_island&item=1230
http://osmose.openstreetmap.fr/en/errors/graph.png?country=canada_saskatchewan&item=1230
http://osmose.openstreetmap.fr/en/errors/graph.png?country=canada_yukon&item=1230
Most likely explanation is that they added a new method of calculating “Duplicate geometry”, my guessing is that they added “Duplicated node without tag”
Cause I doubt anyone is extensively importing CanVEC(most of the issues seem to be from that) in all provinces/territories, so blindly(without fixing errors).

That graphic does not tell 100% the truth if you look into the spike as they changed the metric in which it’s evaluated mid-way(very flawed way of doing it) and would explain the massive spike…

Strange. A few years ago I found such strange edits in the Ijen volcano area in Indonesia. Also overlapping buildings created by users editing at the same time.

Drishtie said it was not Facebook, so let’s believe it.
It most likely is some sort of mapping party/tasking manager related stuff with a very defined area activity and a very limited focus on what to map. I don’t see any of the users involved in the duplicate buildings mapping somewhere else. If we are not getting response we might simply clean up the duplicates and move on with daily business. Most of the building footprints mapped there are fine. Just some don’t match buildings/overlap or are not rectangular at all.

Hello Drishtie and Facebook team,

We had a mapper meeting in Chiang Mai yesterday and spoke about how to deal with your AI-assisted mapping.

We acknowledge that this might speed up completing the road network. As you are keen to try this new technology we think it is better you do it in a controlled environment here in Thailand where mappers are around to check your contributions instead of having it done somewhere in Africa.

Saying that, we had been quite unhappy about your way of interacting with the community.
We ask you to interact with the OSM community in a much more open way than done previously.
We understand that it might be difficult for you to achieve this in a large company with Facebook not being used to interact with Open Source communities. Still we are confident that you will find a way to bring the interaction with the community to a satisfying level.

To proceed with your assisted import we are fine to run another test-drive together with your team.
We have set up some conditions that we believe makes it easier for all of us to run such a test.

  1. We ask you to define an area in Thailand where you plan to import. This area has to be communicated to us and agreed on beforehand.
    We also ask that you choose an area having high-resolution Bing imagery available. As we don’t get access to the DigitalGlobe imagery you are using we want to have a way to verify the accuracy of your work.

  2. We ask you to restrict the import to higher road classes. So don’t import agricultural and rice-field tracks.

  3. We also ask you to only import roads where your algorithm has a high confidence level of the geometry and classification.

After we (Facebook and the community) agreed on a test area you do the import. You will give us feedback on the progress and we will review your edits and give feedback regarding the quality.
In the case of a positive outcome we will jointly discuss on how to scale this up.

We believe the process outlined above is a feasible approach considering the wishes and concerns of all involved parties. Still it only reflects a fraction of the mappers active in Thailand.
We publicly post it and explicitly ask for others not been part of the meeting for comments or suggestions on the proceeding.

I totally agree with this approach. Because we cannot use the same imagery as the Facebook team, there must be some way for us to at least view the work of the team before we can have confidence that its contributions will be useful. Limiting the additions to major highways and working in an area having high-quality Bing imagery will facilitate our evaluation greatly.

This method was the one we agreed upon last night.

Dave Swarthout

Stephan, I agree with your suggestion. Let’s hope the Facebook team will also agree, and we can start working together on a better map of Thailand.

To make it clear. The statement above is a joint response we spoke about on last Friday meeting in Chiang Mai.

I posted a notice on the import mailing list. Suppose Facebook wants to discuss it internally before responding. So I don’t expect an answer before mid next week.

People present at the meeting this Friday was:
http://www.openstreetmap.org/user/Johnny%20Carlsen
http://www.openstreetmap.org/user/Tom%20Layo
http://www.openstreetmap.org/user/stephankn
http://www.openstreetmap.org/user/AlaskaDave
http://www.openstreetmap.org/user/sojo

Hello Stephan and Johnny,

As suggested created a new post here.

Thanks,
Drishtie

I agree with the statement and would like to offer some points for discussion.

  1. Rural and farm roads should be added as well, perhaps in smaller grids, but it is these communities that can benefit the most. Perhaps it can be considered “Alpha” until we think it’s suitable. This is extremely useful data for the cycling community.

  2. Verification, not only in DG but Mapbox imagery is suitable, having pretty good imagery for roads in most of Thailand. But for a controversial idea, I would also like to offer verification with Google’s imagery. We’re not actually tracing or deriving anything, right? deploy_flamesuit()

Thank you everyone for your hard work.

Best regards
Mishari

I’m afraid that would not be legal. Sorry. I know it sucks, and it makes some things easier, but Google won’t let us use their imagery in OpenStreetMap. Maybe someday, but that’s not today alas.

@Rory: I think you missed the twist in Mishari’s idea. It is not about using Google for deriving OSM stuff.

He suggested to overlay a Google Map with another layer (which would be OSM data). This is similar to the example in the maps tutorial showing how to combine Google maps with your own data. As long as this is not used to create OSM data or augment OSM data using Google’s data , this sounds perfectly fine with their terms and conditions.
If you see a specific term restricting this use would be glad you point it out. I did read through their pages and they refer to multiple terms sub-pages. I might have missed something.

@Mishari: We wanted to go without the agricultural tracks as these are a bit harder to detect and validate on the imagery we have. Also while I understand that cyclists like to use minor roads I have to admit that I have never seen them using unpaved agricultural tracks. But as you cycle a lot more than I do I believe you have more insight on this.

Seems Facebook also would prefer to import everything looking like a road and rely on their human team to do the classification. So including tracks could result in declaring the import later having too many faults to be acceptable. What if the opinion the the others on this?

Perhaps a better definition of what is meant by the term “agricultural tracks” would help. My thinking was that they should avoid adding the small tracks that, for example, run from a paved road to a farm field and end there. These little tracks are scattered about everywhere in Thailand. However, an unpaved road that has a discernable beginning and destination with other highways and is well-defined enough to be visible on Bing is a good candidate for inclusion as a highway.

They should also read through the Thailand Wiki information, if they haven’t already, so as to become familiar with the tagging conventions we use.

Maybe I didn’t get the memo, but it looks like work has started: http://www.openstreetmap.org/node/4833439535/history :slight_smile:

Best Regards,
Andy

Have a look at this thread: https://forum.openstreetmap.org/viewtopic.php?id=57942