Test Drive - AI-assisted road import by Facebook

The “wall issue” in Egypt is different from a “wall issue” in Thailand: often new neighborhoods built near towns have a wall around them, and there is only single entry to the neighborhood.
I found residential roads from such a neighborhood connected to a track (which of course was tagged as an unclassified) nearby. The wall was not ignored.

I sent following message to the Facebook Team members who were active in the Phetburi area:

The people contacted are:
VLD003
VLD009
VLD010
RVR007
VLD011
VLD007
RVR002
RVR009

Hello,

Thank you as always for your feedback. Happy New Year! I hope you’ve all had a lovely holiday season. Sorry for the delay in response we are just getting back into work mode :slight_smile:

I’m sad to hear you are seeing issues. I hope you see we have been responsive to all requests and quickly make edits whenever anyone in the community asks. In some cases the requests often conflict each other so it not as easy to made these decisions. I understand your frustration on roads and routing especially having travelled through many rural areas in Africa, South America and Asia including Thailand! So let’s see how we can make this better. Our goals are the same of making OSM better and we are super grateful for your on the ground knowledge.

As far as quality and data integrity, while our roads are generated by AI our team edits each task 3 different times using the Tasking Manager to ensure various team members are editing consistently and at a high level of quality. We also use quality checks created at Facebook and open source ones such as JOSM, OSMCha, OSMOSE and Keep Right. Additionally our Quality Analysts continuously check live data.

There is quite a bit to digest from your e-mails so let me try to summarize so I can work with the team to make sure we can help solve this. The two main issues are.

  1. Too Many nodes on straight roads - We are looking into this and will make changes accordingly.
  2. Road Tagging - The tagging is all done by the editing team and not automated and we are cognizant of the limitations of remote mappers, so try to follow the best practices we’ve learned from the local community and OSM guidelines.
    1. Tracks - Earlier we were told by some community members not to use this tag, however some have found it useful. After studying local edits, we decided to keep a minimum amount of Tracks for roads that are mostly agricultural or forestry.
    2. Residential - In an effort to make sure we do not neglect homes, we chose to mark any roads which serve as an access to housing, without function of connecting settlements. Sometimes this mean we are making the decisions if we see 1-5 buildings on that road. I understand things change on the ground and maybe those building do no actually exist as we see in satellite imagery. In those cases we are super grateful you can re-tag the roads based on your knowledge. If you have a different suggestions to tackle these roads please do share and we will be happy to take that into account and make changes.
    3. Unclassified - As seen from most of the local mapping in Southern Thailand unclassified is used to link villages and hamlets. This is why we mark minor roads of a lower classification than tertiary, but are not used to access houses unclassified roads.
    4. Paved/Unpaved - As much as we would like to add this tag it is difficult to do so with high accuracy so as a remote mapper. As the local community if you have some group guidance on this it would be great for us to follow. We assume based on OSM guidelines that the use of the road is what determines the Highway tag and the quality is then set in this case as either paved or unpaved. Is this the same way you see roads?

The best part of OSM is the community working together to make the map better. If your rules for tagging are different from the way we are doing so please may you provide some examples (way IDs and how you would tag them differently) and so we can learn with you and make the map better especially with your local knowledge. I think reverting thousands of really good edits due to some tagging concerns would be a loss for OSM.

For holidays and weekends you can always also e-mail my team directly osm@fb.com as a faster way to reach us.

Best,
Drishtie

Actually (and this applies to all sides of the debate here) I’d suggest that changeset discussion comments are a better way forward than a private email address. That way everyone in the community can see what potential issues there are and what is being done to address them.

Thank you for your response, DrishT and colleagues. I still welcome your project and your efforts, and hope to be able to provide some hints leading to an improved quality of the data.

I’ll collect some more examples when I’ll work on my data from the holidays.

Meanwhile, you can take a look at
https://www.openstreetmap.org/#map=14/8.5383/99.0690

The 5 “circles” there look like small settlements in a plantation.
I came from North to the settlement in the north-east, via bad tracks, some of them are not yet on the map.
Some roads in that settlement are paved, some are not which can be well detected in the imagery (Digital Globe Premium Imagery).
The “roads” to the west and east should be tracks.
The road leading out of it towards the south was somewhen paved, but most of the asphalt cover has been lost (which cannot be detected in the imagery).
The road in the center of the area leading east towards the major road #4037 is a wide asphalt road (unclassified, or perhaps even tertiary; I did not see kilometer stones or DRR signs there). With the imagery, you can see that (assuming you come from the east) it turns north/right just before you connect it to a track, and at the next junction west/left.
Almost all other “roads” in that area should be tracks.
And it is easy to see that you missed some parts of the circular roads in the settlements.

It will take a few days till I’ll edit that region. My last updates were with the GPS data and waypoints from Dec 7, and I visited the region described here on Dec 23.

Hello Bernhard,

This is a great example and we are in complete agreement on this. Look forward to more examples :slight_smile:

Appreciate your feedback.

Thanks,
Drishtie

By the way, when I wrote the message above, I opened that region in a web browser and started iD. In order to see the roads in the imagery, I had to move some nodes of the ways, because the ways are shown with very thick lines in iD by default (don’t know if or how to change that).
Maybe that could be a problem for your team also? In JOSM, the lines are thinner and the imagery can easily be seen without moving any nodes away.

During the last days I mapped along the coast south of Hua Hin. A very touristic region, hence many mappers have been there already and contributed many data. Consequently, the imports did not contribute so much and the amount of problems was less, while north of Prachuap Khiri Khan.
Well, south of Phrachuap Khiri Khan it was … terrible. But not due the imports. A user did an enormous amount of work in 2013/14, and got many things wrong, apart from highway classification also waterways (generally layer=-1, i.e. below the surface of the earth), and also connected them to roads and railways.
When you learn from such data, you can but learn it the wrong way.

I’d like to add a couple of examples with import issues which you asked for:
way id(s) original classification → correct classification
514017526 unclassified → track
513728209 residential → track
514040248+514040250 joined
514037787+514037788 joined
514332683 service → track
514345749 residential → unclassified
513730178 too many nodes

Individual mappers have to be ultimately responsible for their own individual edits. It’s great that there’s also a direct contact within the team that’s directing these mappers, but one of the strengths of OSM is the ability of individual mappers to communicate among themselves.

W makes the lines thin in the ID editor. Would it be possible to collect Mapillary or OpenStreetCam data to help the remote mappers?

More careless work from the FB import team…

Not sure if a one-off, but good evidence of the imports destroying our core Highway data through either carelessness of lack of vetting. I have corrected it.

Actually, we can’t see from this picture what kind of road it is. We can only see it isn’t paved. I agree that residential roads should have houses alongside, at least from time to time (access to rural houses will not always have dense buildings along).

If the track is legally accessible, why would you suggest to interrupt the road network at this spot, is this about surface or ownership or access restrictions?

+1, although what is a track depends a lot on the local legislation and context. In rural settings, most roads, also “normal public roads” are typically serving agricultural and forestry purposes. The decision for track is if they only serve for these purposes.

So for today’s examples of the AI mess, recently I was in Ban Pong near Phayao. A couple of times my GPS showed me the main 1193 hwy seemes off from the true position. Back on the Computer, and with the benefit of multiple GPS traces and newer imagery, we can see that the Bing image it was traced from is “off”, but the newer DG-Std image is correctly aligned.
However, I also see mass of residential roads have been added by Micheal (VLD011) without checking for image alignment, so now in addition to the Hwy, now we have to correct all those.

Furthermore, in just that one screenshot above, working downwards, I see …

1/. The road below the fuel station is actually a track leading to fields. Thats a market above it, not a house.
2/. The school on the left has its access road draw as residential, and for some reason, does not continue in a loop back to the road. Local mappers would know this and draw it as a service road, with some even adding a permissive tag.
3/. Further down on the left, we see a residential loop going around the back of the village Health Centre… of course, this would also be a service road. The health centre is not, and does not look like a house, and besides, the access road passes across the front of the building too … guess the AI missed that one.
4/. And opposite the health centre, we have the District Office … also marked as residential and with only one section of the access road drawn… in reality its a loop, and of course should be service road.

So my point yet again is … if in one small area, we find so much bad mapping, just how much damage has been done to the whole of the Thailand Map. Corrections that local mappers will never have the time to rectify.

I will also send this post to the team osm@fb.com, but please, I have not got time to comment to individuals, or post in every changeset affected as some OSM purists might suggest.

I can spend an hour a day flagging these issues, or get on with the more worthwhile job of inputting data. I know the DWG sees these posts so please, how do we go about reversing all the “imported roads” so as we local mappers, at least have a chance to enter the data in correctly. Can someone help me write a script that deletes all roads marked as import=yes ?

Is the problem really “all import=yes” roads? Ultimately it’s a human that presses the button to add the data, and I’d expect that different people will have different quality thresholds. To take an example from elsewhere, in an African semi-import by several mappers it was clear that there was one whose quality threshold was lower than the others (and in that case adding lots of duplicates). After engaging with everyone involved there we ended up reverted only that one contributor’s additions and letting everyone else continue - it got rid of most of the problems and left most of the valid additions.

If you have not got time to comment on changesets, then the DWG (just like most OSM mappers all volunteers also) are unlikely to have be able to analyse the type and scale of the problems, and other mappers in Thailand who may not frequent this forum won’t know either. It’s great to see the examples posted here (and a link to the actual data would make those much more useful), but without specific details we simply don’t know who the main offenders are, and without any comment to reply to we don’t know their side of the story.

With regard to problem editors, what we’d like to be able to do is:

  1. See comments made about a particular mapper’s work

  2. See that mapper either not respond or do the same sort of thing again.

  3. If they continue creating the same problems despite being told about them we can then take action against that mapper.

With regard to problem data, the wiki page at https://wiki.openstreetmap.org/wiki/Change_rollback is a summary of the options available (perl revert scripts and JOSM). The two main problems with any approach are:

  1. Identifying the data to be reverted.

  2. Deciding what to do with data that has been edited by other mappers afterwards (undo all their changes, keep all their changes, or review after the revert)

The approach with the perl revert scripts is generally “throw all problem data at the script, and then subject to (4) above, handle any remaining problems”. With JOSM you’d go backwards through all changes and interactively handle (4) as you go. Depending on the individual situation either of the two main options (or even some other approach) may be better.

I have been following this thread but have not commented on it because my feelings about the FB team’s work are not as strong as Russ’s. Agreed, their work isn’t perfect, there are many errors and missed connections, but overall I think having the new highways in OSM is more important than whether the ways are service, residential or otherwise. I was bicycling on a perfectly beautiful highway earlier today that was tagged residential. It is a wide, two-lane road, with smooth asphalt paving that, to my mind at least, should be tagged tertiary or at the very least unclassified. There are a few homes situated on it but in actuality, it is a high speed connector deserving of a higher classification.

These sorts of errors happen all the time, especially when new mappers are coming up to speed. I look back occasionally and am surprised to see the tagging I applied to certain objects I created years ago; I would never tag them that way now! But to go back and retag all of my older work using my current understanding would be way too much work for little real improvement. Keep in mind that the tagging is only one part, although an important part, of an OSM object’s value. Just knowing something is there is important as well.

To revert all changesets having an import=yes tag would be overkill, IMHO. I’m not sure how best to address the problems Russ mentions. These are serious issues that only the particular volunteer mapper can properly address either by taking more time to read and educate him- or herself, or by asking questions about every object they add to the database. But didn’t we all begin that way?

Respectfully submitted,

Dave (AlaskaDave)

Wait a moment…
I asked about the quality of the revert tools in the German forum: https://forum.openstreetmap.org/viewtopic.php?id=60955
woodpeck said that “complex_revert.pl” would fail when another mapper worked on that item later on.
Consequently, I reasoned that I can now do many time consuming edits with the data collected during last December - and I don’t like to postpone that, as I can still remember the situation on ground well with the help of my GPS traces, waypoints, photos, and the imagery.
During my last edits, I could correct (or sometimes verify) edits of the FB team, and lots of my editing time was committed to check their data. I do not want that to be lost.
I hope other mappers active in Thailand will agree that such edited ways should not be reverted.

Are you sure it is? I do not know. Traffic law enforcement in Thailand is … ehm, even less effective than in Italy. Hardly ever will you see a road sign blocking access (or a handwritten “ham khao”).

Just look at the message of Russ:

That’s a user experience we should really try to avoid.
Of course, marking a legally accessible - and usable for common car drivers - road to a track could prevent people from considering it in their trip planning activities. That’s not good also. Bit I believe the impact is not so bad.

Thailand is a well developing mid-income country. Unpaved roads (below tertiary) are still common, but generally they’ have a tracktype=grade2 quality. Rare exceptions do exist in very underdeveloped areas. The area near Phetburi is well developed.

By the way, I found an example of a track which is really not accessible to the public, there is a barrier preventing entry:
http://www.openstreetmap.org/way/514881100/history
And looking at the history, you’ll see that it was tagged as a residential.

I certainly don’t want that to happen, Bernard. I’ve too have edited or enhanced many FB objects in the months since they started and my position is that any reverts, should we decide they’re necessary, need to be handled carefully.

As I understand from his messages, Russ did so too. And I saw some edits of mappers not active here in the forum. So I guess we agree that a “complete” revert (i.e. including objects changed by other mappers) should **not **be done.

That’s a different type of issues, and I think it is not the fault of the FB team. Remember the imperative of the past: “if you do not know the reference number of the road, you must not tag it higher than unclassified”.
A year ago, another user approached me regarding my work in the South: “There are some roads tagged ‘tertiary’ that don’t have any ref. May I change these to ‘unclassified’?” I could fortunately convince him not to do so.
When I compare a map of Thailand with a map of a well-mapped country, it is clear to me that we are missing many secondary and tertiary roads - they are hidden among our too many unclassified roads. I upgraded several roads during my last edits. We surely need more local people knowing the situation on the ground.

And sometimes more contributions from our rather passive members. E.g. we have many GPS traces for Phetkasem road (#4) west of Chumphon, i.e other mappers have been there. A large section there is already a dual carriage highway on the ground- a “trunk”, but not on our map. I upgraded a minor section of it yesterday, and another large section between Kraburi and Ranong.

As for the other tagging issues of the FB team (mainly the unpaved roads/tracks), I can imagine to change the style file for my map. I could add some overlay to all the roads having an “import=yes” tag, or use a different line type which makes them non-routable or some other “hacks” like making them toll roads (toll road avoidance can be switched on/off on Garmin), or something like that. That would be a workaround for me.
But what about other users? They’ll download a map from somewhere, or use some application, where such hacks are not used (and if they were used, those users would not understand them, because of lacking background knowledge). I do not know what’s best to do…

Indeed - that’s the stage at which the person who was doing the revert would then have to decide what to do next. If it’s obvious that local mappers have been correcting the data then obviously the best thing to do is to leave the corrected data in place.