"AI-Based Country Scale Road Import"

I just saw one of their changes.

https://www.openstreetmap.org/way/475128676#map=16/18.5432/99.4948&layers=C

The quality is absolutely low. And reading the comments and discussions they are not willing to work together with us, the community. They just follow their own - most likely commercial - interests.

Seriously. OSM Thailand is off better without their “help”.
Dear Facebook: please go away and play somewhere else.

Agreed. It’s merely a “track” in the middle of nowhere and connected to nothing. How they decided to map that way as a track instead of an unclassified and unpaved highway, is unknown. The changeset comment is not very helpful either. It’s the same old stuff again “#nsroadimport #thailand”. Useless.

How do we stop this train before it crashes and creates another big mess?

One answer is to make the DG data available to skilled OSM members. If we had access to their high-resolution data it could make a huge difference in our ability to add details to the area. But I’m afraid that won’t be possible. DigitalGlobe sold those rights to Facebook and they are not likely to give anybody free access. DG isn’t interested in making roads more accurate, only in making money to cover their investment. Can you blame them?

Oh well, after all that it’s just another day in the 'hood. Copyright is a double-edged sword. It protects the owner and impedes the spread of knowledge at the same time.

Hello Dave,

As we explained in e-mail, we know it’s not ideal, but we cannot share our imagery because of our agreement with DG. This said, we are going to share more about our process and quality checks via the import list.

Stephan,

Thank you for your feedback. We based our tagging on a few things;

  1. The commonly used tags in this area are highway=track. For Example https://www.openstreetmap.org/edit?editor=id&way=156615945#map=16/18.5368/99.4921
  2. This area looks like an agricultural area based on our recent high-resolution satellite imagery and Bing satellite imagery.
  3. This road will eventually connect to another road that is currently tagged highway=track https://www.openstreetmap.org/edit?editor=id&way=156618885#map=16/18.5399/99.4854
  4. We are also using the wiki to guide tagging. In this case http://wiki.openstreetmap.org/wiki/Tag:highway%3Dtrack

We stopped editing last week so gaps in roads are due to that using our tasking manager. Once we have been permitted by the DWG to resume, the holes will be filled.

Best,
Drish

All,

Mishari and I had some of the initial communication with Facebook about their import. We have raised our concerns and answered questions to the best of our knowledge. After what happened with their initial set of edits, they really do want to be extra careful not just with the data, but also with the community.

The way I see it, is that we have an extremely technically talented group at Facebook to our disposal, they have access to resources that we do not have (Like High-Res DigitalGlobe imagery). Instead of dismissing them as Stephan is doing, I’d much rather try to find ways to use those resources.

I don’t think Stephan is fair in saying “Go away and play somewhere else”, especially because he doesn’t seem to communicate his real reasons for saying so. If I understand Stephan correct, then even if Facebook get a 100% hit rate on the geometries and tags, he still wouldn’t want their data.

If I understand Stephan correctly, then he is afraid of the effect it will have on community growth. If all roads are correctly mapped, it gets less fun and more difficult for new mappers to get started. Just like we are seeing in the US with the Tiger import.

I can understand his concern, but we must also realized that one day the road network will be complete with or without the help from Facebook. Should we then all slow down a bit to avoid ever completing? I am personally adding data as fast as I can, I want OSM to be the best map out there and I think Facebook can help.

Stephan let me know if I have presented your opinions incorrectly.

I do not know what the correct way forward is, but I’d much rather work with Facebook on a solution than just dismissing them.

Hi All,

I want to second Johnny on this. The guys at FB have been quite nice and enthusiastic and I think we should discuss ways of incorporating the work done by FB’s team. I’m willing to say that what has transpired has been to a degree my fault as I didn’t have time to think through my feedback to them.

It doesn’t even have to be direct imports but some ideas I have are:

  • done in grids and validated by the community similar to hotosm
  • .osm files that can be validated and incorporated gradually.
  • generate an enhanced maplayer that can then be traced manually

We should consider this a good sign that OSM is so awesome now that FB wants to be involved. Now we just need to find out how we can work together.

Best regards
Mishari

Again a sign that you didn’t understand the fundamental principles of OSM as a community.

It’s not just a storage of free (as in beer) geodata.

It is a community gardening the data. This is one reason the data is of so high quality that even companies chose to use it as a data source.

You think there is a tiny group of “leaders” you have to convince then you have “permission” to do your things on OSM. This is not true.
Neither DWG, nor Johnny, Dave or Mishary have the power to grant you permissions of any sort.

OSM is a community. Your edits have to be considered conforming to our community standards.

Part of this process is transparency. You don’t silently develop something in your office and suddenly present your result. You have to include us in all your steps.

You plan to use “secret” data sources not available to others. This makes it harder to verify your edits. Handing out API keys to your edit interface is not a solution.

Yes, I might be hard by saying “go away”. But I believe this is reasonable as long as you chose no to cooperate in an open and transparent manner with the community. This doesn’t say I disagree with having roads mapped or support in completing OSM data. I disagree with the process you have implemented doing so.

Stephan

I have the feeling you understood the problem. It’s not about having data in or not. It’s about how it is done. It is a meta layer to the actual editing.
It is about finding a way on how to best integrate this with the community. To strengthen the community and grow it.

Referring to a previous comment by Johnny. Yes, even a 100% correct import would be something I see skeptical.

OSM had not yet enough time to grow to master the challenges of gardening the data on that completion level.

With the inflow of occasional mapper using maps.me we already see shortcoming in how the community can integrate this. We have to work on these challenges.

AI has the potential to be useful here. It could be a companion to newbies supporting them in getting started and avoiding common mistakes.

So yes: I see a potential here in multiple areas on how Facebook could support us here. But let me stress it again: It’s not about working in secrecy, it’s about openness and acceptance of community feedback. Just pretending to listen to the community and then continuing as before is not.

Another example that Facebook prefers to work in secret:
They gave a talk with a lot of buzzwords at SOTM us. But declined to have a video recording being made available so that the community can follow their ideas.

http://stateofthemap.us/2016/how-can-ai-help-us-make-maps/

@Facebook: Stop hiding. Open up your communication with the community and share the talk.

As others have stated, the Data Working Group doesn’t give you permission. The community will give you feedback on it. And you haven’t even taken the first step by emailing the imports mailing list!

No, of course not! But in that case there is still more things to map. Buildings (heights, names, postcodes), amenities, landuses, better and more accurate road geometries, road attributes (speed limits, surfaces, lanes, sidewalks, signage, stop signs, traffic lights, road widths, streetlights etc). OSM will never be finished!

It sounds like this Facebook team don’t want to contribute to OpenStreetMap. They appear to want a generic geo database web host. This project is open, we work in the open, we share things in the open, we all work together. If we all don’t have access to the digital globe imagery, then maybe it shouldn’t be used. If you don’t want to discuss your import in the open, and want just find the right high level executives to approve your import, then maybe you don’t really want to contribute to OpenStreetMap.

We shouldn’t throw out the open just because someone says they have nice imagery.

Please work in the open. Please share with us. Please share your imagery. Let’s all work on this project. Stop trying to hide yourself away, and refuse to share your toys. Things are much better if we work together.

As someone who has been active in India for the last 10 years, I can safely say that OSM is struggling here because very few see any point drawing millions of kms of roads across this vast country, most of which are already there on other maps. The majority of contributions of roads in India have come from European mappers, and maybe we should treat that as good as an import as well.

That said, having these roads on the map has acted as a seed for some basic data that makes the map be of use and allow the collection of more valuable data that can only come from local knowledge like streetnames and POIs. In India, we are already seeing that by the way of maps.me contributions from the mobile which is the primary way to access the internet, and would not be possible if there were no roads to begin with.

OSM will be a truly open and global project only when it can build sustainable communities on the ground maintaining the map as stephankn said. If we have access to high quality of open road geometries from FB, it seems like a huge opportunity to improve the quality of the map and boost the utility of OSM, which in turn brings in more editors. Its unreasonable to expect OSM communities in developing nations to form like they have in Europe over beers when basic needs like continuous electricity or safe drinking water is not met. Maybe different regions need different approaches, and this seems like a worthwhile effort if endorsed by the local mappers.

Yes!

Interesting commentary and feedback. I’m always angered when I run into copyright issues because they retard the growth of knowledge. Okay, that’s just the way the world works and I must accept that. I also see the vast amount of mapping that still needs to be done in my primary areas of interest, Thailand and Alaska. I am appreciative of any level of assistance with getting data into OSM because that is truly going to be the proof that the open source model works. When OSM can give Google real competition for offering the best map of the world, we’ll have proved that point. FB could be a valuable ally in helping us reach that goal. Like it or not, we depend on some pretty big outfits to be able to do what we do. Where would we be without Bing?

I have a question about licensing and EULAs. How it is that the FB group can extract information from the DG visual data if that information will be used in OSM, where there are virtually no limits to the further, subsequent uses of that data? Do FB and DG realize that once “their” derived data is in OSM, they have no control of it anymore? That someone may decide to create a map of wheelchair accessible bathrooms in Chiang Mai and turn around and sell that map to anybody with a few baht to spend?

I think we should take a step back, allow the group some latitude (pun intended), let them make a few mistakes, have them participate in this discussion, exchange ideas and advice, and see where it takes us. Only by discussing this important work here will we understand enough about it to feel good about giving the go ahead.

Hi all, I wanted to chime in both with a personal perspective, and one that represents the company I work for, DigitalGlobe.

On behalf of DigitalGlobe, I can confirm that Drishtie’s (Drish’s) statements are accurate and reflect the current agreement between Facebook and DigitalGlobe. Speaking for DigitalGlobe alone, we continue to work with both Microsoft and Mapbox to update satellite imagery available within their respective layers available in the OSM editors. In addition, we are looking at options to publish imagery for OSM editing similar to the imagery that is being used by Facebook.

WRT the copyright comment, I spoke about this at last year’s State of the Map US. In short, DigitalGlobe is a data licensing company. Our goal is to continue to develop, launch and operate really nice satellites so the world continues to benefit from detailed and accurate scientific observations from space. This includes the OpenStreetMap community who have been using DigitalGlobe data by way of Bing and Mapbox for many years. We just want to continue to do that for many more years. You can see my presentation here: https://www.youtube.com/watch?v=1dh60heskmo&index=40&list=PLqjPa29lMiE3eR-gK80irr3xdUiRbIMeg

Last, as an OSM contributor for 5 years, being involved with several State of the Maps both presenting and organizing, I welcome and encourage the work that Facebook is doing to support and improve OSM. We, as a community, must work together; and Facebook has clearly stated it’s willingness to work with us. Let’s continue to be respectful, and acknowledge the commitment Facebook has with OpenStreetMap.

Thanks, Kevin Bullock

Hello all,

Thank you again for all of this feedback. We joined this forum to work with you directly and support high-quality data with your expertise. We are always willing to share and answer any questions you have. My sole focus at Facebook is on OSM, so you will reach me and my team directly if you talk to us through these discussion posts or email us at osm@fb.com (please email if you need a quicker response). Our goal is your goal: to efficiently and accurately create maps powered by our community.

We look forward to continuing to share our derived data so the community can work together to create high-quality maps. We will be sharing sample data with the community next week via the import list. Following that, we’ll share the area we hope to start mapping in Thailand through this forum.

We are currently using the HOT Tasking Manager and iD tool and are working to share our editions/changes to these tools as well. Stay tuned.

Have a lovely weekend wherever you are!
Drish

Thanks to Kevin for clarifying the intentions and philosophy of DigitalGlobe. We are indebted to the people who produce the satellite imagery we use every day and without which we would be lost.

Drish, I wonder if the iD editor is the best one to use for this project. I am not all that familiar with it but the few times I have used it, I found it very limiting. The learning curve is steeper but I think using JOSM will save time and prevent problems in the end. It is very customizable. One can create and use “presets” for example, that make data entry more uniform. In an effort like yours, where many people are adding similar data, you don’t want some of them to tag an object one way while some others choose a different tagging scheme. Also, tagging an object with a preset requires only a couple of keystrokes no matter how many tags are involved.

It’s your project and your decision might need to weigh other important factors. Many of the most prolific contributors to OSM use Potlatch or iD so you are in fine company in that regard. Still, I feel that you might as well be using the most powerful and flexible OSM editor available and that is, unquestionably, JOSM.

Best Regards,

Dave

Another feature of JOSM is the inbuild validator which can detect common mistakes, like crossing ways, not connected way, bad tagging, etc. The Find and To-do plugin can help find and fix errors like this. In computer generated imports like this, you want to be able to automatically detect and fix problems to ensure a high quality of data.

The White Spots on the map of Thailand are still enormously big. Last December in Isaan, I was in places with nothing at all mapped within 10 km from my location. And let’s take a look at the contributors to the map of Thailand: most data were contributed by farangs.
We need some organisations helping us to complete some tasks. Roads, rivers, landuse can be seen on images. Place names, road numbers, shops, hotels, etc. can’t be detected there: we need people on site who will do that. And I hope they’ll more likely be willing to contribute when they see that the map is good (which it isn’t - also Google maps actually isn’t good as it does not differentiate between roads and tracks).
Although the examples shown by Stephan and Dave are very terrible, let’s try to integrate the FB team into the OSM community. With a slower approach, they could get feedback on small change sets, without causing too much disturbances with bad data. Also try to map connected items instead of singular segments. As I see from Dave’s comment, it could be helpful if the FB team tried to work in places where good and recent imagery is available also to us (instead of only to them).

Hello Dave and Rorym,

Thank you for the advice on tooling. As mentioned in our wiki page we do use both JOSM and iD. We totally agree that JOSM is superior for what we are trying to do and are all familiar with both tools. In fact, we started primarily with JOSM for the exact reasons you mentioned :slight_smile: In order to create a better internal process, our engineers have made changes to the iD editor to mirror the cool features JOSM has.

For example here are a few changes we have made to iD that help us be more efficient.

  • The ability to load/save locally generated .osm files much smoother and faster.
  • Customizing colors and highlighting specific point, lines or polygons. For example, we highlight roads so editors can quickly tell the difference between our edits versus what is currently in OSM.
  • Added conflict validation to iD so edits cannot be saved unless all conflicts are resolved.
  • Batch tagging for multiple selected roads
  • Automated suggestion for road connections. So we can easily connect roads in neighbouring tiles when working in the bounding box of a task from the tasking manager.

We are happy to share these and are working on ways to do this in a thoughtful way.

Best,
Drish

These sound great and make powerful editing features that were only available in JOSM more accessible to mappers. Looking forward to a publicly usable fork.

Again FB shows that they are not interested in cooperation with the community. Instead of presenting their ideas and working together with the community on improving the tooling they speak about creating their own toys and a fork.

Besides that:
When will the existing mess be cleaned up? Does Facebook expect the community to clean up after them?
I’m tempted to immediately revert uploads done by RVR007 and others. Removal of rice-field tracks are no big loss.

Duplicates for example in those changesets:
https://www.openstreetmap.org/changeset/46112980
https://www.openstreetmap.org/changeset/46117993

The number of duplicate geometry skyrocketed from non-existent to 10.000 Duplicated in 2017 thanks to technically bad executed Facebook imports.