Recently the Ministry of Transport has released a complete set of GTFS data for public transport in Israel.
From this data we could:
1) Add (via a script) a complete set of bus stops with names in Israel
2) Based on the stop names, add (manually) street/road names and addresses in less-mapped areas that are currently missing names/addresses
3) Add (manually I believe) bus route relations throughout Israel
There are two possible issues:
A) Legal: The data is obviously intended for public use, but maybe it does not meet exact requirements for OSM. If so, we might have to ask directly for permission to use it here.
B) Technical: somebody would have to write the import script(s).
Does anyone have ideas where to get started on A) or B)?
According to http://code.google.com/p/gtfs-osm-sync/, it in the transport agency’s best interest to do the synchronization, and therefore it should be their job. Since I do not see that happening on its own, it’s our job to get the ball rolling. The project page says that “it's highly recommended that GO_Sync users understand OpenStreetMap well and let other OpenStreetMap users know a big change is coming BEFORE any upload/revert.” If you need help with using GO_Sync, ask here and I will try to do my best (I do not know much more about it than you do, but we can learn together).
Thanks for the news, BTW, glad to hear about it!
Also, from http://www.tlv1.co.il/?p=5930&cpage=1#comment-16786; forgive my use of Hebrew:
בנוגע לתנאי שימוש – מותר להעתיק ולשכפל את המידע ולהפיצו כראות עיניכם. ידידי, יהונתן קלינגר מסביר:
ראשית, אתר דאטה.גוב אומר בצורה מפורשת כי 'המידע פתוח לשימוש חופשי ולא נדרש כל אישור מיוחד'; אבל, גם אם הדבר לא היה כך, העקרון הוא פשוט: סעיף 5 לחוק זכויות יוצרים קובע כי על 'עובדה או נתון' לא יחולו זכויות יוצרים. לכן, במספר מקרים בהם הועתקו מאגרי מידע כמו ספרי טלפונים, מודעות דרושים או מדריכים מקצועיים, פסק בית המשפט כי ההעתקה מותרת, וכי אין זכות יוצרים בליקוט (מעריב נ' אול יו ניד, קווי מידע נ' בל תקשורת). הסיבה לכך היא פשוטה: במאגר מידע אין את ה'יצירתיות' הדרושה לצורך מתן הגנה בזכויות יוצרים.
Which means, no special permission required for sync'ing the data to OSM.
Wow. Thats great news. Amazing news.
Before we import we need a confirmation that we are allowed to use that data in openstreetmap.
We are not allowed to just import! But as i know the goverment they will approve it.
Can somebody write a email in hebrew to them and ask for a approval?
No hebrew to help in that point.
The import should very easy with the gtfs osm sync.
Btw:the link to download the zip is not working at the moment.
Last edited by Mr_Israel (2012-05-26 08:54:08)
Now that's official, no need for confirmation:
מאת ניר הירשמן, data.gov.il:
29 במאי 2012 בשעה 8:19
אין שום צורך באישורים לשימוש במידע מדטה גוב איל. ישנם תנאי שימוש באתר שמתירים שימוש חופשי במידע.
And the download is working again (I think it's actually a significantly smaller file now, not sure what they changed).
Now, anyone want to give GO_Sync a try?
I just checked the file from today and have the tool is working and I could import the data if the Java tool GO_Sync would know Hebrew.
But it doesn't.
Maybe we should consider uploading the data manually. We have all we need to upload it, right?
I uploaded the data manually (using a modified version of csv2osm.pl and then JOSM).
I never knew how many stops there were. Blue dots all over the map. It seems that in some cases there are duplicates near each other, perhaps placed by different bus companies. Questionable whether we should edit/delete in that case.
The accuracy appears good - in most places I looked, there was <5m deviation from the roads as we mapped them.
I made a couple partially successful upload attempts before the final successful one. If anyone can delete these without affecting the successful changeset, that wouldn't hurt. Keep in mind that the original dataset includes 241 (IIRC) duplicates and perhaps this was intentional, to represent multiple adjacent stops (opposite directions?) or something similar.
The next task, for those who choose to accept it, is to give names to currently unnamed streets/places based on the name of the stop adjacent to them.
Do you mean that the partially successful attempts resulted in duplicated data?
Can it be fixed by just reverting the entire corresponding changesets?
After taking a look at the data, it appears it was duplicated, so we simply need to delete the old changesets.
Could somebody please revert changesets 12028202 and 12028208? I'm having plugin problems.
Could somebody please revert changesets 12028202 and 12028208? I'm having plugin problems.
What is the next step? How we going to merge old manually added stops to the new one?
In many places I see new stops appear between ways. How accurate was the imported location data?
I think very few of the manually added stops have any useful information other than the location. So I would recommend deleting them (manually after being checked, not by a script), and if necessary moving the GTFS stops to the location of the old manual stop. The GTFS stops include information, like the official stop name and reference number, which is potentially very valuable if coordinated with other data (for example, the route database).
As for GTFS location accuracy - the first few stops I looked at appeared very accurate. But since then I have seen less accurate ones, particularly outside urban areas. And the fact that there are often stops with identical names a significant distance away from each other implies that the accuracy may not be so high. Of course, manually entered stops can also inaccurate. So I don't know.
Last edited by eric22 (2012-06-29 17:49:07)
So, we doing merge and giving location priority to the old ones. Works for me.
eric22, can you somehow add fixme tag for all the new bus stops, so we can monitor manual validation status?
just checked the import and its amazing detailed.
Several street names could be fetched from this import.
Can be very very helpful in the future.
Great you found the time to do the job.
Beside that we should import more value from the data we got:
For example the bus numbers that are stopping on each stop.
And also the name of the bus company taking care of the stop.
Also the route ID could be from interest. Those kind of information are in the Israeli public data and we should really use them.
Currently I will not be able to create a bus route with the data we imported.
Its simply missing more information.
We should consider getting more data into OSM.
Is that possible somehow?
I just added (upload in progress) fixme tag for all (33051) imported bus stops.
Now, each time we see bus stop with fixme tag, we can check it, move it and/or merge it with existing one... until all stops will be approved.
We have a problem if we merge the bus stops. We would loose the "ref" number what will cause a overlapping for the next import.
We should not touch the location at the moment as we will have to import it again in the future.
What do you think?
Mr_Israel, no, we must only merge imported with existing and never two stops with different refs.
By the way, I see many stops which isn't exists at all. It's not a good idea to show them on the map and we need to find any solution to leave them in db but not show on rendering. maybe we can mark them as construction or proposed.
We could add the REF of the bus stop to a separated field.
Here some examples:
I simply don't want to loose any kind of information.
I'm expecting that they will fix the GTFS data in the future as it is currently not usefull for Google I think.
It would also mean for Google to have bus stops all over the street without marking the exact bus stop position.
I'd prefer to leave different stops as separate nodes. maybe slightly move them so the nodes will not have same position.
I have been working on the GTFS route data, and now I have a .osm file with a large number of ways, each representing a bus route. Here is a screenshot in JOSM:
The question is how to incorporate this data into OSM. A bus route in OSM consists of a relation which contains all the street segments the bus travels on. However, my data is not relations but ways. Furthermore, we presumably want to use our pre-existing ways, rather than the ways I'm uploading, to represent streets. (Though we could update our ways if necessary to reflect information in my ways.)
Right now it looks like I should upload all my data as-is to OSM. It will not show on the rendered map (the only tags in each way are ref,name,operator,source). But when we edit, we will see a number of ways overlaid on our streets. One by one, these can then be manually deleted and relations created to hold the equivalent information.
That is a labor-intensive process. Also, we need to find a reliable way of uploading this 300MB file (35MB when bzipped). My relatively new computer basically grinds to a halt when the file is opened in JOSM. But there must be some smaller script that can do the upload, perhaps by breaking the file into pieces.
Any thoughts, or alternative suggestions?
I would suggest to split it to smaller chunks (by operator/area/city/...).
Also, may be an option to create many small files, such as we can get small piece of data, open in in josm, and create normal relation without uploading temporary data and then deleting it.
eric, can you share some example? any single route?
I agree with yrtimiD to have smaller chunks and to try not to upload everything and then delete it.
It's better to do it slowly but with a focus to finish it together. You are welcome to add all routes to Israel by yourself :-)
You are preparing several OSM files with the data you have and split it to the smallest chunks we can do. Maybe bus route by bus route.
And then make it available to download. So everybody that wants to help needs to download the OSM file and add his name to the list at the route.
This way we would add route by route... will take a long time but better that "spaming" OSM with data that needs to be deleted afterwards.
Adding a bus route to osm is a very annoying process as a lot of streets needing to be cut into very small peaces to add the correct relation.
Doing it very slowly and with a system would really help.
I started separating the data by operator, and when looking more closely in JOSM, I realized a portion of the data is wrong. Not sure yet if the problem is in my script or their data. But I'm glad I noticed before uploading...