Automated incremental Bus Stop (GTFS) updates

Thanks!

I’ll be running the script live in a week or so, unless there are objections or reported problems.

Edit: rephrased to reflect newer info.

The update is currently underway. Some manual intervention is required in some cases:

The removal of stops that are part of route relations causes JOSM conflicts that I’m manually resolving. For most routes I am simply removing the stop from the route.

Many routes are possibly out of date. No one is actively maintaining them.

There were other minor conflicts that were trivial to resolve (e.g. user adding stops as members of highways).

Update complete. This is a massive update because it’s the first one. From now on there will be weekly updates that are much smaller.

Changesets:

I’ve come across some duplicate bus stop nodes in Ramat Bet Shemesh.

http://overpass-turbo.eu/s/sLK

I have no idea if this is an isolated case.

Can upload issues cause this? The JOSM upload was interrupted multiple times, and I switched the chunk size in the midst of the upload at some point in an attempt to mitigate interruptions.

No duplicates appear for me when I run the script on a local copy of the older stops.

This is not an isolated case. There are 295 cases. Apparently the first chunk was uploaded twice or something like that. I will try cleaning this up.

Fixed in https://www.openstreetmap.org/changeset/53524487

If someone can explain how and when this occurs, I’d appreciate an explanation to avoid this in the future.

There are cases where people have glued bus stops to roads. The bot moved some of those stops, dragging the road along with them. I am manually fixing these cases one by one. See also this discussion: https://www.openstreetmap.org/changeset/49148167

I believe I fixed all of them. But this could have ended up worse. I’ll need to take stops glued into roads into account the next run.

http://www.openstreetmap.org/way/280293501 - here stop glued to the park…

Oh man, I was focused on roads.
I’ve fixed that one. Thanks!
I’ll make a more thorough cleanup this weekend or if I have the time before the weekend.

I am fixing this now. For anyone who has done fixes: Please make sure that after the fix, the stops remains precisely where the bot has moved it, the exact same coordinates.

Done. Did I miss anything?

I’ve done lots of duplicate cleanups. As of now we still have 545 bus stops that do not appear in GTFS, most of them are duplicates with the real GTFS stops meters nearby. Most of them are clustered in Jerusalem, Ber Sheva, and Tel Aviv district. If you are familiar with one of those areas, you can help by cleaning up duplicate stops. Most of them are marked with “fixme=Suspected duplicate stop. Flagged by SafwatHalaby_bot (flag-gtfs1).”

I propose the following tagging changes:

  • Add public_transport=platform, bus=yes, for compatibility with “public transport v2” and for future routes.

  • remove the gtfs:verified tag. Reasons:
    [list=*]

  • The Israel gtfs database is rather accurate most of the time

  • The update bot tolerates and respects user changes. If a user finds a wrong stop, they should simply update or delete it rather than fiddle with the tag.

[/*] [/list]

More about transport V2:


Two incremental updates were performed today. It appears 34 stops were updated by MOT in a single day.

Morning update - probably yesterday’s GTFS file: https://www.openstreetmap.org/changeset/53606095 (The numbers are misleading. Deletions include duplicate removals due to an earlier Overpass mistake that was fixed).
Evening update - today’s GTFS file: https://www.openstreetmap.org/changeset/53621985

The update had a minor positive unexpected effect on https://www.efobus.com/: When you scroll around their map, the markers need a few seconds to appear, but thanks to the synchronized blue square markers on the background map, you can see the position of the stops even before the click-able markers appear.

Here’s a visual map of osm-gtfs conflicts:

http://www.safwat.xyz/stops/ (rudimentary work in progress)

Hi

Thanks for a update.
I’ve found (really osmose found) duplicate stop here: http://www.openstreetmap.org/#map=18/30.01790/34.95113
Import bug or GTFS database has a glitch?

Thank you for the report. I’ve fixed this in changeset #55249140. 99 stop duplicates were removed.

This appears to be an upload issue and has happened before. It seems when JOSM fails an upload, things can get duplicated when I resume it. I’ll investigate this further.

Full list of duplicated stops:

Following this question on Facebook, how does one know when was the last Israeli gtfs import to OSM?

The import frequency is not clear from the Israeli Wiki.