OpenStreetMap Forum

The Free Wiki World Map

You are not logged in.

#51 2018-04-04 07:17:48

zstadler
Member
Registered: 2012-05-05
Posts: 326
Website

Re: Automated incremental Bus Stop (GTFS) updates

Sounds good!

Offline

#52 2018-04-21 20:18:35

dsh4
Member
Registered: 2017-06-24
Posts: 62

Re: Automated incremental Bus Stop (GTFS) updates

zstadler wrote:
dsh4 wrote:

Thanks everyone for all the enlightening feedback.  It's clear there's no consensus on proceeding so I'll drop the matter (and seek some non-OSM-based solution to my original problem).

Can you elaborate on the original problem? Perhaps we could assist...

The discussion on routes was essentially about having a partial copy of MOT GTFS information within the OSM DB. The idea to link this data with other OSM data, such as roads, was dropped during the conversation. As such, I wonder what value could OSM bring to your original problem.

I thought that if imported the set of stops in each bus route into the OSM DB, then bus routing apps could be written that would work both in Israel and in other countries, without depending on the peculiarities of each country's upstream bus route formats.  If we don't do such an import, I'll keep using per-country public transport routing apps, that's all.

Cheers.

Offline

#53 2018-05-10 09:25:41

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

I apologize for the incremental updates delay. The code requires certain updates before the next run which I hadn't had the time to do. I will do this eventually, as soon as I can.

Offline

#54 2018-06-06 08:15:08

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

An incremental update has been applied with the new code: https://www.openstreetmap.org/changeset/59589851

The "description" tag has been refined, but the tag was intentionally not updated and will be updated in a separate changeset.

Offline

#55 2018-06-07 10:53:13

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

Version 2 is out. As a mapper, here is what you should know:

https://wiki.openstreetmap.org/wiki/Use … or_mappers

Version 2 now adds "addr:address, addr:number", a much cleaner description tag, and overriding user changes for certain tags. Additionally, the PTv2 is now used in addition to highway=bus_stop. Lastly, certain parts of the source code have been cleaned up.

No runs were performed with version 2 as of now. (The run described in the previous post was v1 + name overrides). But the source code can be found at the repository. I am making sure everything works as expected first. Also, this is a final opportunity for anyone to leave comments prior to running it.

Last edited by SafwatHalaby (2018-06-07 11:09:07)

Offline

#56 2018-06-08 16:03:56

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

Version 2 first run: #59668811, #59670297, #59670983

Offline

#57 2018-06-08 17:05:02

dsh4
Member
Registered: 2017-06-24
Posts: 62

Re: Automated incremental Bus Stop (GTFS) updates

@Safwat Appreciate the continued work on this. What's to be done if the name in MoT's database doesn't match the name on the ground? (The station's name is included in the black-ruled-yellow-background station sign and usually also in the in-station maps.) The wiki page says that the MoT database name wins, but I think we should map both names: the MoT name for interoperability with other services, the on-the-ground name due to https://wiki.openstreetmap.org/wiki/Goo … the_ground .

Offline

#58 2018-06-08 19:23:59

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

Hello dsh4,

I am glad you like my work smile

I wish you've brought up the issue earlier. This has been decided after several messages and after I waited a long time for comments. Nevertheless, I am open for change. Let us discuss this. As of now, I believe it's not worth the effort to handle this problem. Here are some reasons.

This change is based on past experience. Nearly all mapper bus stop edits are based on armchair fixes. They are not the "on the ground name"; the mapper sees a typo, or a stop having the name of a long gone road, and changes it. Even if the change makes sense, it is not OK to change it so long as the actual stop name has not changed on the sign and in the MOT data. It seems most mapping nowadays is armchair mapping, and that kind of mapping cannot possibly tell the physical name differs from the MOT name.

Experience also shows that we as a community are incapable of maintaining the stops. There are 27k stops that change rapidly every day. Maintenance requires a full time job for several people. The staleness of the stops prior to the introduction of the script, and the "gtfs:verified" tag, which I removed yesterday, are both testimony of this. That tag was effectively dead. We don't have the power to physically survey the stops and verify them and flip the gtfs:verified flag or update the stops. An "on_the_ground_name" tag would be the new stale "gtfs:verified", consuming 27KiB multiplied by ~10 or more with little use.

The physically printed bus stop name on the sign is increasingly becoming less important. Digital systems screens, which are becoming increasingly common right next to the stops, voice systems in buses, other apps and services, they all use the digital MOT name. Consequently, the digital names are highly maintained.

MOT has an active support E-mail. They are willing to change their data if it has mistakes. Highly confusing mistakes can be fixed on their part, and trivial mistakes too. Since the data is heavily relied on, perhaps they are even willing to physically change the on-the ground name for critical mistakes, though I have never tried this.

If this is a solely theoretical problem, then I think we ought to ignore it. A "third dataset" in addition to the two existing ones worsens everything - the code, mapper confusion, consumer confusion, the amount of warning logs I get, the debugging time required, the Israel map size, and more. However, if actual problems are arising because of this, then that's something to consider.

So I think it should be added only if it's really needed, and right now, I am not convinced it is. But I am open minded and would love to hear other opinions.

Offline

#59 2018-06-08 19:31:03

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

By the way, if there are specific examples of MOT-ground discrepancies, I would love to hear about them.

Offline

#60 2018-06-08 20:14:52

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

On further thought, I think there's a way that already works: Manually add different name tag such as alt_name whenever a ground name differs from the MOT name. This doesn't require 2 name tags per stop and doesn't require any script changes because the script would ignore alt_name or similar tags.

Adding the on-the-ground name in the "name" tag and the MOT in another does complicate things.

Last edited by SafwatHalaby (2018-06-08 20:19:45)

Offline

#61 2018-06-09 10:57:09

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

Would adding something like this to the Wiki be sufficient?

If the on the ground name is not the same as the MOT name, please:
1. add it to alt_name.
2. Optionally add a note that mentions this difference.
3. Optionally report the problem to MOT.

Last edited by SafwatHalaby (2018-06-09 10:57:29)

Offline

#62 2018-06-09 11:35:54

zstadler
Member
Registered: 2012-05-05
Posts: 326
Website

Re: Automated incremental Bus Stop (GTFS) updates

Good approach!
I'd like to suggest adding the appropriate email or web address to number 3 above.

Offline

#63 2018-06-10 14:55:45

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

The on-the-ground notes have  been added to the Wiki.

From now on, updates will usually run at Saturday evenings.

Most bus stops have an "addr:street" and an "addr:number" now. This covers most of Israel. I wonder how this affects navigation apps such as Osmand. Does it improve the search?

Offline

#64 2018-06-10 16:18:25

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

What would be the suitable OSM equiavlan of "רציף"? E.g.

55137,ת. רכבת כרמיאל/רציפים, רחוב:  עיר: כרמיאל רציף: 2   קומה:  ,32.923817,35.298353
55137,ת. רכבת כרמיאל/רציפים, רחוב:  עיר: כרמיאל רציף: 3   קומה:  ,32.923817,35.298353
55137,ת. רכבת כרמיאל/רציפים, רחוב:  עיר: כרמיאל רציף: 4   קומה:  ,32.923817,35.298353
55137,ת. רכבת כרמיאל/רציפים, רחוב:  עיר: כרמיאל רציף: 5   קומה:  ,32.923817,35.298353
63111,מסוף אורנית, רחוב:  עיר: שומרון רציף: 1   קומה:  ,32.107438,34.999197

Last edited by SafwatHalaby (2018-06-10 16:19:23)

Offline

#65 2018-06-10 19:56:09

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

I have made a stupid mistake. I added "addr:number", which is a nonexistent tag.

But what should be used instead? "addr:housenumber" doesn't seem right either.

https://wiki.openstreetmap.org/wiki/Key:addr

Offline

#66 2018-06-11 17:37:22

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

addr:number changed to addr:housenumber in https://www.openstreetmap.org/changeset/59727988

Offline

#67 2018-06-15 11:12:37

Sanniu
Member
Registered: 2017-04-19
Posts: 26

Re: Automated incremental Bus Stop (GTFS) updates

Osmose now jumps on "suspicious tag combination
highway together with addr:* "

Offline

#68 2018-06-15 11:15:25

anonymous_gushdan_mapper
Member
Registered: 2016-12-17
Posts: 32

Re: Automated incremental Bus Stop (GTFS) updates

I don't think the address should be saved in the bus stop node in OSM. This might cause duplicate addresses when there's already an address node (or tag) on a nearby building.

Offline

#69 2018-06-17 11:05:05

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

I apologize for the problematic tagging. I am not very familiar with address tagging and should have studied this further first.

What do you propose? It is possible to revert this and put "addr:housenumber" and "addr:street" in the description tag. But I would have preferred something which allows the clients like Osmand to use the addresses even in areas where no one has tagged the houses. That would be much better for usability.

Offline

#70 2018-06-17 11:14:35

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

Is it OK to keep "addr:street" and put the housenumber in the description?

Osmose now jumps on "suspicious tag combination
highway together with addr:* "

Isn't Osmose wrong here? (Assuming we keep addr:street only)

Offline

#71 2018-06-20 13:11:58

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

Since address duplication is dangerous and may cause unknown behavior in client address lookup, I have decided to move addr:housenumber to gtfs:addr:housenumber in the meantime, even before the discussion is finished. Please feel free to post your opinions on what should be done with addr:housenumber and addr:street.

Changesets: #60009323, #60010547

Last edited by SafwatHalaby (2018-06-20 13:58:29)

Offline

#72 2018-08-18 15:02:10

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

I'm investigating ways to increase the "bus factor". Currently, when fetching new MOT GTFS files, the script needs to compare them with the files fetched the previous run. This means that if I lose both my PC and my PC backup, or if I ever get hit by a bus, running the script properly would be a bit fiddly, because one would have to reconstruct the lost file.

I would like to make the script completely stateless locally. This should be technically possible: Fetch latest bus stop changeset by SafwatHalaby_bot, and use that as the "old gtfs file". This would make the script solely dependent on OSM servers, and not on any local hard drive.

This might be a major overwrite, so while we're at it, I would like to rewrite the script such that it does not depend on JOSM. This should make running it headless natural and pave the way for complete automation.

Last edited by SafwatHalaby (2018-08-18 15:30:56)

Offline

#73 2018-08-23 18:54:32

dsh4
Member
Registered: 2017-06-24
Posts: 62

Re: Automated incremental Bus Stop (GTFS) updates

Replying to an old point:

SafwatHalaby wrote:

What would be the suitable OSM equiavlan of "רציף"? E.g.

According to https://wiki.openstreetmap.org/wiki/Tag … 3Dplatform , ref=<platform number> should be used for that. I'm not sure whether that tag is supposed to go on the highway=bus_stop node or on something else.

Glad to hear of the planned bus factor improvements.

Offline

#74 2018-09-20 13:23:09

SafwatHalaby
Member
Registered: 2017-04-10
Posts: 442
Website

Re: Automated incremental Bus Stop (GTFS) updates

There is a semantics issue here. The thing called "platform" in the Wiki is essentially a bus stop, we already use the "ref" tag for bus stop numbers.

The thing called a "ratzeef/platform" in the MOT GTFS is better translated into a "terminal number", available only for some bus stops, mainly in central bus stations. So far I couldn't find a Wiki entry which discusses it.

Last edited by SafwatHalaby (2018-09-20 13:24:47)

Offline

#75 2018-09-23 13:47:17

anonymous_gushdan_mapper
Member
Registered: 2016-12-17
Posts: 32

Re: Automated incremental Bus Stop (GTFS) updates

I don't think you can call it "terminal number", the רציף/platform in the GTFS is a platform just like a train station platform.

A bus terminal is what we call מסוף אוטובוסים in Hebrew. The Tel Aviv Central Bus Station or Arlosoroff Terminal are bus terminals. They don't have numbers.

The "platform" in the GTFS file is there so that transit navigation apps will point users to the right platform for their bus. The GTFS file does not have the accurate coordinates for each platforms, so it uses the same coordinates for all and the same name for all, for example "Tel Aviv Central Bus Station 7th Floor/Platforms" ( https://www.openstreetmap.org/node/1803094757 ) is one node in OSM and one stop_code in the GTFS, but it has many lines in stops.txt with different stop_id for each, and the difference is just the stop_id and the platform number.

Considering this, I don't see how keeping the "platform number" is useful to OSM in any way, and I suggest you ignore this in your automated imports, and only have one OSM node per GTFS stop_code.

Offline

Board footer

Powered by FluxBB