You are not logged in.

#1 2017-05-27 19:10:06

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Automated edits for name tags

Israel is multilingual and getting name tag consistency is tricky. This script helps with that. It copies name:en, name:he, name:ar to name and vice-versa in Israel.

Script pages:

The script is part of SafwatHalaby_bot. (Click for contact details, opt-out, other scripts, bot overview, etc.).

Last edited by SafwatHalaby (2017-10-20 11:35:10)

Offline

#2 2017-05-27 19:37:13

zstadler
Member
Registered: 2012-05-05
Posts: 343
Website

Re: Automated edits for name tags

Sound good.

However, is this a real issue? How many such elements exist?

Offline

#3 2017-05-27 19:57:27

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

70, excluding west bank or gaza (getting overpass statistics for Israel controlled territory in west bank is a bit tricky).

(edit: far more than 70 in the newer algorithm. See below)

Last edited by SwiftFast (2017-06-01 16:53:16)

Offline

#4 2017-05-27 19:59:17

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

It's not a serious issue at all but it's trivially solvable and the scripts are ready to go. Wouldn't have mentioned it except that some people don't like unmentioned bulk edits.

Offline

#5 2017-05-27 20:05:13

zstadler
Member
Registered: 2012-05-05
Posts: 343
Website

Re: Automated edits for name tags

If the problem/fix is not significant, IMHO it is not a. "bulk edit" that requires a discussion.

Offline

#6 2017-05-27 20:07:45

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

Ok. Will do it now. We have 538 including west bank and gaza (both IL and pl contolled territories). Perhaps I should ask their permission to also bulk-edit those areas too.

Edit: https://www.openstreetmap.org/changeset/49031645

Last edited by SwiftFast (2017-06-01 16:46:28)

Offline

#7 2017-06-01 12:04:12

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

The other way around appears to be a bigger issue: Things that have name but no name:he or name:ar. This would involve language detection. I will update the algorithm and report back.

Last edited by SwiftFast (2017-06-01 12:04:29)

Offline

#8 2017-06-01 15:54:48

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

Local test for Haifa and Krayot with a new algorithm: (not uploaded)

Total modifications: 3685
Total name:en to name: 32
Total name to name:en: 503
Total name:ar to name: 0
Total name to name:ar: 0
Total name:he to name: 0
Total name to name:he: 3150

Last edited by SwiftFast (2017-06-01 18:35:12)

Offline

#9 2017-06-01 16:00:41

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

New algorithm:

  • If name exists, deduce language, if name:lang doesn't exist for that language, copy name to name:lang. (If it exists, warn if the two names are different)

  • if only one name:lang exists, but name doesn't, copy name:lang to name. (or warn if name:lang's language isn't really lang)

name:lang is currently only for he,ar,en.

Offline

#10 2017-06-01 16:28:01

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

Offline test for Israel + PL (not uploaded)

Total modifications: 77440
Total name:en to name: 743
Total name to name:en: 26520
Total name:ar to name: 435 **
Total name to name:ar: 3693
Total name:he to name: 1*
Total name to name:he: 46048

* Probably low because many have both name:he and (name:ar or name:en or name:ru) so
 the script doesn't know which one to copy and skips. Plus because of my previous run.

** Probably high because my previous run didn't touch most of PL.

Last edited by SwiftFast (2017-06-01 16:45:55)

Offline

#11 2017-06-01 18:30:30

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

Live test on a very small scale: https://www.openstreetmap.org/changeset/49171684

Offline

#12 2017-06-02 16:39:33

wowik
Member
From: Zelenograd
Registered: 2009-09-29
Posts: 8,799

Re: Automated edits for name tags

Places (villages, towns and so on) in OSM sometimes have both a node and a boundary (way or relation).
It is rather important all tags which exists simultaneously on both objects should be equal.
So different software will use the same tag value regardless where they get the data

Here is some list of such errors.
Different name:en on node and boundary for example
http://wowik.000space.com/places/il/err.htm#err17

P.S. There are also a lot of non equal name (name:he) tags
https://www.openstreetmap.org/node/278470039 https://www.openstreetmap.org/way/92001545
https://www.openstreetmap.org/node/1068786999 https://www.openstreetmap.org/way/92001538
https://www.openstreetmap.org/node/278474036 https://www.openstreetmap.org/way/92001543
...

Last edited by wowik (2017-06-02 17:53:17)

Offline

#13 2017-06-02 19:53:53

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

Sounds like a good idea, but I think it should be on a separate script.

Offline

#14 2017-06-02 20:11:48

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

Actually, I don't think a script can do it. You'd need a human to manually tell which is the preferred name, right?

Edit: At least partial automation is possible: I could also obtain a list of the official names, and assign higher priority to them, when there's a conflict.

Last edited by SwiftFast (2017-06-02 20:22:20)

Offline

#15 2017-06-03 18:17:59

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

I just conducted a run on the northern district. Some problems cannot be auto fixed and need human help, they are mentioned in the changeset comment:

https://www.openstreetmap.org/changeset/49223571

Last edited by SwiftFast (2017-06-03 18:19:04)

Offline

#16 2017-06-03 18:48:53

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

Changes applied to Haifa District. I'm going to pause for a few days to see if some issues arise, before doing the remaining districts.

Haifa errors that need human help: https://www.openstreetmap.org/changeset/49225264

Offline

#17 2017-06-04 06:05:41

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

It appears we have two different space characters in names.

\u00A0 - Unbreakable Space (NBSP)
\u0020 - Space (SP)

Last edited by SwiftFast (2017-06-04 06:06:02)

Offline

#18 2017-06-06 10:53:51

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

@wowik Many places have node, landuse=residential, and also an administrative boundary from the MOIN import. So that's 3 places per ... place.

Offline

#19 2017-06-07 19:16:59

zstadler
Member
Registered: 2012-05-05
Posts: 343
Website

Re: Automated edits for name tags

Please note that massive updates are causing delays in the updates of the Israel Hiking and Biking maps.
Since 2-June it takes more than 36 hours to complete the maps updates, compared to 1-4 hours previously.

Also see https://www.facebook.com/groups/9949606 … 540697402/

Offline

#20 2017-06-07 19:49:55

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

Thanks for notifying me!

1. 99%+ of the changes were name to name:he copies. This should be a one-time load.

2. I updated 5 of the 7 Israel districts in one week. I could have spread this over weeks had I considered the load. I will update the remaining 2 districts in 1 week intervals or more, and will pause the updates for now until things propagate properly.

3. I am glad you reported this now. I was about to run it for the remaining districts.

Offline

#21 2017-06-08 17:35:15

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

I halted my changes for now, but is IHM down? And is it related?

https://www.facebook.com/groups/9949606 … 870288769/

Last edited by SwiftFast (2017-06-08 17:41:01)

Offline

#22 2017-06-14 12:07:38

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

Would this algorithm addition be acceptable?

- If name and name:he mismatch (and they're both Hebrew), see which one was the most recently updated, and update the other one accordingly.

Offline

#23 2017-06-15 14:34:15

Sanniu
Member
Registered: 2017-04-19
Posts: 40

Re: Automated edits for name tags

SwiftFast wrote:

Would this algorithm addition be acceptable?

- If name and name:he mismatch (and they're both Hebrew), see which one was the most recently updated, and update the other one accordingly.

Strongly against it - I found many streets where last edit was problematic. When you do automatic edit over problematic one it make very hard to spot an error and make harder to find the source of error.

Offline

#24 2017-07-12 16:47:43

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

I accidentally pushed an experiment. Will revert in a second.

Offline

#25 2017-07-12 17:31:43

SwiftFast
Member
Registered: 2017-04-10
Posts: 449
Website

Re: Automated edits for name tags

Reverted. Further explanation: The autofix code has been ready for a while, but didn't work due to a bug in the scripting plugin. Today, that bug was fixed, and I went ahead and tested the code, but I also accidentally uploaded.

Now that it works, we need to decide if it's desired.

Sanniu wrote:

Strongly against it - I found many streets where last edit was problematic. When you do automatic edit over problematic one it make very hard to spot an error and make harder to find the source of error.

I believe the benefits are greater than the drawbacks, but I am willing to stand corrected. Here is why:

  • Most people make good edits (I hope!), if so, the the last edit should usually be correct. We need to check out the diff to confirm this: #50233725 (which is now reverted)

  • Name/name:lang mismatch are very hard to manually detect because sometimes it's the good name that renders or shows up in editor descriptions. Because of that, sometimes these mistakes survive for months or even years. Example here. I argue it's often *easier* to see the bad edit when it appears in both tags. Quite often, a name mismatch can only be detected by someone actively looking for mismatches.

  • When I find a mismatch, I often trace the history to figure out which name is newer. That's mechanical and boring, and the bot can do it for me, faster.

  • I want the bot to "simulate" a single name tag. When you edit one tag, you are forced to edit the other whether it's a good or a bad edit. I believe this makes life easier.

Last edited by SwiftFast (2017-07-13 05:34:43)

Offline

Board footer

Powered by FluxBB