You are not logged in.

#1 2019-08-04 08:47:23

c2r
Member
Registered: 2019-08-04
Posts: 1

London data corrupton

At https://www.openstreetmap.org/#map=14/51.5510/-0.0173 I'm seeing what appears to be large scale corruption of map data, with a pattern of closely spaced vertical roads being shown.  It looks like it might be programaticaly created?  Is anyone looking at trying to revert it all?

Offline

#2 2019-08-04 08:59:19

wambacher
Member
From: Schlangenbad/Wambach, Germany
Registered: 2009-12-16
Posts: 16,441
Website

Re: London data corrupton

c2r wrote:

Is anyone looking at trying to revert it all?

I think, yes: https://www.openstreetmap.org/user/MacLondon/history

don't panic

walter, Germany

Offline

#3 2019-08-05 16:32:57

PaulRichardson
Member
Registered: 2019-08-05
Posts: 3

Re: London data corrupton

Hello,

This problem is still apparent within central London at zoom >=16 and maps become unusable as a result.

Has the revert been made but perhaps not updated yet or still cached etc?

Thanks,

Paul

Offline

#4 2019-08-05 17:32:34

SomeoneElse
Member
Registered: 2010-10-13
Posts: 1,218

Re: London data corrupton

(for the avoidance of any doubt) - yes, the oroblem was fixed around midday yesterday.  Depending on what map you're looking at, where it gets its data from and how any tiles are cached you might still see a problem, but it'll get fixed as soom as the problem tiles are refreshed (something that happens periodically as edits are made).

Offline

#5 2019-11-07 22:15:48

kreuzschnabel
Member
Registered: 2015-07-03
Posts: 6,167

Re: London data corrupton

Happened again today in https://www.openstreetmap.org/changeset/76764349, which has been fully reverted by me in https://www.openstreetmap.org/changeset/76773501.

Parts of Oslo were affected too this time. The nodes shifted coincided with nodes of building polygons already mapped in this area: https://www.openstreetmap.org/#map=17/53.13141/50.13179

How on earth could this happen?

--ks


Verwaltungsverfahrensgesetz § 44 (1): Ein Verwaltungsakt ist nichtig, soweit er an einem besonders schwerwiegenden Fehler leidet und dies bei verständiger Würdigung aller in Betracht kommenden Umstände offensichtlich ist.

Offline

#6 2019-11-07 22:49:24

SomeoneElse
Member
Registered: 2010-10-13
Posts: 1,218

Re: London data corrupton

kreuzschnabel wrote:

Happened again today in https://www.openstreetmap.org/changeset/76764349, which has been fully reverted by me in https://www.openstreetmap.org/changeset/76773501.

Firstly - thanks for noticing and fixing so quickly!

kreuzschnabel wrote:

Parts of Oslo were affected too this time. The nodes shifted coincided with nodes of building polygons already mapped in this area: https://www.openstreetmap.org/#map=17/53.13141/50.13179

How on earth could this happen?

I orginally suspected a "JOSM node drag" (of which we've had a few before) but as Richard on IRC noted, it's more likely a pre-existing .osm file (with numbers that matched existing nodes) was involved.  That might explain how it hit nodes in both London and Oslo from the same

Offline

#7 2019-11-07 23:36:34

kreuzschnabel
Member
Registered: 2015-07-03
Posts: 6,167

Re: London data corrupton

SomeoneElse wrote:

Firstly - thanks for noticing and fixing so quickly!

I couldn’t help it – I was working on the https://www.openstreetmap.org/#map=14/51.8789/9.9066 area which happens to be just on the beeline smile

Here’s a JOSM screenshot of the London roads’ eastern end:

Screenshot-20191107-194840.png

--ks

Last edited by kreuzschnabel (2019-11-07 23:40:55)


Verwaltungsverfahrensgesetz § 44 (1): Ein Verwaltungsakt ist nichtig, soweit er an einem besonders schwerwiegenden Fehler leidet und dies bei verständiger Würdigung aller in Betracht kommenden Umstände offensichtlich ist.

Offline

#8 2019-11-08 06:51:29

wowik
Member
From: Zelenograd
Registered: 2009-09-29
Posts: 8,512

Offline

#9 2019-11-08 13:06:54

alphensebezorger
Member
Registered: 2016-02-08
Posts: 156

Re: London data corrupton

SomeoneElse wrote:
kreuzschnabel wrote:

How on earth could this happen?

I orginally suspected a "JOSM node drag" (of which we've had a few before) but as Richard on IRC noted, it's more likely a pre-existing .osm file (with numbers that matched existing nodes) was involved.  That might explain how it hit nodes in both London and Oslo from the same

I happened to notice that both this recent changeset and the ones in August seem to only move nodes with id's roughly in the range between 106000 and 118000. Does this suggest a JOSM or OpenStreetMap API bug? Or is there a more likely explanation? Could the JOSM option to upload the data in chunks be related somehow?
Both users seem to use JOSM version 15238, although that could be faked I suppose.

Hopefully a better understanding of the causes of these changes can avoid future problems.

Offline

#10 2019-11-08 14:43:50

SimonPoole
Member
Registered: 2010-03-14
Posts: 1,847

Re: London data corrupton

alphensebezorger wrote:
SomeoneElse wrote:
kreuzschnabel wrote:

How on earth could this happen?

I orginally suspected a "JOSM node drag" (of which we've had a few before) but as Richard on IRC noted, it's more likely a pre-existing .osm file (with numbers that matched existing nodes) was involved.  That might explain how it hit nodes in both London and Oslo from the same

I happened to notice that both this recent changeset and the ones in August seem to only move nodes with id's roughly in the range between 106000 and 118000. Does this suggest a JOSM or OpenStreetMap API bug? Or is there a more likely explanation? Could the JOSM option to upload the data in chunks be related somehow?
Both users seem to use JOSM version 15238, although that could be faked I suppose.

Hopefully a better understanding of the causes of these changes can avoid future problems.


Likely

- JOSM or API bug
- broken stealth import script

unlikely

- actually moving the nodes.

Unluckily it is practically impossible to determine what went wrong without cooperation from the contributor in question (and even then there is no real way to do a proper root cause determination).

Simon

PS: SomeoneElse the thing is that the node ids are extremely low and essentially sequential, aka very very early data and very unlikely to be present in any accidental extract, so likely something is going very wrong with the placeholder ids in one of the three potential culprits.

Last edited by SimonPoole (2019-11-08 14:48:16)

Offline

#11 2019-11-08 14:54:27

SomeoneElse
Member
Registered: 2010-10-13
Posts: 1,218

Re: London data corrupton

alphensebezorger wrote:

I happened to notice that both this recent changeset and the ones in August seem to only move nodes with id's roughly in the range between 106000 and 118000.

I've just looked at the August one and that includes node IDs over 25,000,000 so I don't think it's the same sort of thing.  I suspect that that was a "normal" node drag that is all too easy to do in JOSM.

alphensebezorger wrote:

Does this suggest a JOSM or OpenStreetMap API bug?

I don't think it's an API bug - that pretty much just does what you tell it to, and if it "occasionally updated the wrong object" we'd know about that by now.

With JOSM you could argue the case that it should at least have displayed a suitable warning dialogue in front of the user, but of course we don't know that that didn't happen.  With a DWG hat on I often use JOSM to break referential integrity in the data when doing a revert, because "how well things are mapped" will still be better afterwards than before.  JOSM's conflict resolution dialogues are actaully enough of a pain for me to tend to use the perl revert scripts for larger reverts, and patch up the problems afterwards.  It's really a question to which there is no right answer.

alphensebezorger wrote:

Both users seem to use JOSM version 15238, although that could be faked I suppose.

Although faked version numbers of JOSM (and iD) are definitely a thing that spammers et al sometimes use, I have no reason to believe that either user here was using other than a "normal" copy of JOSM.

Offline

#12 2019-11-09 01:51:17

daganzdaanda
Member
Registered: 2013-04-25
Posts: 81

Re: London data corrupton

Seems like many of the node IDs involved were very old (6 digits) and the rest were mostly new and mostly in sequence.
Quite a lot of the old node IDs had been deleted/unused for a long time:
https://www.openstreetmap.org/node/106698/history
https://www.openstreetmap.org/node/108772/history
Does the API automatically "recycle" unused old node IDs?

Maybe someone who knows Russian could contact the mapper to ask what plugins or scripts s/he was using for adding the houses in Samara?

Offline

Board footer

Powered by FluxBB