OpenStreetMap Forum

ff5722

Member

Registered: 2016-05-20

Posts: 124

Bulk added notes

Someone is adding hundreds of notes that seem to be added in bulk on bridges in China, only saying ''мост" ('bridge') sometimes with the date the note was posted (e.g. "мост 10 апр"). The notes are useless, because at that location there is a bridge tagged correctly already. I closed a number of them manually (with a comment), but they are still being added. Later I contacted the DWG but they said there's nothing they could do really.

Thank you for writing to Data Working Group.  With anonymous notes, the DWG can hide offensive ones but other than that has the same rights to the notes API as everyone else.  The API is searchable, so for example

http://api.openstreetmap.org/api/0.6/no … 2&closed=0

will return the first 100 "bridge" ones (including legitimate notes as well).

Perhaps what happened here was that someone took the output from a QA tool or some other input and fed it into the anonymous notes API?  I'm not sure which QA tool would flag http://www.openstreetmap.org/note/941434 as a problem though.

I'd actually suggest asking the wider community about it - someone may actually think that they're being helpful by adding these, and they won't see feedback on the anonymous notes.  It might not be the Russian community, since "мост" is "bridge" in most Slavic languages, but of the Slavic-speaking, Cyrillic-writing communitys in OSM, the Russian one is by far the biggest.

Today i started getting notifications on notes of my own where most likely the same anonymous is just spamming letters to comment, presumably to trigger my email notifications.

http://www.openstreetmap.org/note/808687

http://www.openstreetmap.org/note/790967

So I wonder if anyone else had to deal with this 'anonymous' and his obsession for bridges, or if there truly is no way to bulk delete notes.

Last edited by ff5722 (2017-05-01 18:19:55)

---

http://openstreetmap.org/user/ff5722

ff5722

Member

Registered: 2016-05-20

Posts: 124

Re: Bulk added notes

I see the anonymous user is also spamming random letters on nearby notes by other users:

http://www.openstreetmap.org/note/400943

http://www.openstreetmap.org/note/396874

http://www.openstreetmap.org/note/411593

---

http://openstreetmap.org/user/ff5722

chnav

Member

From: Russia, mapping Kazakhstan

Registered: 2010-03-18

Posts: 3,303

Re: Bulk added notes

I have reviewed in JOSM and closed about 800 of those dummy notes in Central Asia (Kazakhstan, Uzbekistan, Kyrgizstan, Turkmenistan).
I left closing comments at first like "no information to map" or "already mapped", then realized my comments as useless as those notes are, and just close it.

https://www.openstreetmap.org/note/792263
https://www.openstreetmap.org/note/792264
https://www.openstreetmap.org/note/792265
https://www.openstreetmap.org/note/792266
https://www.openstreetmap.org/note/792267
https://www.openstreetmap.org/note/792269
https://www.openstreetmap.org/note/817402
https://www.openstreetmap.org/note/942555

Some notes are updated/commented anonymously few times.

Translation:
мост -> bridge
путепровод -> bridge
трасса -> road
обход -> bypass

Last edited by chnav (2017-05-02 09:15:00)

chnav

Member

From: Russia, mapping Kazakhstan

Registered: 2010-03-18

Posts: 3,303

Re: Bulk added notes

Something really goes wrong, this bot is getting insane
https://www.openstreetmap.org/note/903966
https://www.openstreetmap.org/note/881890

(added)
https://www.openstreetmap.org/note/978423
https://www.openstreetmap.org/note/942674

Last edited by chnav (2017-05-05 20:54:00)

muralito

Member

Registered: 2012-09-04

Posts: 1,918

Re: Bulk added notes

Contact teh DWG. http://wiki.osmfoundation.org/wiki/Data_Working_Group
Maybe they are already working on this issue.

escada

Moderator

Registered: 2011-08-13

Posts: 1,754

Re: Bulk added notes

I doubt it is a bot, there is #mapsme in the note. I assume those notes are made with the maps.me app.

chnav

Member

From: Russia, mapping Kazakhstan

Registered: 2010-03-18

Posts: 3,303

Re: Bulk added notes

I doubt it is a bot, there is #mapsme in the note. I assume those notes are made with the maps.me app.

maps.me app user created normal note, but later anonymous comments followed.

Last edited by chnav (2017-05-06 07:32:55)

ff5722

Member

Registered: 2016-05-20

Posts: 124

Re: Bulk added notes

Contact teh DWG. http://wiki.osmfoundation.org/wiki/Data_Working_Group
Maybe they are already working on this issue.

Haven't you read the first post? DWG can't do anything.

---

http://openstreetmap.org/user/ff5722

muralito

Member

Registered: 2012-09-04

Posts: 1,918

Re: Bulk added notes

Contact teh DWG. http://wiki.osmfoundation.org/wiki/Data_Working_Group
Maybe they are already working on this issue.

Haven't you read the first post? DWG can't do anything.

Yes I read it, but i didt't come to your conclusion.
They describe the current situation "the DWG can hide offensive ones but other than that has the same rights to the notes API as everyone else. "
but they didn't say that this policy can't change, being (for me) the most obvious change, rate limiting new anonymous notes or anonymous comments.

Last edited by muralito (2017-05-07 15:41:16)

SomeoneElse

Member

Registered: 2010-10-13

Posts: 1,601

Re: Bulk added notes

... the most obvious change, rate limiting new anonymous notes or anonymous comments.

That falls into the category of "patches welcome" I think? 

You'd probably need to persuade the people who look after the website that there's a problem with anonymous notes being created and contribute a technical fix to that problem.  It's not that difficult to review and close "problem" notes (hence me giving the API link for the "bridge" ones) - I've done something similar to that for various sorts of problem notes, including the "Free Parking" ones that infest the map because of a French app that people use to record "where I parked my car", and also for some sorts of MAPS.ME and speed limit ones.

muralito

Member

Registered: 2012-09-04

Posts: 1,918

Re: Bulk added notes

... the most obvious change, rate limiting new anonymous notes or anonymous comments.

That falls into the category of "patches welcome" I think? 

You'd probably need to persuade the people who look after the website that there's a problem with anonymous notes being created and contribute a technical fix to that problem.  It's not that difficult to review and close "problem" notes (hence me giving the API link for the "bridge" ones) - I've done something similar to that for various sorts of problem notes, including the "Free Parking" ones that infest the map because of a French app that people use to record "where I parked my car", and also for some sorts of MAPS.ME and speed limit ones.

It will become a technical problem if (and when) the policy changes and there is a need to enforce the new policy.
No only in OSM, but for every API service, rate limiting could be considered as a security issue.

The API link you provided does not return the IP address of who made the anonymous change.
- Does the database keep this info?
- Could the search API be extended to have the IP address as a search parameter?

escada

Moderator

Registered: 2011-08-13

Posts: 1,754

Re: Bulk added notes

uld the search API be extended to have the IP address as a search parameter?

I thought that for privacy concerns the IP addresses were stripped from all information available from OSM.

Martin Borsje

Moderator

From: Puth

Registered: 2011-10-22

Posts: 3,187

Re: Bulk added notes

I am not an IT expert, but I guess it is simple to program that:
1. anonymous notes without a reply will have a limited stay-alive time and then vaporise (one month proposed)
2. anonymous notes without a reply will appear with a different colour in OSM
3. notes with a reply, though not resolved also with a different colour
4. finally resolved notes in green that vaporise after a certain period (which is now the case already)

ligfietser

Member

Registered: 2008-10-09

Posts: 5,353

Website

Re: Bulk added notes

In the Netherlands we are flooded with notes that are on micro map scale. It is impossible to see the forest for the trees and mappers are starting to ignore them. So this system has to improve, it would be great help if there is a possibility to ignore notes (turn them grey or hide them) that are irrelevant for the mapper.

---

http://www.openfietsmap.nl

SubieLibero

Member

Registered: 2016-08-03

Posts: 328

Re: Bulk added notes

This is Winschoten, the Netherlands. All the red notes are filled with just a few words, no additional information let alone the source. This one for example just mentions 'A.M. minicars', I suppose it's a car dealer or something, but I'm not sure...
Anonymous is adding notes that first seem somewhat useful, but if you are keep adding them in the whole country with a speed that's just insane, it's hard for the community to solve them all by doing surveys or have a look on Mapillary. Also, you can't send a PM to anonymous, because he/she doesn't have an account.

This results in a situation that ligfietser already described:

It is impossible to see the forest for the trees and mappers are starting to ignore them.

If we really want to go to town, I think the suggestion of Matrin Bosje is a good one.

I am not an IT expert, but I guess it is simple to program that:
1. anonymous notes without a reply will have a limited stay-alive time and then vaporise (one month proposed)
2. anonymous notes without a reply will appear with a different colour in OSM
3. notes with a reply, though not resolved also with a different colour
4. finally resolved notes in green that vaporise after a certain period (which is now the case already)

But that may be too complicated.

Last edited by SubieLibero (2017-05-08 16:04:10)

ligfietser

Member

Registered: 2008-10-09

Posts: 5,353

Website

Re: Bulk added notes

For me it is not important WHO (anonymous or a registered user) entered a note, it is more of importance WHAT someone has marked.

---

http://www.openfietsmap.nl

SubieLibero

Member

Registered: 2016-08-03

Posts: 328

Re: Bulk added notes

For me it is not important WHO (anonymous or a registered user) entered a note, it is more of importance WHAT someone has marked.

You've got a point there. But that's also the hardest part of it all I think, how do we create some sort of system that can determine if the given information in a note is useful or not?

Last edited by SubieLibero (2017-05-08 09:52:42)

hadw

Member

Registered: 2014-09-02

Posts: 1,081

Re: Bulk added notes

I am not an IT expert, but I guess it is simple to program that:
1. anonymous notes without a reply will have a limited stay-alive time and then vaporise (one month proposed)
2. anonymous notes without a reply will appear with a different colour in OSM
3. notes with a reply, though not resolved also with a different colour
4. finally resolved notes in green that vaporise after a certain period (which is now the case already)

Notes don't actually vaporise; they are just hidden.  The request made by the web application specifies how recently a closed note has to have been closed for it to still be displayed.  If you change that parameter, you can see more, and if you know the node ID, you can retrieve very old notes.

As such, the vaporise part of item 1 may be rather more work to implement, as it may require changes to the map database, but the display colour coding is all being done in the web application, so could easily be changed in just the code that handles the web page.  I haven't looked, but his may even be code downloaded to the browser.

However, having had a comment made on it is not a reliable indication that a note is good, as non-anonymous users will comment on bad notes, requesting more information.

ligfietser

Member

Registered: 2008-10-09

Posts: 5,353

Website

Re: Bulk added notes

I'd like to see a webmap where all the notes appear and where you can mark on/off/grey/hide whatever you like. This information is stored on your computer so nobody else but you can decide what is important and what not. A slider with date is also handy, so you can choose to see notes that are less than x days old or between a chosen interval. Another thing that I would like is an export function to export selected notes as gpx file (I know such thing exist, but it contains all notes, not just a selection).

Last edited by ligfietser (2017-05-08 10:05:19)

---

http://www.openfietsmap.nl

hadw

Member

Registered: 2014-09-02

Posts: 1,081

Re: Bulk added notes

I'd like to see a webmap where all the notes appear and where you can mark on/off/grey/hide whatever you like. This information is stored on your computer so nobody else but you can decide what is important and what not. A slider with date is also handy, so you can choose to see notes that are less than x days old or between a chosen interval. Another thing that I would like is an export function to export selected notes as gpx file (I know such thing exist, but it contains all notes, not just a selection).

If you want to do something like that in a serious way, you should write code to run on own machine that downloads recently changed notes and constructs the full set of notes.  Refetching the whole history every time is likely to be wasteful of resources in the main map API servers.

SomeoneElse

Member

Registered: 2010-10-13

Posts: 1,601

Re: Bulk added notes

Another thing that I would like is an export function to export selected notes as gpx file (I know such thing exist, but it contains all notes, not just a selection).

Does https://github.com/SomeoneElseOSM/Notes01 do what you want?

ligfietser

Member

Registered: 2008-10-09

Posts: 5,353

Website

Re: Bulk added notes

Thanks Andy, I have used your tool before but I see you have added more options, I'll check it out.

---

http://www.openfietsmap.nl

muralito

Member

Registered: 2012-09-04

Posts: 1,918

Re: Bulk added notes

I'd like to see a webmap where all the notes appear and where you can mark on/off/grey/hide whatever you like. This information is stored on your computer so nobody else but you can decide what is important and what not.

Maybe It would be better, not only stored in your computer, as you can collaborate with others mappers sharing the changes you made to notes, and retrieving changes from others, and also seeing the same information from all your devices. The changes to notes made by yourself or by other mappers you trust would have more confidence than changes made by everyone else.
We already have the concept of "friends" in the website. maybe using the same list, or adding a "trusted" list.

From another point of view, as the problem is not the number of notes, but the usefulness of them, maybe a "wizard" style UI instead of only a textbox could help anonymous (and registered) users write more useful notes.

muralito

Member

Registered: 2012-09-04

Posts: 1,918

Re: Bulk added notes

For me it is not important WHO (anonymous or a registered user) entered a note, it is more of importance WHAT someone has marked.

You've got a point there. But that's also the hardest part of it all I think, how do we create some sort of system that can determine if the given information in a note is useful or not?

Not a guarantee but, you could help users with a wizard UI asking for the useful information, in the website but also in the apps that are adding notes.

I suppose people behind anonymous notes are not trying to hide themselves, they are collaborating like everyone else just without being registered users, so maybe asking for an contact email (but without requiring it) would be good. Based on that if an anonymous users made a note is because they have interest in some error being fixed, i suppose they want to answer comments to their notes.

hadw

Member

Registered: 2014-09-02

Posts: 1,081

Re: Bulk added notes

In my estimation, a lot of anonymous notes are:

personal mapping;
vanity (e.g. their own house number, with no other information);
advertising (from the name of the business through to a large screed of purple prose about the business).