You are not logged in.

#1 2019-08-13 13:38:04

woodpeck
Member
Registered: 2009-12-02
Posts: 885

Forest import

Hi,

I have just blocked user posiki_import_forest in https://www.openstreetmap.org/user_blocks/3103 with a request to please discuss their import here or in another channel with the Finnish community. I could not immediately find any such discussion, and an endeavour of such magnitude clearly mandates discussion. I might of course have overlooked a discussion that has been going on.

I have randomly checked some of the imported forest polygons against aerial imagery and found many to be not matching the imagery (but this could also be outdated imagery). Many of the imported polygons had totally artificial "wobbly" and over-noded outlines e.g. the southern edge of this

https://www.openstreetmap.org/way/71337 … 7/24.87802

is a very clean curve along the road, and not a 30-node wobbly line. The imported objects carry a source tag on the object rather than on the changeset.

I don't think that the methodology used here - documented here https://wiki.openstreetmap.org/wiki/Fin … 9_Creation - is sufficiently honed to roll it out for all of Finland but I would defer to the Finnish community for that.

Bye   
Frederik

Offline

#2 2019-08-13 14:10:00

posiki
Member
Registered: 2010-03-26
Posts: 214
Website

Re: Forest import

Hi Frederik,

Yes, we can have discussion inside Finnish community about this forest creation.

Rgs,

P

Offline

#3 2019-09-01 02:38:40

Zaltys
Member
Registered: 2019-09-01
Posts: 5

Re: Forest import

Good. This has done more than enough damage already.

It seems to work acceptably well in areas that have already been mapped almost perfectly. But in areas that remain unmapped, especially ones with missing buildings and roads, at most 10% seem to be drawn correctly.

Gardens and parks, scrubs, grassland, fields, often even pavement and buildings - all now marked as forest. This needs to be completely reverted, because manually fixing all of these messes is wasting too much time and it's sapping all my will to continue contributing to OSM. Not to mention that it has made the map useless for navigation in remote regions: undermapped is still helpful for getting around, but when there's nothing but incorrectly mapped forests everywhere... that's completely useless.

Last edited by Zaltys (2019-09-01 07:23:35)

Offline

#4 2019-09-01 04:26:42

Zaltys
Member
Registered: 2019-09-01
Posts: 5

Re: Forest import

Look at this, for example:
B9nYUWv.jpg

The blob on top left is clearly not forest. Neither is the one at bottom right. And the two drawn on the top of river are better tagged as scrub, not forest.

And from what I've seen, there's literally thousands of such blobs all over the map.

Offline

#5 2019-09-01 08:30:49

Zaltys
Member
Registered: 2019-09-01
Posts: 5

Re: Forest import

Then there's places like this:
o1t3ouU.jpg

I was just trying to map the area, but then ran into that. And trying to untangle it is taking forever.

Offline

#6 2019-09-02 20:00:53

Hadena
Member
Registered: 2018-09-08
Posts: 7

Re: Forest import

I also hope this change is reverted. It's good to add more landcover, but now everything turned into weirdly shaped forests.

Offline

#7 2019-09-08 14:48:54

posiki
Member
Registered: 2010-03-26
Posts: 214
Website

Re: Forest import

Hi!

We have also separated discussion in Finnish: https://forum.openstreetmap.org/viewtopic.php?id=67102.

Rgs,

P

Offline

#8 2019-09-08 14:50:20

posiki
Member
Registered: 2010-03-26
Posts: 214
Website

Re: Forest import

Zaltys wrote:

that's completely useless.

Ok, I agree that you disagree.

Just post / describe areas where I revert this automatic creation and I will fix those.

Rgs,

P

Offline

#9 2019-09-08 14:52:57

posiki
Member
Registered: 2010-03-26
Posts: 214
Website

Re: Forest import

Hadena wrote:

I also hope this change is reverted.

Hi,

I will gladly revert forests of those areas, which you will take care of later on. Please describe municipality and/or map sheets and I will clear those areas.

Rgs,

P

Offline

#10 2019-09-08 15:22:59

posiki
Member
Registered: 2010-03-26
Posts: 214
Website

Re: Forest import

woodpeck wrote:

I would defer to the Finnish community for that.

So far, 8 people has commented this Forest creation task. 2 opposite, 6 favor (+ me).

If we are following person / vote -method, over 75% of Finnish community members (who have said their opinion) agree that this good thing to do.

Rgs,

P

Offline

#11 2019-09-10 18:40:52

Zaltys
Member
Registered: 2019-09-01
Posts: 5

Re: Forest import

posiki wrote:
Zaltys wrote:

that's completely useless.

Ok, I agree that you disagree.

Just post / describe areas where I revert this automatic creation and I will fix those.

Far too many to list.
Would be easier to list the places that don't need to be fixed. I've already manually fixed or deleted literally hundreds if not thousands of these messes, and there's countless more wherever I scroll the map.

J5FX99Q.jpg
ZWDR21w.jpg
fGLzp7F.jpg(Funny how it drew a forest in the only spot where there are no trees.)

The whole country is like that now, except for cities and residential areas which were already zoned. This project may have mapped the missing forests (though with weird curvy lines), but it made a complete mess of places where people are likely to use maps: residential areas.

Did you actually test this at all before running it? Did you check the results to see if it's working correctly? If this had been limited to larger blobs and had less smoothing, it would've likely worked quite nicely.

Last edited by Zaltys (2019-09-13 03:56:49)

Offline

#12 2019-09-17 16:48:25

posiki
Member
Registered: 2010-03-26
Posts: 214
Website

Re: Forest import

Zaltys wrote:

Did you actually test this at all before running it? Did you check the results to see if it's working correctly? If this had been limited to larger blobs and had less smoothing, it would've likely worked quite nicely.

Yes, I tested and change code heavily during the development. Did you read wiki page where I describe process?

I'm aware that there are problems and I agree that manual mapping will be best way to do this. But unfortunately we don't have enough mapper here in Finland.

Some of the problems are coming from data updates. I presume background imageries are newest from National Land Survey of Finland. If somebody would been create landuse polygon (like residential area),  there shouldn't be any forests.

In other discussion (https://forum.openstreetmap.org/viewtopic.php?id=67102), I mention some reasons why to do this. I translate those to English:

Why to make this forest polygons?
- Corine data import has not been finalized and Corine data is not very accurate.
- Forests will not been mapped in Finland very soon (very few mappers)
- Automatic process is quite fast and good enough
- Finnish OSM is pretty bad for certain use cases, because of missing forests (and over 70% of total area is forest)

Why not to make it? (+my feedback)
- All OSM features have to map by humans. How about imports (like from NLS.fi)?
- Quality is perfect, humans will make better quality. Agree, but we don't have anough mappers
- No need forests, only visual candu. Well, Finland is living from forests (or at least in forests ;-)

I don't have clear idea how we can fix Corine Import forests: there is a lot of errors.


P

Offline

#13 2019-09-20 01:34:33

Zaltys
Member
Registered: 2019-09-01
Posts: 5

Re: Forest import

posiki wrote:

I'm aware that there are problems and I agree that manual mapping will be best way to do this. But unfortunately we don't have enough mapper here in Finland.

Well, you have one less now. No wonder there aren't more mappers. If this kind of completely nonsensical mapping gets approved in OSM, then it's absolutely not worth contributing to - nor is it worth using.

Last edited by Zaltys (2019-09-20 01:36:06)

Offline

#14 2019-09-20 19:12:23

Hadena
Member
Registered: 2018-09-08
Posts: 7

Re: Forest import

The problem is, that there are just too many places where your algorithm did not work out well. All those weird curves, its not like it really is. I fully agree that the Corine import did not work well either (and was stopped). I rather have a map with missing forests than one with forest on the wrong places in complicated patterns that are very hard to correct, if somebody decides to. The fact that there are few mappers also means that it is even less likely that wrongly placed forests will get corrected. One of the main tips for mappers is to map correctly, because you cannot assume that someone else will correct it ever. If Finland consists of 70% forest, wouldn't it perhaps not be better to add first other types of landuse? In the national map, agricultural fields are indicated. Are these data not available for import?

Offline

#15 2019-09-20 21:35:11

JRA
Member
Registered: 2007-12-17
Posts: 668

Re: Forest import

I would consider to use the forest mask from the Forest Centre of Finland as an alternative source for the import. People taking part on this discussion probably know that in Finland even the topographic database of the National Land Survey of Finland does not have forests. Forest it the default and everything that is not classified to be something else is supposed to be forest.

The documentation of this product is in https://www.metsakeskus.fi/sites/defaul … amaski.pdf. This is my translation of the process:

Parcels with an area of max. 1.5 hectares and which contain buildings have been removed from the total area.
If parcel is bigger than 1.5 hectares an area of 0.5 ha has been removed around the buildings.
Features which are something else than forest in the topographic database of NLS has been removed (fields, buildup area, lakes etc.) Railway areas has been removed as cadastral parcels. Roads and power lines, if they are not formed into cadastral parcels, has been removed by the width according to the corresponding feature class. 

The forest mask is certainly not perfect but the process feels very reasonable.

The Finnish description of the dataset follows:

Kokonaisalasta on poistettu muun muassa kokonaan palstat, joiden
pinta-ala on enintään 1,5 hehtaaria ja joilla on rakennuksia.  Suu-
remmilla palstoilla rakennusten ympäriltä on poistettu 0,5 hehtaaria.
Maastotietokannan aluemaiset kohteet, jotka ovat metsätalouskäytön
ulkopuolella (esimerkiksi vesistöt, pellot, taajama-alueet), on poistet-
tu sellaisenaan kokonaan. Rautatiealueet on poistettu kiinteistöinä.
Kiinteistöiksi erottamattomat tie- ja sähkölinja-alueet on poistettu
luokan perusteella määritellyn leveyden mukaan. Lisäksi aineistosta
on poistettu pienet ilman omistajatietoa olevat palstat sekä geomet-
riaprosesseista syntyneet pienet kaistaleet. 

-Jukka Rahkonen-

Offline

#16 2019-09-20 22:31:43

JRA
Member
Registered: 2007-12-17
Posts: 668

Re: Forest import

Another option could be to use these vector datasets https://aineistot.metsaan.fi/avoinmetsa … /Maakunta/ but I am not sure if the license is compatible with OSM. And if I look at the data of my own forests I think it is all too detailed for OSM and most of the parcels should be merged before import.

Last edited by JRA (2019-09-20 22:32:03)

Offline

#17 2019-09-29 16:24:38

posiki
Member
Registered: 2010-03-26
Posts: 214
Website

Re: Forest import

Zaltys wrote:

Well, you have one less now..

I'm sorry to hear this.

P

Offline

#18 2019-09-29 16:27:00

posiki
Member
Registered: 2010-03-26
Posts: 214
Website

Re: Forest import

Hadena wrote:

In the national map, agricultural fields are indicated. Are these data not available for import?

Yes, there are fields in National Land Survey Topo data and I presume those can be imported quite easily. However, there shoudl be some kind of filtering: OSM could have newer information (like residential area in former agricultural areas.

P

Offline

#19 2019-09-29 16:32:33

posiki
Member
Registered: 2010-03-26
Posts: 214
Website

Re: Forest import

JRA wrote:

Another option could be to use these vector datasets https://aineistot.metsaan.fi/avoinmetsa … /Maakunta/ but I am not sure if the license is compatible with OSM.

Those are licensed CC-BY-4.0 and I think it is compatible with OSM.

https://www.metsaan.fi/yleistietoa-avoi … satiedosta

P

Offline

#20 2019-09-29 22:03:11

woodpeck
Member
Registered: 2009-12-02
Posts: 885

Re: Forest import

posiki wrote:
woodpeck wrote:

I would defer to the Finnish community for that.

So far, 8 people has commented this Forest creation task. 2 opposite, 6 favor (+ me).

If we are following person / vote -method, over 75% of Finnish community members (who have said their opinion) agree that this good thing to do.

I'm afraid this is not just about numbers.

You cannot import auto-generated low-quality data into OSM full stop. If 100% of the Finnish community want auto-generated low-quality data then they need to find themselves another project, perhaps "OpenAutoImportMap" or so.

The Finnish community will never be able to manually rectify this huge amount of data; future members of the Finnish community will despair and run away because it is much more work trying to fix your data than to create it from scratch.

Your attitude towards Zaltys - essentially "if you criticize my import then you have to commit to tracing the forest manually in the areas where you object" is not acceptable.

An import can be good if it forms a basis for the community to work on; if there is a commitment and a plan to make it good. But your import is a capitulation: "We'll never be able to map this so we might as well import it so that the map at least looks nice and green". But with this attitude you are gravely harming OSM's prospects in Finland.

What you can do is this:

Run your algorithm to prepare parcels of auto-generated forest, and then find mappers who are willing to take such a parcel, manually verify and correct it against imagery, and then upload. If nobody is interested in doing it in some areas, then those areas remain without forest until someone does. (And if mappers just take your data and upload without fixing obvious problems, their contributions are reverted for foul play.) This will be a slow process but at least one that generates quality data.

What you can also do is this:

Set up an openstreetmap.fi tile server that generates map tiles which draw forests based on your auto-generated data. Then if anyone wants a map that looks nice and green at the expense of correctness, they can go to openstreetmap.fi, without having to pollute OpenStreetMap with "guessed" forest data.

What you can not do is:

Leave your buggy data in OSM and/or upload more of the same.

Best
Frederik

Offline

#21 2019-10-05 15:06:37

posiki
Member
Registered: 2010-03-26
Posts: 214
Website

Re: Forest import

woodpeck wrote:

What you can not do is:

Leave your buggy data in OSM and/or upload more of the same.

Ok.

Any suggestion how to remove all those forest plots? I saw some perl scripts done by you, but I don't know Perl.

Rgs,

P

Offline

#22 2019-11-14 21:27:03

Hadena
Member
Registered: 2018-09-08
Posts: 7

Re: Forest import

So when are those forest plots removed? It is now nearly impossible to draw new roads, tracks, paths (in Potlatch) because they get connected to the borders of all those forest plots where a very thin line between them is left open (where the finnish national map shows a path).

Offline

#23 2019-11-15 08:23:45

PHerison
Member
From: Rhein-Main
Registered: 2008-04-04
Posts: 1,506

Re: Forest import

You should consider using JOSM as editor for OSM. In JOSM you can disable "snap to line" by holding Crtl while placing a node.

Offline

#24 Today 14:46:41

posiki
Member
Registered: 2010-03-26
Posts: 214
Website

Re: Forest import

Hadena wrote:

So when are those forest plots removed?

I have been travelling quite recently. I haven't got any feedback/help for deleting processes. So it seem obvious that I have to plan and excute it myself (and I'm willing to do it). However, it will take a little bit longer.

I have idea just remove all those forest plots which are imported by me. Unless someone has been edited those.

P

Offline

Board footer

Powered by FluxBB