OpenStreetMap Forum

The Free Wiki World Map

You are not logged in.

#26 2008-12-27 06:36:26

liosha
Member
From: Moscow
Registered: 2008-03-04
Posts: 8,446
Website

Re: osm2mp.pl: OSM -> Polish format converter

it is under GPL v2 in svn: http://code.google.com/p/osm2mp/

Offline

#27 2009-01-07 15:44:10

Trumbun
Member
Registered: 2008-12-25
Posts: 7

Re: osm2mp.pl: OSM -> Polish format converter

10x for your work liosha. I am u junior programer(Still studying) but in Java and c# would like to help one day.

Offline

#28 2009-01-14 15:11:47

Zwennie
Member
From: Leipzig
Registered: 2008-05-25
Posts: 46

Re: osm2mp.pl: OSM -> Polish format converter

Hi,

i tried your script and it works great but only with small areas. It worked with the state of saxony (a german state) and most other german states, but if i try Germany as a whole it crashes with some memory errors. Do you what i can do to fix this?
Ok, the resulting .mp file was also converted by cgps-mapper and MapSource shows it and can calculate routes and i'm able to transfer them with MapConverter onto my Mac.
But a problem arises for me:
If i create maps for every german state, i can show only one of the maps and also routing is possible inside one map only. How can i link or unify more maps to one bigger map? I know it should work with sendmap, but there i need img files while cgpsmapper created directories.

Thanks an best regards
Sven

Offline

#29 2009-01-14 15:41:33

Lambertus
Administrator
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: osm2mp.pl: OSM -> Polish format converter

I don't think our current tools are able to identify the roads that cross different tiles. So currently you could use GPSMapEdit to add the necessary information to the map images. This is described in the wiki.

Some interesting info on this


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#30 2009-01-14 16:58:15

nyem
Member
Registered: 2007-08-02
Posts: 20

Re: osm2mp.pl: OSM -> Polish format converter

That's an old posting before liosha's script was available lol

My solution to the routing across tiles problem is more of a hack, rather than a solution. I use osmosis to split my OSM data into tiles, and the connecting nodes of the splitted roads are manually tagged as "garmin:is_external_node=true" (they are currently not that many in OSM Malaysia). In liosha's script I added a conditional statement that will change the nod values for these connecting nodes (see http://forum.openstreetmap.org/viewtopi … 274#p13274)

My previous tests shows that it works when the tiles are square (the connecting nodes are all at tile borders). I'm not sure how it'll work if the tiles are irregular as in the shape of a state or country. Had a plan to experiment on this but haven't got started yet sad

It'd be great if osmosis or any tile splitter tool can automatically tag the connecting nodes as such, rather than doing a major overhaul of your current tools.

Offline

#31 2009-01-15 10:16:51

Lambertus
Administrator
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: osm2mp.pl: OSM -> Polish format converter

nyem wrote:

That's an old posting before liosha's script was available lol

Oh ok, but it's about the same method as described on the mkgmap/routing page.

My solution to the routing across tiles problem is more of a hack, rather than a solution. I use osmosis to split my OSM data into tiles, and the connecting nodes of the splitted roads are manually tagged as "garmin:is_external_node=true" (they are currently not that many in OSM Malaysia). In liosha's script I added a conditional statement that will change the nod values for these connecting nodes (see http://forum.openstreetmap.org/viewtopi … 274#p13274)

Tagging nodes as routing node manually is out of the question for me. It should be done automatically or it doesn't work cool

My previous tests shows that it works when the tiles are square (the connecting nodes are all at tile borders). I'm not sure how it'll work if the tiles are irregular as in the shape of a state or country. Had a plan to experiment on this but haven't got started yet sad

If the tools are good at detecting edge nodes then irregular shapes should also work imho.

It'd be great if osmosis or any tile splitter tool can automatically tag the connecting nodes as such, rather than doing a major overhaul of your current tools.

Yes, that would be awsome, but until now neither Osmosis as well as Osmcut are able to split tiles properly (let ways continue right onto the edge and insert dummy nodes on the edge). That's why every Garmin map created with OSM tools shows gaps in the roadnetwork near tileborders.


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#32 2009-01-15 11:18:16

nyem
Member
Registered: 2007-08-02
Posts: 20

Re: osm2mp.pl: OSM -> Polish format converter

Lambertus wrote:

Yes, that would be awsome, but until now neither Osmosis as well as Osmcut are able to split tiles properly (let ways continue right onto the edge and insert dummy nodes on the edge). That's why every Garmin map created with OSM tools shows gaps in the roadnetwork near tileborders.

Kevin Lindsey has an excellent intersection routines example done with javascript/svg. Could be useful in finding locations of the dummy nodes.

Offline

#33 2009-01-15 13:54:57

Lambertus
Administrator
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: osm2mp.pl: OSM -> Polish format converter

I think the problem with the current planet splitting tools like (osmosis and osmcut) is that it requires a lot of preprocessing or RAM to gather all the road polylines. Determining the intersection with a bbox when you have the polylines available is CPU intensive (but doable), but I guess the OSM XML format isn't very suitable to read polylines from.

I see three possible solutions:
Load all ways into memory (lots of RAM needed) and check each node to see to which way it belongs (CPU intensive), then save the road/node combination as polyline. Finally check each polyline agains every tile bbox to determine if crosses the border. Then add a node on the border position. Then split the way on the border node and save the resulting ways in the individual tile files. That is a lot of work.

Do as described above, but convert the planet file into a shape file first using osm2shp (dunno what how much RAM/CPU time that requires).

While writing this I forgot the third option I had in mind hmm

Edit: I cannot find any OSM to Shapefile converter in the SVN repository, so this leaves us at the first option...


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#34 2009-01-15 14:02:03

liosha
Member
From: Moscow
Registered: 2008-03-04
Posts: 8,446
Website

Re: osm2mp.pl: OSM -> Polish format converter

Hi all!

I've updated osm2mp in svn.
Now it supports maxspeed and access=private

Offline

#35 2009-01-16 21:08:13

Lambertus
Administrator
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: osm2mp.pl: OSM -> Polish format converter

nyem wrote:

My solution to the routing across tiles problem is more of a hack, rather than a solution. I use osmosis to split my OSM data into tiles, and the connecting nodes of the splitted roads are manually tagged as "garmin:is_external_node=true".

My previous tests shows that it works when the tiles are square (the connecting nodes are all at tile borders). I'm not sure how it'll work if the tiles are irregular as in the shape of a state or country. Had a plan to experiment on this but haven't got started yet sad

Do you mean you add new nodes to the way where it crosses the edge and tag those as "garmin:is_external_node=true", or do you tag the nearest existing nodes to the edge that way? If it is the 2nd option, how does the Garmin know which route nodes connect to each other?

It'd be great if osmosis or any tile splitter tool can automatically tag the connecting nodes as such, rather than doing a major overhaul of your current tools.

I'm looking at osmcut currently to do the job. Osmcut already knows which ways cross a tile edge so adding some more functions to enable routing shouldn't be very difficult.


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#36 2009-01-17 03:47:13

nyem
Member
Registered: 2007-08-02
Posts: 20

Re: osm2mp.pl: OSM -> Polish format converter

Lambertus wrote:

Do you mean you add new nodes to the way where it crosses the edge

Yes, the new nodes should be right at the edge of the tile.

Offline

#37 2009-01-17 10:18:15

Lambertus
Administrator
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: osm2mp.pl: OSM -> Polish format converter

nyem wrote:
Lambertus wrote:

Do you mean you add new nodes to the way where it crosses the edge

Yes, the new nodes should be right at the edge of the tile.

Ok, should the node have the same id in both tiles?


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#38 2009-01-17 16:32:13

nyem
Member
Registered: 2007-08-02
Posts: 20

Re: osm2mp.pl: OSM -> Polish format converter

Lambertus wrote:

Ok, should the node have the same id in both tiles?

Doesn't have to be the same id. Here's an example (in polish format) of a way that extends to another tile. The most important thing is that the coordinates should match up, and it is exactly at the edge of the tile.

Tile 1
[POLYLINE]
Type=0x01
EndLevel=4
Label=~[0x04]E2 NSE
DirIndicator=1
Data0=(3.0391936,101.7062182), ... (2.6274582,102)
RoadID=1299
RouteParams=6,4,1,0,0,0,0,0,0,0,0,0
Nod0=0,1046,0
Nod1=
...
Nod16=224,2649,1
[END]

Tile 2
[POLYLINE]
Type=0x01
EndLevel=4
Label=~[0x04]E2 NSE
DirIndicator=1
Data0=(2.6274582,102), ... (2.1308003,102.7412188)
RoadID=285
RouteParams=6,4,1,0,0,0,0,0,0,0,0,0
Nod0=0,1143,1
Nod1=56,462,0
Nod2=97,754,0
Nod3=215,375,0
Nod4=223,769,0
[END]

Some info from Rottweiller Auto-Routing guide (via http://cgpsmapper.com/route.htm)

Routing across tiles. If you are connecting several maps, or have split a larger map into smaller pieces (tiles), make sure that the nodes where the connections are to take place ACTUALLY will connect (by right clicking each connecting node, select Node Properties, and ensure that “This node is routing graph node” is checked. That’s all (except if you intent to provide turn restrictions at a certain intersection, in which case you can add those to each connecting node as well). The rest will be completed automatically.
The current (most recent versions of GPSMapEdit and cGPSMapper do a splendid job of linking maps – BUT… there is one very important caution – irregular-shaped map tiles (individual maps) MAY not link correctly. While I am not certain of the reason, this may be a function of the ‘backgrounds’ that cGPSMapper automatically adds to each tile during processing.

All of the above are if we are using the gpsmapedit method. It seems that maproute can detect and join the matching nodes automatically, which I think is the best way to work with irregular-shaped maps.

Offline

#39 2009-01-17 20:44:10

Lambertus
Administrator
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: osm2mp.pl: OSM -> Polish format converter

Saved by the bell for me big_smile

http://www.mkgmap.org.uk/page/tile-splitter

Now only Mkgmap needs to be adapted to set the correct edge routing information.


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#40 2009-01-17 20:51:25

Lambertus
Administrator
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: osm2mp.pl: OSM -> Polish format converter

The following is a copy from the IRC talk about this, contains some additional info:

#osm wrote:

[12:44]    <stever>    Lambertus: ping
[13:31]    <Lambertus>    stever: pong
[13:31]    <stever>    hi
[13:31]    <Lambertus>    i saw your low-key splitter announcement smile great!
[13:32]    <stever>    ha!
[13:32]    <stever>    have you tried it
[13:32]    <Lambertus>    i was just started to work on adapting osmcut to do something similar, but i'm glad you've made something like this
[13:32]    <Lambertus>    it seems like a difficult job
[13:34]    <Lambertus>    anyway, i was wondering if the edge node that you add to the ways that crosses tiles is added to both ways at the same coordinates?
[13:34]    <Lambertus>    no, i did not try it yet, but I certainly will try to do so this weekend
[13:35]    <Lambertus>    i have split the world in half using osmosis and will try to use your splitter to divde that further into small tiles
[13:35]    <stever>    its probably in the same place, but the splitter doesn't actually do that. It down to mkgmap to cut the line according to the bounding box
[13:36]    <Lambertus>    ah, ok, i misunderstood that apparently
[13:36]    <stever>    half the planet might be a bit ambitious
[13:37]    <Lambertus>    second question: does mkgmap set the special 'route' bit on those nodes?
[13:37]    <Lambertus>    well I have a machine with 3 or 4gb ram and 64bit linux to try it on
[13:37]    <stever>    no, but that is my plan.
[13:37]    <Lambertus>    cool!
[13:37]    <stever>    oh cool, well I'd be interested to know if it works
[13:38]    <Lambertus>    do you have any idea how splitter will work when it tries to write to thousands of files ?
[13:38]    <Lambertus>    i.e. do you open a file handle to all those files and keep them open all the time?
[13:39]    <Lambertus>    i mean, usually there is a limit on the number of files a process may have open at any given time
[13:39]    <stever>    yes it keeps a file open. I wouldn't expect thousands of files though
[13:40]    <stever>    the europe file I was working with is about 1/4 of the planet and its only about 70 files
[13:40]    <Lambertus>    well, currently I create tiles measuring 0.25 x 0.25 degrees, if i continue to do that, then it will be certainly thousands
[13:41]    <Lambertus>    ah, but those tiles will fill up a garmin unit without sd card very quickly smile
[13:41]    <chippy>    is the topology kept between slices?
[13:41]    <chippy>    I was looking for a .osm cutter the other day myself
[13:42]    <Lambertus>    i guess you've solved the problem of the 3.5 mb tile size at one point? Because in the early days of mkgmap I could not put the whole of rotterdam in a single tile
[13:42]    <chippy>    osmcut could only handle 1 degree tile sizes for my area
[13:42]    <stever>    yes that 3.5mb was a bug
[13:43]    <stever>    the splitter creates variable sized tiles so you don't get very many
[13:43]    <Lambertus>    yes i read that, that's amazing smile
[13:44]    <Lambertus>    smart little piece of software already wink
[13:44]    <stever>    chippy: what do you mean by 'is the topology kept'
[13:45]    <chippy>    so, huge shapefile -> osm. split the .osm file so we can upload in chunks, and it keeps things connected
[13:45]    <chippy>    or many .osm files -> routing, and the routing is unbroken over tile edges
[13:46]    <stever>    chippy: right. Nothing is lost that is required for Garmin maps, but if you want to use the OSM files for other purposes then there could be problems. I've not thought about that
[13:46]    <chippy>    hmm, actually with creation, and negative ids, I don't think the first would be possible
[13:47]    <chippy>    stever, yep
[13:48]    <stever>    chippy: I'm hoping that routing will work across tiles in garmin maps, but again for routing in other apps, it may or may not be be ok
[13:49]    <chippy>    stever, ok smile cool stuff
[13:57]    <Lambertus>    stever: the input osm file, can it be a gz file as well?
[13:57]    <stever>    yes
[13:57]    <Lambertus>    k
[15:15]    <stever>    Lambertus: it just puts its files in the directory where it was started. You've probably noticed that by now though smile
[16:00]    <Lambertus>    stever: yes, it took a while, but i see the files now
[16:00]    <ted_>    Lambertus: my machine has 4gb ram and osm2mp choked trying to convert all of Pennsylvania
[16:00]    * ted_    goes looking for stever's announcment
[16:00]    <stever>    how long? it is pretty slow
[16:01]    <ted_>    well, i shouldn't say choked, it finished
[16:01]    <Lambertus>    it's still running, but so it the gosmore update, so they're fighting for resources big_smile
[16:01]    <ted_>    it just ate up all my ram and swapped like crazy first
[16:01]    <Lambertus>    ted_: i'm seeing that too
[16:02]    <stever>    heh, they will probably both fail then...
[16:03]    <ted_>    stever: where's this announcement of yours?
[16:04]    <stever>    http://www.mkgmap.org.uk/page/17
[16:04]    <stever>    and http://www.mkgmap.org.uk/page/tile-splitter
[16:05]    <ted_>    stever: awesome!
[16:05]    <ted_>    i was just bemoaning this the other day when i tried to create a routable map of pennsylvania
[16:05]    <ted_>    and realized that it was too big
[16:06]    <stever>    ok, but note that we don't have routing between tiles just yet
[16:06]    <ted_>    ah, bummer
[16:06]    <ted_>    but you think it should be possible?
[16:09]    <stever>    yes, it should be possible
[16:09]    <ted_>    cool
[16:14]    <ted_>    stever: glad to hear you've run this on all of europe, so PA should be no problem
[16:16]    <stever>    how big is the PA osm file?
[16:16]    <ted_>    2.7Gb uncompressed
[16:23]    <ted_>    Total time 250s
[16:23]    <ted_>    not bad at all
[16:23]    <stever>    heh, how many files did it split it into
[16:24]    <ted_>    8
[16:24]    <ted_>    clearly i need to convert areas.list into something i can view graphically
[16:24]    <stever>    that would be cool
[16:25]    * ted_    ponders what format would work best
[17:06]    <Lambertus>    ted_: I'm thinking about reading it into OpenLayers and show the tiles as features, something like this: http://wiki.openstreetmap.org/wiki/Image:Garminmaps.jpg
[17:09]    <ted_>    i just wrote a really quick script to generate a .kml file from it
[17:09]    <ted_>    can OL handle KML?
[17:09]    <crschmidt>    Yes
[17:10]    <crschmidt>    (Wow, I have good timing)
[17:10]    <ted_>    hah
[17:10]    <crschmidt>    But the same liitations apply as with all vectors
[17:10]    <ted_>    these are just rectangles
[17:11]    <crschmidt>    But how many of them?
[17:11]    <ted_>    http://maps.google.com/maps?f=q&source= … e=UTF8&z=7
[17:11]    <ted_>    depends on how big the input data is
[17:12]    <ted_>    stever: ^^
[17:12]    <crschmidt>    ted_: I'm just saying, more than 500 is asking for trouble.
[17:12]    <ted_>    i think stever said all of europe split down into ~70
[17:12]    <ted_>    Lambertus, stever: http://mavra.perilith.com/~luser/convertarealist.txt
[17:12]    <ted_>    is the script
[17:13]    <ted_>    pretty simple
[17:13]    <stever>    ted_: cool
[17:16]    <ted_>    also, thanks to google maps, you got my home address for free in that first link
[17:20]    <crschmidt>    ted_: Did you see my comments last night re: browser data editor?
[17:21]    <ted_>    yeah, why do you say that?
[17:22]    <crschmidt>    I think that even the data browser goes too far, really; I think that the OSM website is really designed to be about getting the data *out*, not in.
[17:22]    <crschmidt>    Also, I think that a real data editor is a full fledged application in its own right -- the same way potlatch is
[17:22]    <crschmidt>    (I also think that Potlatch belongs outside of the OSM homepage)
[17:23]    <ted_>    ok
[17:23]    <ted_>    makes sense
[17:23]    <ted_>    but would be nice to have an openstreetbugs type of interface on the map, at least
[17:23]    * ted_    wanders off
[17:23]    <crschmidt>    Also, I think that tying an editor into the website is just limiting
[17:23]    * crschmidt    guesses this is really more appropriate for #osm-dev
[17:24]    <chippy>    +1 for browser based editor also
[17:24]    <chippy>    does JOSM have a way to view the history of an object?
[17:32]    <stever>    europe tiles shown using ted's script: http://maps.google.com/maps?f=q&source= … e=UTF8&z=3
[18:09]    <ted_>    stever: man, germany really gets tiled
[18:11]    <stever>    yes its interesting seeing the best mapped places


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#41 2009-01-18 12:51:27

nyem
Member
Registered: 2007-08-02
Posts: 20

Re: osm2mp.pl: OSM -> Polish format converter

Lambertus wrote:

Saved by the bell for me big_smile

http://www.mkgmap.org.uk/page/tile-splitter

Now only Mkgmap needs to be adapted to set the correct edge routing information.

Great stuff indeed tongue

Offline

#42 2009-01-27 20:51:02

japa-fi
Member
Registered: 2008-09-24
Posts: 44

Re: osm2mp.pl: OSM -> Polish format converter

Could someone point me to wiki article or similar describing the tiles you are referring to?
What defines the tile edges? I create map from southern half of finland. Looking at the link http://maps.google.com/maps?f=q&source= … e=UTF8&z=3
seems that my map contains 3 tiles. Is that the case?

Offline

#43 2009-01-28 09:37:08

Lambertus
Administrator
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: osm2mp.pl: OSM -> Polish format converter

The tilesplitter has some kind of tile optimisation algorithm which aparently produces these tile bboxes. Luckily you can define your own bboxes as well, placing the while of Finland in a single tile and have the algorithm determine the other bboxes.

This is my result when working with half the planet (Europe, Asia, Africa and Oceania)


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#44 2009-01-28 14:25:20

emj
Member
From: .se (59.3N17.99E) 0735969076
Registered: 2006-06-18
Posts: 949

Re: osm2mp.pl: OSM -> Polish format converter

Lambertus wrote:

This is my result when working with half the planet (Europe, Asia, Africa and Oceania)

That's nice, prolly a picture of the month..  And also fun seeing Google maps being slow.

Offline

#45 2009-02-03 20:03:02

liosha
Member
From: Moscow
Registered: 2008-03-04
Posts: 8,446
Website

Re: osm2mp.pl: OSM -> Polish format converter

Hi all!

I've added basic cropping functionality to osm2mp. It also creates boundary road nodes.
This time available in svn only.
Use --bbox <minlon,minlat,maxlon,maxlat> to specify boundary box.

Offline

#46 2009-02-04 08:27:31

weolvi
Member
Registered: 2008-12-17
Posts: 26

Re: osm2mp.pl: OSM -> Polish format converter

Does this mean that I can take one large file (say country.osm), split it into smaller tiles via multiple runs of osm2mp with several matching --bbox'es and routing will work across tile borders?

Offline

#47 2009-02-04 09:03:09

liosha
Member
From: Moscow
Registered: 2008-03-04
Posts: 8,446
Website

Re: osm2mp.pl: OSM -> Polish format converter

weolvi,
yes, you can. And routing will work  smile
You can also use tile splitter first, and then osm2mp with bboxes from areas.list
I've successfully built map of Russia such way: http://gpsmapsearch.com/osm/mp/
B6gS76i4tB.png

Note that turn restrictions disabled this time

Last edited by liosha (2009-02-04 09:22:31)

Offline

#48 2009-02-04 09:51:21

Lambertus
Administrator
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: osm2mp.pl: OSM -> Polish format converter

Awesome! It looks like we now have a full chain of software capable of generating routable Garmin maps without human intervention for the first time. big_smile

Very cool and good work liosha!

/me is going to implement an automated chain this weekend smile


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#49 2009-02-04 10:24:02

mtrax
Member
From: Canberra, Australia
Registered: 2008-05-20
Posts: 103

Re: osm2mp.pl: OSM -> Polish format converter

great work , I'm sure we are all waiting for this
thanks

Offline

#50 2009-02-05 18:48:02

cdavila
Member
Registered: 2009-02-05
Posts: 6

Re: osm2mp.pl: OSM -> Polish format converter

Always a convert a map from an osm file to mp with osm2mp I get some messages saying: Warning! Turn restriction RelID=XXX is not properly processed, but I check relations and seem to be correct (at least I don't find the error). Could someone check relations in this area [1] and tell me if there's something wrong or if it's a problem of osm2mp?
[1] http://www.openstreetmap.org/?lat=39.48 … rs=B000FTF

Offline

Board footer

Powered by FluxBB