Complete OSM conversion

Hello,

I am a complete beginner. Maybe I should have posted this to the general list but hope you can help.

I plan to work on the OSM for Niokolo-Koba National Park in south-east Senegal (there are some OSM data but not part of the Senegal OSM project).

I have successfully downloaded the existing data into JOSM from the OSM server and saved as an OSM file called Niokolo-Koba.osm (the file is good - I can load it back into JOSM without any problem).

Now I am trying to generate files for i) Garmin eTrex 30 and ii) Garmin BaseCamp.

For these two requests I have issued the following commands in a Ubuntu 12.04 LTS terminal:

i)

java -jar mkgmap-r3336/mkgmap.jar -n 70000000 --description=Niokolo-Koba --country-name=Senegal --country-abbr=SEN --family-id=2001 --product-id=1 --gmapsupp Niokolo-Koba.osm

ii)

java -jar mkgmap-r3336/mkgmap.jar -n 70000000 --description=Niokolo-Koba --country-name=Senegal --country-abbr=SEN --family-id=2001 --product-id=1 --tdbfile Niokolo-Koba.osm

I can load gmapsupp.img file from i) into my Garmin (microSD card) and can load the files from ii) into BaseCamp with MapSetToolkit (under Windows XP) but in both cases I only get a small part of my OSM map (extreme western edge). I guess this is because there is only one img file instead of several?? How can I get the whole OSM map converted?

Thanks and best regards,
John

Hi,

I think JOSM downloads only vary small part of OSM. To get a big map I suggest:

To get best map for your purposes, you should tune mkgmap style, documentation is included in mkgmap package.

Or you can try any ready map based on OSM:
http://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/Download

Including my map of Africa:
http://www.gmaptool.eu/en/content/africa-osm-topo

Thanks popej,

It is true that OSM server only downloaded small pieces, but I downloaded several adjacent views and got what appears to be a well formed OSM file of the area I am interested in (when I open it in JOSM I get back all the display information on the designated zone). Thus I would really prefer to use my OSM may than to start again generating a file according to one of your options.

I believe that my problem is not using the correct parameters for mkgmap, but the documentation on the website http://www.mkgmap.org.uk/ is cryptic for me. I think that I should probably be getting several numbered img files but I get only one numbered 70000000.img according to what I asked for on the command line (if I left the default 63240001.img file it would duplicate a segment number in other maps in my system).

I would greatly appreciate further advice on the correct mkgmap command(s) for my case, else I will regenerate an OSM file as you suggest and try again (but I am almost sure that I will get the same result).

Best regards,
John

Mkgmap creates a single img (tile) from each input data file. If data is too big, you get an error. Maybe your data really contains only a part of requested area? Or maybe your data contains bounding box, which crops area for mkgmap?

Splitter is the program, which can divide big OSM data file into multiple parts for mkgmap.

Thanks popej,

My osm file covers about 150x100 km but does not have much detail (4.8 MB).

I ran splinter on it with the following command:

java -Xmx2000m -jar splitter-r412/splitter.jar --keep-complete=true Niokolo-Koba.osm > splitter.log

I got a log file saying that only one tile was generated, as well as the following output files: areas.list, areas.poly, densities-out.txt, template.args and 63240001.osm.pbf .

I then ran mkgmap on the pbf file as follows:

java -jar mkgmap-r3336/mkgmap.jar -n 70000000 --description=Niokolo-Koba --country-name=Senegal --country-abbr=SEN --family-id=2001 --product-id=1 --tdbfile 63240001.osm.pbf

I got output files with the same names as before running mkgmap directly on the osm file: osmmap.img, 70000000.img, ovm_70000000.img, osmmap.tdb. As before I installed the map in BaseCamp with MapSetToolkit (under Windows XP), and this time got a more complete map with nearly (or all?) of the area of the osm file, showing the rivers, but missing the roads, the park boundry and most of the place names. In MapSetToolkit I specified the tdb file as osmmap.tdb and the overview map as osmmap.img. I did not see a place to specify ovm_70000000.img .

I’m getting closer, could you please advise?

Best regards,
John

ovm_* is a temporary file, which is used by mkgmap to create overview img, probably osmmap.img in your case. MapSetToolKit finds all img by analyzing tdb (so I think).

I would try an alternative way of downloading OSM data, if not geofabik.de, than for example:
http://download.bbbike.org/osm/

Dear popej,

I downloaded the OSM data for the rectangle with which I am concerned from http://download.bbbike.org/osm/. Many output formats are available and I selected Garmin OSM which gave me a gmapsupp.img file of about 160 kB (similar in size to the img files generated with mkgmap but much smaller than the 4.8 MB in my original osm file). When I loaded this file into my Garmin eTrex 30 or into BaseCamp (using MapSetToolKit) I got the same result as previously: only the rivers, international boundary between Senegal and Guinea, and a small number of places are shown. Most of the data which are available for this zone on the OSM server (roads, national park boundary, most place names) are not displayed.

I went back to my old method of converting my original osm file with splitter and mkgmap and tried splitter the splitter parameter --mixed=true (and by necessity with --keep-complete=false), but I got the same result as before (most data still missing).

The missing data were mainly entered in 2012-2013 and by several community members - hard to see why these data in particular should be lost in treatment.

If someone were willing to try for themselves I could provide the osm file in which all of the data appear clearly when loaded into JOSM. Else what to do? The data are clearly in the OSM but I have not yet been able to get them into Garmin.

Thanks and best regards,
John

For others who may see some of these problems.
Ensure that you are zoomed in far enough on the Garmin device to see minor roads and tracks.
Check that you are not viewing the basemap that is supplied with the garmin - go to map>>setup>>map>>map information select map and usually you need to only have the one map ‘enabled’.
Also under map setup>>advanced map setup, select ‘most detail’.
@John_Rose if you give a link to the file i’m sure many would be interested to have a poke around.
I can’t as I only have iphone access currently, but I am intrigued.

Neither of the explanations you provide is pertinent (I am definitely seeing new data not on the Garmin basemap - detailed traces of the rivers with their names). The 4.8 MB osm file compresses to a 420 kB zip file (I guess because the downloading of multiple segments causes an increase in size) which have have uploaded to http://niokolo-safari.com/IMG/zip/Niokolo-Koba-osm.zip.

Looking forward to learning what you find (first load it into JOSM).

Thanks and best regards,
John

John,
I tried to convert your osm file into a garmin map with my scripts and I also saw a lot of details missing (I see only one road and a much smaller area).
I don’t have a clue what is wrong here, but I suggest to grab the area with overpass instead of JOSM:
Go to http://www.openstreetmap.org/#map=9/13.0126/-12.8897, click on the export tab, choose Overpass Api, save it as map.osm and process it further with mkgmap (if it is not too big, splitter isnt necessary).
When I do this I see much more map elements and a bigger area. Somehow mkgmap cannot process your JOSM output?

BTW on http://garmin.openstreetmap.nl you can get a weekly update of the whole area without all the hassle of the mkgmap process.

These data seems to be complete, when inspecting with JOSM or Global Mapper. There is not many objects in this area.

Problem is that this data contains multiple bounding boxes. Probably only first is used, limiting map area. I guess that this is due to merging data. Remove with text editor all <bounds … /> tags at the beginning of file and you will get full map.

Thanks popej and ligfietser.

a. When I take ligfietser’s advice and select my rectangle with http://www.openstreetmap.org/#map=9/13.0126/-12.8897 , export with Overpass Api, then process the resulting osm file with mkgmap, then install in BaseCamp with MapSetToolkit (under Windows XP), almost all of the elements are there (see below). Same result when I ask mkgmap to generate a gmapsupp.img file to load into my Garmin eTrex 30.
b. When I take popej’s advice and remove the 18 “<bounds … />” lines from the osm file, the saved new osm file seems to be the same as the original when read in JOSM. When I generate the Garmin map with mkgmap and load it into the GPS or BaseCamp I get the same results as for the Overpass Api. *
c. I went back and checked the Garmin map generated from my original osm file by splitter and then mkgmap. I was wrong in saying that lots of data were missing; in fact the result is the same as in the above methods: splitter removes the excess boundaries and generates a single well formed tile (I guess since there not very much data even through the zone covered is large).

What I did not notice before is that you have to zoom to 3 km/inch in BaseCamp to see the main road (N7) and to 700 m/inch to see tracks and place names - very inconvenient when using as a prospecting or planning tool. Same problem but worse for the gmapsupp file in the GPS unit where everything is blank until you get down to 300 m for the scale bar. I resolved this for BaseCamp by adjusting the “levels” and “overview-levels” parameters in mkgmap as follows:

```
–levels=“0:24, 1:24, 2:24, 3:24, 4:24, 5:24”

<br>```
--overview-levels="6:22, 7:20, 8:18, 9:16, 10:16, 11:16"
<br>```
<br>In brief I am in much better shape than before your advice, but two problems remain:
<br>1) For the gmapsupp file it seems that levels are not taken into account. It's almost impossible to zoom in to the wanted feature since everything is blank until you get down to 300 m for the scale bar, at which point you are too close to navigate effectively for more than a few hundred meters. Is there a way to show details at low resolution in the eTrex30?
<br>2) The national park boundary (named multipolygon) does not render in either BaseCamp or the GPS unit (apparently a problem with mkgmap). Could you advise on how to make it display?
<br>Thanks and best regards,
John*

You should follow popej advice.

Please post your mkgmap command options and style files and the OSM id of the objects you are trying to display.

Dear popej,

I have downloaded the mkgmap “Conversion Style manual” which is quite complicated. I understand that I should modify the default styles as found in the examples/styles folder of mkgmap.

Up until now I have not used anything other than the default styles. My command is:

java -jar mkgmap-r3336/mkgmap.jar -n 70000000 --description=Niokolo-Koba --country-name=Senegal --country-abbr=SEN --family-id=2001 --product-id=1 --tdbfile --gmapsupp --levels="0:24, 1:24, 2:24, 3:24, 4:24, 5:24" --overview-levels="6:22, 7:20, 8:18, 9:16, 10:16, 11:16" --index --route my_osm_file.osm

Here are the changes that I would like to make if possible:

  1. Display in both BaseCamp map and gmapsupp the multipolygon of the Park boundary (I am not sure how to find the id of the multipolygon - the most important segments (which are not already displayed as rivers, administrative boundaries, etc.) are id=238416622 and id=238416621.

  2. Show all place names on the map in both modes (BaseCamp and gmapsupp); right now some of the place names (like id=2901070775) only show if you move over the site with the cursor.

  3. Show the most possible data at the lowest resolution. This is not a problem in BaseCamp with my “levels” and “overview-levels” parameters, but is a major problem in the gmapsupp file.

Any concrete advice you could provide would be most appreciated as it would considerably reduce my learning curve.

Best regards,
John

This doesn’t look good, I think mkgmap should give a warning or even stop processing. Option --levels defines resolution for map levels. Resolution should decrease. I don’t know what mkgmap is doing with your definition of levels, but I don’t think results are good. Maybe in Basecamp but probably img for device is damaged. I suggest to try something more common:

--levels="0:24,1:22,2:20,3:18"
--overview-levels="4:17, 5:15"

If you’d like to make some object visible when zooming out, then you have to change definition of this object in style. Or you can try to change settings in your GPS, there are some options for map details depending on map scale. There is option in BaseCamp too, right click on upper toolbar and enable something like “Details level”.

id=238416622 is a line with tags: boundary=national_park. There is no standard Garmin line for national park, you should invent something yourself and add a new definition to “lines” file in style. For example you can add after line:

boundary=political [0x1c resolution 19]

your own definition:

boundary=national_park [0x1c resolution 19]

This definition will create lines for national park boundary, using the same line type 0x1c as political boundary.

You should learn how to create a TYP file for your map. Visibility of labels can be changed in TYP file. Here is a good TYP editor:
https://sites.google.com/site/sherco40/
but I’m not able to suggest a good tutorial for TYPs.

Yes, see above explanations for levels.

  1. Those ways are not converted. The default style only convert the following “boundary”

> grep boundary 

lines:# The following boundary styles are after the highway rules because ways
lines:boundary=administrative { name '${mkgmap:boundary_name}' }
lines:boundary=administrative & admin_level<3 [0x1e resolution 12]
lines:boundary=administrative & admin_level<5 [0x1d resolution 19]
lines:boundary=administrative & admin_level<7 [0x1c resolution 21]
lines:boundary=administrative & admin_level<9 [0x1c resolution 22]
lines:boundary=administrative [0x1c resolution 22]
lines:boundary=national [0x1e resolution 17]
lines:boundary=political [0x1c resolution 19]

1b. ¿Why did you use a way almost parallel to the river https://www.openstreetmap.org/way/238416624 and not cut the waterway and use some parts for the relation?

Thanks popej,

The code for boundaries you cite is in the file called “lines” within the default style folder. I added the line

boundary=national_park [0x1e resolution 12]

and I now see the Park boundaries in both BaseCamp and my eTrex 30. [Please note that the boundaries were entered by others and should be adjusted as you noticed; but first I have to find out where the real boundaries are, for example in the middle of the river, on the bank, etc.]

But I still have the problem of resolution on the eTrex 30. Even though the boundary is set as above in the style, when I load the corresponding gmapsupp file I have to go down to 200 m per cm on the screen in order to see the boundary, making it virtually impossible for me to navigate around the Park on the GPS unit since everything is blank until a very high resolution. Taking your advice I have tried deactivating the Garmin Worldwide Basemap and then I get a much better situation (the boundary and roads show up when the resolution is equal to 3 km per cm on the screen), but this is not quite enough to be convenient (I would like to kick in my gmapsupp data at perhaps 15 km per cm on the screen. Can you advise on this last remaining problem (maybe there is a physical setting the the eTrex 30 which could resolve it?)?

Best regards,
John

The trick is do not use overview-levels in your option file.
Those levels are not rendered in the gmapsupp, in order to show the gps basemap.
Just specify --levels=“0:24,1:22,2:20,3:18,4:16,5:14” is all you need to do.

If this map is offline and you are not yet editing the online osm, you could change the tag boundary=national_park to a different type of boundary that does show on the etrex at lower zoom levels. Some of the admin levels should be suitable or if able to and desperate use highway=motorway. Often national parks are in remote areas and other admin areas should’nt interfere.
Then once you have sorted everything out you can apply the changes to the online osm using the correct tags.

I believe the garmins will still not display some elements at lower zoom levels even though you edit the typ file and style sheet but I do not have access to a computer to confirm this at the moment.

Edit … Take ligfietser’s advice … He send his reply while I was still typing.

Dear ligfietser,

Removing the

--overview-levels

parameter did not change anything. However, when I choose the “Setup Map/Advanced Map Setup/Detail” menu on the eTrex 30 and select “Most” I can view my data when zoomed to 12 km for 1 cm on the screen (with the Garmin Worldwide Basemap disabled - 1.2 km if it is enabled) which is satisfactory for my work.

Thanks again to all those who have advised on this; I can now move ahead and learn the details gradually.

Best regards,
John