Newbie - how to make large maps offline available

Hello Lambertus,

I also tried it with ‘rebuild’ and by piping, no success. I’ll try it on a UbuntuVM now, as it may be a windows issue.

Best Regards,
Nicolas

So i got it almost working in a VM (Ubuntu 8.04) (all packages installed) but receive an error:

jars@jars-desktop:~/Desktop/OSM/gosmore$ bzip2 -d bw.osm.bz2 | ./gosmore rebuild 
  1 while (xmlTextReaderRead (xml))
:1: parser error : Extra content at the end of the document

^

At that point the program hangs and I have to exit it with CTRL+C
The bw.xml ends with the tag:

</osm>

Note: I’m not working with the complete planet.osm, but with a small part: ‘http://download.geofabrik.de/osm/europe/germany/baden-wuerttemberg.osm.bz2’. May this be the problem?

2nd question: Is the output of ( http://yournavigation.org/gosmore.php ) the same that I can expect in my installation of gosmore? (So I can already start writing a parser)

Best Regards,
Nicolas

use “bzip2 -dc” instead

Thx, I think I got it working (?):

486 for (pairs = 0; pairs < PAIRS && s2grp < S2GROUP (0) + S2GROUPS; )
487 for (pairs = 0; pairs < PAIRS && s2grp < S2GROUP (0) + S2GROUPS; )
488 for (pairs = 0; pairs < PAIRS && s2grp < S2GROUP (0) + S2GROUPS; )
489 for (pairs = 0; pairs < PAIRS && s2grp < S2GROUP (0) + S2GROUPS; )
490 for (pairs = 0; pairs < PAIRS && s2grp < S2GROUP (0) + S2GROUPS; )
491 for (unsigned i = 0; i < cycleNet.size (); i++)
492 while (fread (&ndWrite, sizeof (ndWrite), 1, pak) == 1)
493 for (int i = 0; i < IDXGROUPS; i++)
./gosmore is in the public domain and comes without warrantee

And there is a gosmore.pak created. :slight_smile:

Should this be updated in the wiki ?

Best Regards,
Nicolas

No, the gosmore.php wraps the csv output from Gosmore into a generic web format (KML).

Yes, the Gosmore database rebuild was successfully completed.

If you will, please.

Verstehe ich das ganze richtig?

Es ist keinem hier gelungen, unter Window das gosmore.pak zu erzeugen?

Der Umweg über Linux erscheint mir etwas kompliziert!

Sorry, I switched over to German.

So again, in English!
Do I understand the discussion correctly: Nobody in this thread has ever successfully build a
gosmore.pak file under windowsXP?

Hi,

If you could tolerate using a non-OSM specific software then some GIS program with real-time GPS connection might be an option. I had a try with QGis and it opens fine all the shapefiles of England (downloaded from geofabrik.de, total filesize some 200 MB, QGis memory consumption after loading the files 50 MB). Screen is updated very soon when zoomed a little bit closer, but just after creating spatial index for the shapefiles with gdal/ogr shptree utility. QGis is missing real-time GPS plugin yet, but such is under development and it may already work well enough, see http://n2.nabble.com/Real-Time-GPS-Plugin-for-QGIS–td2905157.html#a2912921
I haven’t tried the plugin myself.

Hi guys,
I would also like to have the maps available offline. So, you say that optimal viewer for OSM data would be Gosmore but there is no way to use it under windows because of the pak file.

I use some shapefiles from Geofabrik in QGIS, but using it only for maps is not much comfortable - for example because searching places is done through attribute tables etc., also the file loading is not so quick. Current Qgis also has some OSM extension, but did not tried yet.

Has anybody an experience with some good and simple map viewer? I need only something for map viewing and serching - similar functionality as Openstreetmap website, GPS not needed. I am running win XP, but if it was also for linux, would be nice as my brother is running PclinuxOS.

Currently thinking about Thuban:
http://thuban.intevation.org/index.html
which looks promising

I run PCLinuxOS on a desktop PC and Acer Aspire One netbook. I use TangoGPS <http://www.tangogps.org/gps/cat/About> on the netbook. The website says:

“The maps are automagically downloaded and cached for offline use while you drag or zoom the map. Furthermore you can conveniently pre-cache areas with tangoGPS.”

Pre-caching maps would seem to meet your needs.

Did you already have a look at
http://wiki.openstreetmap.org/wiki/Map_displays
and
http://wiki.openstreetmap.org/wiki/Software/Mobilephones ?

An update based on my own experiences.

Gosmore isn’t capable of building (rebuild) a database on Windows because the code for rebuilding is not included in the Windows versions (for programmers: disabled using #ifndef WIN32 constructs).

Why? Because Gosmore uses POSIX functions like mmap() for accessing the database and temporary files which are not supported by Windows. I’ve been trying to implement the same functionality using the Windows mmap() equivalent CreatFileMapping(), but abandoned it after encountering bump after bump. No doubt it can be done, it just takes someone with a bit more perseverance then me. All I wanted was to run Gosmore in a usable debugger environment (VS C++ 2003) so I could get a new database update for www.yournavigation.org, not rewrite a large section of Gosmore’s code…

If someone want’s to give it a try, I have a VS 2003 project file and patched Gosmore code with a lot of WIN32 fixes. Contact me to get a copy.

Hi

I’m trying to rebuild the north american database for gosmore on Ubuntu Saucy (using the version from universe repo). How long does it take, and how do I know if it is still working?

I started it around 24 hours ago. Top shows gosmore is using 100% of CPU, but as of right now, it stopped writing to /usr/share/gosmore directory more than 12 hours ago.

The output on the console:

Building gosmore.pak using style elemstyles.xml...
Icon public.png not found
Icon public.png not found
Icon religion/synagogue.png not found
Icon religion/mosque.png not found
Icon rendering/landuse/cemetery.png not found
Icon wlan.png not found
Icon rendering/landuse/cemetery.png not found
  1 while (xmlTextReaderRead (xml) == 0
  2 while (fread (&nd, sizeof (nd), 1, groupf[i]) == 1)
  3 while (fread (s, sizeof (s), 1, groupf[i + NGROUPS]) == 1)
  4 while (fread (&nd, sizeof (nd), 1, groupf[i]) == 1)
  5 while (fread (s, sizeof (s), 1, groupf[i + NGROUPS]) == 1)
  6 while (fread (&nd, sizeof (nd), 1, groupf[i]) == 1)
  7 while (fread (s, sizeof (s), 1, groupf[i + NGROUPS]) == 1)
  8 while (fread (&nd, sizeof (nd), 1, groupf[i]) == 1)
  9 while (fread (s, sizeof (s), 1, groupf[i + NGROUPS]) == 1)
 10 while (fread (&nd, sizeof (nd), 1, groupf[i]) == 1)

Am am not sure, but are you sure whether all the tools you use are aware of OSM IDs with 64bit identifiers?

See http://wiki.openstreetmap.org/wiki/64-bit_Identifiers for details … there is an entry for gosmore.

I installed gosmore from the Ubuntu universe repository using apt-get. The version listed there is 0.0.0.20100711-2ubuntu2, which looks to be older than the one in the link (24th of October, 2012). I downloaded the OSM data from http://downloads.cloudmade.com/americas.

Should I uninstall gosmore and build it from source? Is there somewhere I can download ready-built .pak files? If so, I’d really be interested in the planet database.
My goal is to duplicate the YOURS navigation server locally.

BR,
Harold Ship

Hi I built gosmore from source and it segfaults on make install during the rebuild of default.pak. I have debugged it and got it working. It’s running now on North America.
I’ve submitted a defect re: the segfault: https://trac.openstreetmap.org/ticket/5110

I think I ran Into this problem before as well. Symptoms are: no disk activity but 100% cpu usage for a long time. Gosmore is in a never-ending loop. This used to be a repeating problem, but I think Nic fixed this with this commit about a year ago.

Can you check jni/libgosm.cpp, around line 900:

#define MAX_BUCKETS (1<<26)
#define IDXGROUPS 676
#define NGROUPS 60
#define S2GROUPS 129 // Last group is reserved for lowzoom halfSegs
#define NGROUP(x)  ((x) / max_nodes % NGROUPS + IDXGROUPS)
#define S1GROUPS NGROUPS
#define S1GROUP(x) ((x) / max_nodes % NGROUPS + IDXGROUPS + NGROUPS)
#define S2GROUP(x) ((x) / (MAX_BUCKETS / (S2GROUPS - 1)) + IDXGROUPS + NGROUPS * 2)
#define PAIRS (16 * 1024 * 1024)
#define PAIRGROUPS 120
#define PAIRGROUP(x) ((x) / PAIRS + S2GROUP (0) + S2GROUPS)
#define PAIRGROUPS2 120
#define PAIRGROUP2(x) ((x) / PAIRS + PAIRGROUP (0) + PAIRGROUPS)
#define FIRST_LOWZ_OTHER (PAIRS * (PAIRGROUPS - 1))

Note ‘max_nodes’ in lower-case.

Also, did your perform the sortRelations step first as described here?

Edit 1: our post crossed :slight_smile:

Edit 2: I see you installed gosmore from the ubuntu repository. It is probably better to get the latest source code and compile it yourself because these repository packages are often quite old, though it seems you already did that judging by your latest comment.

Here is mine, looks the same to me:


#define MAX_BUCKETS (1<<26)
#define IDXGROUPS 676
#define NGROUPS 60
#define S2GROUPS 129 // Last group is reserved for lowzoom halfSegs
#define NGROUP(x)  ((x) / max_nodes % NGROUPS + IDXGROUPS)
#define S1GROUPS NGROUPS
#define S1GROUP(x) ((x) / max_nodes % NGROUPS + IDXGROUPS + NGROUPS)
#define S2GROUP(x) ((x) / (MAX_BUCKETS / (S2GROUPS - 1)) + IDXGROUPS + NGROUPS * 2)
#define PAIRS (16 * 1024 * 1024)
#define PAIRGROUPS 120
#define PAIRGROUP(x) ((x) / PAIRS + S2GROUP (0) + S2GROUPS)
#define PAIRGROUPS2 120
#define PAIRGROUP2(x) ((x) / PAIRS + PAIRGROUP (0) + PAIRGROUPS)
#define FIRST_LOWZ_OTHER (PAIRS * (PAIRGROUPS - 1))

Didn’t, but I’ve just started it, after stopping the rebuild and removing the temp files.

Yes, I have done that now.

I’m not sure yet but it might be worth while to remove any non-routing stuff from the elemstyles.xml when using gosmore as a pure routing engine (i.e. not for rendering or looking up geolocations). This should reduce the size of the gosmore.pak which might speed-up routing a bit.

Including unnecessary information:
Eurasia pak size is: 12 GB (12008443424 bytes)
Americas pak size is: 6 GB (6394700916 bytes)

A “cleaned-up” version (ahem!) is here.

Thanks @Lambertus and @stephan75. :slight_smile: I am running the rebuild - hope it works.

For clarity, the 100% CPU with no disk activity was with the gosmore installed from the Ubuntu repo. The bug I opened https://trac.openstreetmap.org/ticket/5110 is on the current source, and the small change I made fixes it for me.

Harold