You are not logged in.

#1 2009-03-30 13:19:10

pberck
Member
Registered: 2008-09-16
Posts: 111

Rendering GPX trails with mapnik

Hi,

I have installed and played around with mapnik on my computer.

I was wondering if it is possible to draw GPX trails with mapnik as well. I looked around a bit but I couldn't find anything that worked. I thought it would be possible by converting the GPX file to data in the postgis database, possibly with its own tag, and then a rule in the mapnik stylesheet to render it.

The closest I found was a "gpx to shape" converter, which allowed to me to get some kind of blob on the map :-) but that's not quite it.

It would seem to me that most tools already exist (?) but I haven't really found something that works. Or is it much more difficult than I think to convert a bunch of coordinates to "ways" in the database?

Thanks for any pointers,
-peter

Offline

#2 2009-04-01 23:14:38

emj
Member
From: .se (59.3N17.99E) 0735969076
Registered: 2006-06-18
Posts: 949

Re: Rendering GPX trails with mapnik

You could try to tools that are availabe for converting gpx to osm which you then import into postgis and render with mapnik. Not really sure what is wronf with the gpx to shape solution, it should be the same as ugin postgresql

Offline

#3 2009-04-02 13:37:25

pberck
Member
Registered: 2008-09-16
Posts: 111

Re: Rendering GPX trails with mapnik

emj wrote:

You could try to tools that are availabe for converting gpx to osm which you then import into postgis and render with mapnik. Not really sure what is wronf with the gpx to shape solution, it should be the same as ugin postgresql

The shape became just that, a big blob on the map smile

I also tried gpsbabel to convert a .gpx file to osm format, and then use osm2pgsql to put it into the database. That fails with an error: "Operation on two GEOMETRIES with different SRIDs" which doesn't mean much to me.

-peter

Offline

#4 2009-04-02 15:47:05

emj
Member
From: .se (59.3N17.99E) 0735969076
Registered: 2006-06-18
Posts: 949

Re: Rendering GPX trails with mapnik

The coastlines are rendered from shapefiles, try to look at your shapefile in other tools first and compare in what way they are different.  I can confirm that mapnik can render shapefiles fine, since I've created my own, I had no problems so it would be interesting to know what you problem was.

GPX->osm: Try to make a simple import first then go back and see what you did wrong.


This should work, it's just reality messing with the results again.

Offline

#5 2009-04-03 06:35:58

pberck
Member
Registered: 2008-09-16
Posts: 111

Re: Rendering GPX trails with mapnik

emj wrote:

The coastlines are rendered from shapefiles, try to look at your shapefile in other tools first and compare in what way they are different.  I can confirm that mapnik can render shapefiles fine, since I've created my own, I had no problems so it would be interesting to know what you problem was.

I looked a bit closer at the options of gpx2shp - and I found a "edge" option which seems to do what I want!

Tack så mycket!

-peter

Offline

#6 2009-04-03 20:26:37

emj
Member
From: .se (59.3N17.99E) 0735969076
Registered: 2006-06-18
Posts: 949

Re: Rendering GPX trails with mapnik

np good thin you gt it working. Any pics?

Offline

#7 2009-04-03 21:21:58

the_winch
Member
Registered: 2008-02-24
Posts: 21

Re: Rendering GPX trails with mapnik

I have used mapnik to render my gpx traces, you can see the result here http://lin.pinkbile.com/gpx/
I rendered transparant tiles with a black dot for every track point. Openlayers then overlays my tiles over the osm tiles.

I wrote a simple ruby script to do the gpx -> osm conversion. It just made a node tagged with amenity=post_box for every track point in the gpx files.

Then imported the osm file to postgis with a vanilla osm2pgsql. Modified the mapnik stylesheet. Then finally rendered the tiles with generate_tiles.py. I added a line to generate_tiles.py to delete blank tiles.

Last edited by the_winch (2009-04-03 21:23:04)

Offline

#8 2009-04-04 06:41:39

pberck
Member
Registered: 2008-09-16
Posts: 111

Re: Rendering GPX trails with mapnik

emj wrote:

np good thin you gt it working. Any pics?

Here's one: http://pixelz.se/map_bjare.jpg My tracks rendered in HotPink smile Converted with gpx2shp, and drawn with a few lines of code in the mapnik python script. (A lot in NV Skåne still needs to be mapped.)

the_winch wrote:

I wrote a simple ruby script to do the gpx -> osm conversion. It just made a node tagged with amenity=post_box for every track point in the gpx files.

Nice! Are you willing to share the ruby script? big_smile

-peter

Offline

#9 2009-04-04 14:10:27

emj
Member
From: .se (59.3N17.99E) 0735969076
Registered: 2006-06-18
Posts: 949

Re: Rendering GPX trails with mapnik

pberck wrote:

Nice! Are you willing to share the ruby script? big_smile

-peter

Nice looking map, the pink is really stellar. But are you willing to share the commands use to create this? :-)

Offline

#10 2009-04-04 16:14:42

pberck
Member
Registered: 2008-09-16
Posts: 111

Re: Rendering GPX trails with mapnik

emj wrote:

Nice looking map, the pink is really stellar. But are you willing to share the commands use to create this? :-)

Of course :-)

First of all, convert a .gpx file from my  60CSx to a shape file:

gpx2shp -e 20080808.gpx

Then in my python code to draw a map, add some rules:

s = Style()
r = Rule()
r.symbols.append( LineSymbolizer( Color('#FF1493'), 2 ) )
s.rules.append( r )
m.append_style( 'My Style', s )
lyr = Layer('1')
lyr.datasource = Shapefile(file='/home/pberck/GPS/20080808_trk_edg')
lyr.styles.append('My Style')
m.layers.append(lyr)

And render then rest of the map as usual cool

-peter

Offline

#11 2009-04-04 21:07:15

springmeyer
Member
Registered: 2009-04-04
Posts: 6

Re: Rendering GPX trails with mapnik

Hey guys, if you install Mapnik 0.6.0 (the latest release) you can render directly from the GPX files.

Mapnik can do this through the OGR plugin, so just compile Mapnik with...

$ python scons/scons.py PLUGINS=all

which will enable the ogr and other plugins (only shape,postigis, and raster are default)

Then create a new OGR datasource like:

lyr.datasource = OGR(file='/home/pberck/GPS/20080808_trk_edg.gpx',layer='tracks')

you can get a sense of how to control with gpx layer is used by running the 'ogrinfo' command on your gpx file, eg:

$ ogrinfo test_point_line.gpx
Had to open data source read-only.
INFO: Open of `test_point_line.gpx'
      using driver `GPX' successful.
1: waypoints (Point)
2: routes (Line String)
3: tracks (Multi Line String)
4: route_points (Point)
5: track_points (Point)

Offline

#12 2009-04-05 09:07:15

pberck
Member
Registered: 2008-09-16
Posts: 111

Re: Rendering GPX trails with mapnik

springmeyer wrote:

Hey guys, if you install Mapnik 0.6.0 (the latest release) you can render directly from the GPX files.
Mapnik can do this through the OGR plugin, so just compile Mapnik with...
$ python scons/scons.py PLUGINS=all
which will enable the ogr and other plugins (only shape,postigis, and raster are default)
Then create a new OGR datasource like:
lyr.datasource = OGR(file='/home/pberck/GPS/20080808_trk_edg.gpx',layer='tracks')

Now you tell me :-)

This works great!

I don't seem to have the ogrinfo command though. (I have a mapnik svn checkout).

Thanks!
-peter

(edit) Example of a track rendered with Ogr datasource: http://pixelz.se/mymap_edges.png, as a red dotted line.

Last edited by pberck (2009-04-05 20:39:25)

Offline

#13 2009-04-06 07:51:51

emj
Member
From: .se (59.3N17.99E) 0735969076
Registered: 2006-06-18
Posts: 949

Re: Rendering GPX trails with mapnik

pberck wrote:
springmeyer wrote:

Hey guys, if you install Mapnik 0.6.0 (the latest release)

Now you tell me :-)

Hey! it was released after you asked your question.. :-)

Last edited by emj (2009-04-06 07:52:05)

Offline

#14 2009-04-07 13:16:28

sly
Member
Registered: 2008-07-15
Posts: 75

Re: Rendering GPX trails with mapnik

Even if you found your solution, in case someone is reading this thread later, I propose another solution to draw gpx tracks by inserting them in postgres/postGIS

( The advantage I see Is that you can create more complex queries to draw what you want, draw more tracks faster, ...)

1) Convert all your gpx tracks to shape files
2) Use shp2pgsql to import them in your database
3) define a mapnik style+layer with srs="+proj=latlong +datum=WGS84"

you'r done.

Here was my script to do the whole import (in a directory with 6,000 paragliding tracks in IGC format :

( First you need to prepare the table with -p option of shp2pgsql )

#!/bin/bash

for x in *.igc ; do
FILE=`echo $x | sed s/".igc"//`
gpsbabel -i igc -f $FILE.igc -o gpx -F $FILE.gpx
gpx2shp $FILE.gpx > /dev/null 2>/dev/null
shp2pgsql -a -g way "$FILE"_trk.shp cfd 2>/dev/null | psql gis_france > /dev/null 2>/dev/null
done

Here is my simple style+layer :
<Style name="cfd">
     <Rule>
    <LineSymbolizer>
        <CssParameter name="stroke">#bd42f6</CssParameter>
        <CssParameter name="stroke-width">1</CssParameter>
        <CssParameter name="stroke-opacity">0.5</CssParameter>
    </LineSymbolizer>
    </Rule>
</Style>
<Layer name="cfd" status="on" srs="+proj=latlong +datum=WGS84">
    <StyleName>cfd</StyleName>
    <Datasource>
      <Parameter name="type">postgis</Parameter>
      <Parameter name="user">LOGIN</Parameter>
      <Parameter name="password">PASSWORD</Parameter>
      <Parameter name="dbname">DB</Parameter>
      <Parameter name="table">
      (select way from cfd) as cfd</Parameter>
<Parameter name="extent">-180,-89.99,180,89.99</Parameter>
    </Datasource>
  </Layer>


Here is the result :
http://beta.letuffe.org/?zoom=11&lat=45 … B00FFFFFFT

Last edited by sly (2009-04-07 13:17:55)

Offline

#15 2009-04-07 15:37:55

pberck
Member
Registered: 2008-09-16
Posts: 111

Re: Rendering GPX trails with mapnik

sly wrote:

Even if you found your solution, in case someone is reading this thread later, I propose another solution to draw gpx tracks by inserting them in postgres/postGIS

( The advantage I see Is that you can create more complex queries to draw what you want, draw more tracks faster, ...)

1) Convert all your gpx tracks to shape files
2) Use shp2pgsql to import them in your database
3) define a mapnik style+layer with srs="+proj=latlong +datum=WGS84"

you'r done.

That was actually what I wanted to do initially (almost, I tried to go from gpx via osm to pgsql directly with osm2pgsql but that didn't work). I shall try your method tonight!

Cool!

-peter

Offline

#16 2009-04-08 13:54:22

pberck
Member
Registered: 2008-09-16
Posts: 111

Re: Rendering GPX trails with mapnik

sly wrote:

1) Convert all your gpx tracks to shape files
2) Use shp2pgsql to import them in your database
3) define a mapnik style+layer with srs="+proj=latlong +datum=WGS84"

Grmbl, can't get it to work. No errors, just no tracks on the map (probably means it is something simple smile )

I take a Garmin .gpx track, convert it to a shape, and import it into the DB. Selecting "way" from the created tables has lots of entries in there, so everything appears to be working.  Hmmm.... how can I debug this...?

-peter

Offline

#17 2009-04-08 14:10:48

sly
Member
Registered: 2008-07-15
Posts: 75

Re: Rendering GPX trails with mapnik

1) check if there is something in your table after the import
SELECT * FROM your_gpx_table

2) check postgres logs to see if there isn't an access denied

3) confirm there is a "access denied" to your new database/table for the mapnik user you created

4) remind that you fogot to :
ALTER TABLE your_gpx_table OWNER TO username;

That's just a guess, but that's what I forgot and exactly got nothing like you, without errors.
I'm discovering postgres and don't know why my user doesn't have access to all tables in the database...

Last edited by sly (2009-04-08 14:12:47)

Offline

#18 2009-04-09 07:01:23

pberck
Member
Registered: 2008-09-16
Posts: 111

Re: Rendering GPX trails with mapnik

sly wrote:

1) check if there is something in your table after the import
SELECT * FROM your_gpx_table

2) check postgres logs to see if there isn't an access denied

3) confirm there is a "access denied" to your new database/table for the mapnik user you created

4) remind that you fogot to :
ALTER TABLE your_gpx_table OWNER TO username;

That's just a guess, but that's what I forgot and exactly got nothing like you, without errors.
I'm discovering postgres and don't know why my user doesn't have access to all tables in the database...

1) There is data in the DB:

gis=# select count(*) from cfd;
 count 
-------
  7059

2) Hmm, there are errors in the log:

2009-04-08 06:27:33 CEST ERROR:  column "the_geom" does not exist at character 17
2009-04-08 06:27:33 CEST STATEMENT:  SELECT AsBinary("the_geom") AS geom from 
          (select way from cfd) as cfd WHERE "the_geom" && SetSRID('BOX3D(12.545 56.17817209103231, 13.0856.47480622614124)'::box3d,-1)

Does that mean I didn't prepare the tables in the correct way? As you see, I used the same tablenames as in your example.

Offline

#19 2009-04-09 10:57:02

sly
Member
Registered: 2008-07-15
Posts: 75

Re: Rendering GPX trails with mapnik

pberck wrote:
2009-04-08 06:27:33 CEST ERROR:  column "the_geom" does not exist at character 17
2009-04-08 06:27:33 CEST STATEMENT:  SELECT AsBinary("the_geom") AS geom from 
          (select way from cfd) as cfd WHERE "the_geom" && SetSRID('BOX3D(12.545 56.17817209103231, 13.0856.47480622614124)'::box3d,-1)

Does that mean I didn't prepare the tables in the correct way? As you see, I used the same tablenames as in your example.

I was wrong ;-) but this error might help greatly, when you prepared the DB with -p you didn't specified the geometry column so "the_geom" (default value with shp2pgsql) was choosen, but the query (select way from cfd) tries to get the column "way" wich was the one I choose (to keep it like the other mapnik requests)

To prepare the table you should use :

shp2pgsql -p -g way one_shp_file.shp | psql your_db
                         ^^^

Offline

#20 2009-04-09 13:09:49

pberck
Member
Registered: 2008-09-16
Posts: 111

Re: Rendering GPX trails with mapnik

sly wrote:

I was wrong ;-) but this error might help greatly, when you prepared the DB with -p you didn't specified the geometry column so "the_geom" (default value with shp2pgsql) was choosen, but the query (select way from cfd) tries to get the column "way" wich was the one I choose (to keep it like the other mapnik requests)

To prepare the table you should use :

shp2pgsql -p -g way one_shp_file.shp | psql your_db
                         ^^^

Hm, I must be misunderstanding something totally smile If I do:

shp2pgsql -p -g way 20090407_trk_edg cfd | psql gis
shp2pgsql -a -g way 20090407_trk_edg cfd | psql gis

It still gives me the same error:

2009-04-09 14:04:05 CEST ERROR:  column "the_geom" does not exist at character 17
2009-04-09 14:04:05 CEST STATEMENT:  SELECT AsBinary("the_geom") AS geom from 
          (select way from cfd) as cfd WHERE "the_geom" && SetSRID('BOX3D(4.883334414476785 51.303,5.959665585523214 51.97099999999999)'::box3d,-1)

So it still seems to want the "the_geom" column somehow?

Offline

#21 2009-04-09 16:23:54

sly
Member
Registered: 2008-07-15
Posts: 75

Re: Rendering GPX trails with mapnik

Hm, I must be misunderstanding something totally smile If I do:

shp2pgsql -p -g way 20090407_trk_edg cfd | psql gis
shp2pgsql -a -g way 20090407_trk_edg cfd | psql gis

Looks like it's perfectly good, but there still is some crap somewhere that prevent it from working. There must be a reference to that unexisting column "the_geom" somewhere.

What's in your geometry_columns table ?

Mine is :

select * from geometry_columns;
 f_table_catalog | f_table_schema |    f_table_name    | f_geometry_column | coord_dimension |  srid  |      type
-----------------+----------------+--------------------+-------------------+-----------------+--------+-----------------
                 | public         | cfd                | way               |               2 |     -1 | MULTILINESTRING
                 | public         | planet_osm_point   | way               |               2 | 900913 | POINT
                 | public         | planet_osm_line    | way               |               2 | 900913 | LINESTRING
                 | public         | planet_osm_polygon | way               |               2 | 900913 | POLYGON
                 | public         | planet_osm_roads   | way               |               2 | 900913 | LINESTRING

Offline

#22 2009-04-09 17:03:40

pberck
Member
Registered: 2008-09-16
Posts: 111

Re: Rendering GPX trails with mapnik

sly wrote:

What's in your geometry_columns table ?

This is mine (some experimental entries left):

gis=# select * from geometry_columns;
 f_table_catalog | f_table_schema |    f_table_name    | f_geometry_column | coord_dimension |  srid  |      type       
-----------------+----------------+--------------------+-------------------+-----------------+--------+-----------------
                 | public         | planet_osm_point   | way               |               2 | 900913 | POINT
                 | public         | planet_osm_line    | way               |               2 | 900913 | LINESTRING
                 | public         | planet_osm_polygon | way               |               2 | 900913 | POLYGON
                 | public         | planet_osm_roads   | way               |               2 | 900913 | LINESTRING
                 | public         | way                | way               |               2 |     -1 | MULTILINESTRING
                 | public         | cfd                | the_geom          |               2 |     -1 | MULTILINESTRING
                 | public         | cfd                | way               |               2 |     -1 | MULTILINESTRING

(thanks for all your patience and help by the way!)

-peter

Offline

#23 2009-04-09 21:46:47

sly
Member
Registered: 2008-07-15
Posts: 75

Re: Rendering GPX trails with mapnik

pberck wrote:
sly wrote:

What's in your geometry_columns table ?

This is mine (some experimental entries left):

gis=# select * from geometry_columns;
 f_table_catalog | f_table_schema |    f_table_name    | f_geometry_column | coord_dimension |  srid  |      type       
-----------------+----------------+--------------------+-------------------+-----------------+--------+-----------------
                 | public         | planet_osm_point   | way               |               2 | 900913 | POINT
                 | public         | planet_osm_line    | way               |               2 | 900913 | LINESTRING
                 | public         | planet_osm_polygon | way               |               2 | 900913 | POLYGON
                 | public         | planet_osm_roads   | way               |               2 | 900913 | LINESTRING
                 | public         | way                | way               |               2 |     -1 | MULTILINESTRING
                 | public         | cfd                | the_geom          |               2 |     -1 | MULTILINESTRING
                 | public         | cfd                | way               |               2 |     -1 | MULTILINESTRING

(thanks for all your patience and help by the way!)

-peter

okay, try to flush all entries related to "cfd" and restart the "prepare table" from shp2pgsql

( there shouldn't be the "the_geom" geometry_column in that table

Offline

#24 2009-04-10 06:24:07

pberck
Member
Registered: 2008-09-16
Posts: 111

Re: Rendering GPX trails with mapnik

sly wrote:

okay, try to flush all entries related to "cfd" and restart the "prepare table" from shp2pgsql

( there shouldn't be the "the_geom" geometry_column in that table

That worked! Now I get my track! I guess the old stuff left in the DB was the cause of the error.

Thanks!
-peter

Offline

Board footer

Powered by FluxBB