Using OSM data to convert into shapefil format

nvidia · 2008-10-28 13:02:39

Hi,

i was told that i could use osm2pgsql to dump osm data into a postgis db and from there i could export in a shapefil format. My question is how exactly do i do that? I have post gis and postgres and postgresql but i'm a bit lost as to what i need to do next in order to convert to shapefile format. Also, do i need to using Mapnik for this?

Cheers

nvidia · 2008-10-28 13:24:27

Hi,

i am begun reading the wiki http://wiki.openstreetmap.org/index.php/Mapnik/PostGIS tells me how create a spatial db on linux, however, i'm running windows, is there a method on windows for creating spatial db?

Cheers

JRA · 2008-10-28 16:32:45

Hi,

I think I should write an improved how-to into some better place but I hope this page offers some help for you. See
http://wiki.openstreetmap.org/index.php/User_talk:JRA

nvidia · 2008-10-28 20:26:17

Hi,

Thanks for your reply, i also understand that you can use osm to be converted to gml http://wiki.openstreetmap.org/index.php/GML

which states

After successful conversion to GML there are other tools like OGR2OGR which can turn GML e.g. into Shapefiles (shp).

and i'm reading the python section about running the osm2gml.py script with the followng command cat yourfile.osm | ./osm2gml.py > yourfile.gml

however when i type that into my python command line i get an error >>> cat map_01_data.osm | ./osm2gml.py > map_01_data.gml
SyntaxError: invalid syntax. I have i done something wrong??? I have both the files under my c:\

What do i need to do to correct this?

Cheers

emj · 2008-10-29 15:47:20

BTW what are you using shape files for? (Both of you)

nvidia · 2008-10-29 21:19:23

Well, once converted into shapefil, i'd like to use it with GeoTools and search for locations. Therefore hopefully, i can also show my map visually because my osm data is just simple xml.

Cheers

emj · 2008-10-30 08:22:15

Have you tried Kosmos? Btw I did a small list of OSM renderers.

PS I know you want a name finder as well DS.

nvidia · 2008-10-30 13:23:18

Hi,

i entered a similar command on windows dos prompt osm2gml.py < map_01_data.osm > map_01_data.gml but got errors

Traceback <most recent call last>:
File "C:\osm2gml.py, line 86, in <module>
xml.sax.parse <sys.stdin, osmParser >
IOError : [Errono 9] bad discriptor

although i did manage to conver my osm data into gml but it clearly it was not successful and it's only 1kb. Do you know how i can rectify this becuase if i can convert this to gml, i can conver it to shapefile. I want to try using the osm2gml.py < map_01_data.osm > map_01_data.gml code first before doing anything else.

Last edited by nvidia (2008-10-30 17:38:39)

emj · 2008-11-01 23:47:53

"xml.sax.parse <sys.stdin, osmParser >
IOError : [Errono 9] bad discriptor"

probably means that it didn't understand the "<infile.osm >oufile.gml" syntax, where infile is "sys.stdin" in his case.. You can try to write "cat infile.osm | osm2gml.py" instead not sure if it will work.. Best would be if the script could support opening files on its own insed of using STDIN.

nvidia · 2008-11-03 01:41:59

Hi,

I've managed to convert my osm into gml then to shape file succesfully now using FWTools. I've got two other corresponding files, the .dbf and .shx. I used FWTools to view the shapefile and it showed a visual representation of the of the roads and how they joined. However, now what it is missing is the road names ON the shapefile. Is there anyway for me to use FWTools and my other 2 corresponding files to help make another layer, road_names.shp say,and add it on my map_data_01.shp file.

Thanks

emj · 2008-11-03 11:26:38

Links to FWTools, what is that? The process to create shape files should not remove the name attributes on street polyline/linestrings what every they are called.

nvidia · 2008-11-03 15:07:35

Well, from osm file, i converted it to gml format using the osm2gml.py file. From that, i used FWTools (http://fwtools.maptools.org/) which has ogr2ogr built into it using a command like ogr2ogr planet.shp planet.gml
to convert it to shape file. I mean, the orginal file ie. osm was all taken from the osm website, so i do not know why it would not show the names of the roads?? Any suggestions?

JRA · 2008-11-03 18:23:44

Hi,

First step you've done was a conversion to gml. That's xml as well, and you can check if street names are still there by opening the gml file with text editor. When it comes to shapefile format, it is actually a bunch of files with the same base name and different extensions. Attributes like street names are stored into .dbf file. It is dBase format and you can open it with spreadsheet programs like Excel or OpenOffice Calc.
FWTools is the Swiss knife of GIS. The package inludes OpenEV viewer and I suppose you have used it for viewing your shapefile. It can show the attributes as well but I suggest you to let it be and use some more feature rich software. OpenEV if fine viewer for raster data and it can handle for example native Landsat scenes with all 8 bands but its vector handling capabilities are limited.
You may try OpenJUMP or Quantum GIS (version 0.11 does not work with my Vista but the older 0.10 does). Some people like uDig. All those can be used not only for viewing but also for editing data. If you want just to look at the data then Tatuk GIS viewer is pretty good for that.

Last edited by JRA (2008-11-03 18:24:43)

nvidia · 2008-11-04 00:00:00

Well,

what i ultimately want to do is to convert my osm file, into a shapefile which not only shows the roads, but shows the names corresponding to them. Because at present it just shows the roads a no names which i found very strange. I assumed that after converting it from gml to shapefile, i would see both roads and the names. With my db file, is there anyway for me to add those names onto my shapefile because ideally with any road map, there needs to have roads names, not just a bunch of lines connected together.

In a nut shell, my objective is to convert my osm into shapefil so i see BOTH the roads and the NAMES!!

Clearly it did not work when u used FWTools because again, it did have 0gr20gr built into it but from what i tell i'm getting close.
What should i do inorder for me to have my road names appear on my shapefile?

I don't want to download lots gis viewers aswell.

Cheers

Last edited by nvidia (2008-11-04 00:27:54)

emj · 2008-11-04 04:02:06

nvidia wrote:

there needs to have roads names, not just a bunch of lines connected together.
In a nut shell, my objective is to convert my osm into shapefil so i see BOTH the roads and the NAMES!!

That's because it's not showing roads, just lines, but the lines have attributes that make them roads, e.g. names and road type. Since OSM have all kinds of types of lines roads/lakes/buildings, you need to style your render in some way using a stylesheet.. (I'm just guessing here)

Last edited by emj (2008-11-04 21:56:47)

JRA · 2008-11-04 11:00:25

nvidia wrote:

Well,
what i ultimately want to do is to convert my osm file, into a shapefile which not only shows the roads, but shows the names corresponding to them. Because at present it just shows the roads a no names which i found very strange. I assumed that after converting it from gml to shapefile, i would see both roads and the names. With my db file, is there anyway for me to add those names onto my shapefile because ideally with any road map, there needs to have roads names, not just a bunch of lines connected together.
In a nut shell, my objective is to convert my osm into shapefil so i see BOTH the roads and the NAMES!!
Clearly it did not work when u used FWTools because again, it did have 0gr20gr built into it but from what i tell i'm getting close.
What should i do inorder for me to have my road names appear on my shapefile?
I don't want to download lots gis viewers aswell.
Cheers

Well, shapefile is just a binary file containing data. Vector coordinates are stored into .shp file and attribute data into .dbf file. It is not much to look at if you don't use some sort of GIS viewer.

Your principal problem with OpenEV viewer, for example, is that by default it does not show the names or "labels", which they are often called. If you open the layer control, right click on your converted OSM data layer you will see on the second tab "Draw Styles" that the last box "Labels" is by default set to "disabled". In theory you can see the labels in OpenEV user if you select the field to be used for labeling from the pull-down menu. In theory because I think that OpenEV has some kind of the bug in this. Naturally I now assume that you have the names transferred correctly into shapefile, but that you have for sure already checked from the .dbf file.

I recommend next to download one GIS viewer, they all behave in a similar way. Find out a place to tell the software that you want to see your ways labeled and point out the field to be used for labeling. The software do not know without you telling it if you want to see road names of reference numbers or what as labels.

Quantum GIS and OpenJUMP do both offer fine features for styling roads and labels, like selecting the colours and widths, label fonts, font size, the scale when labels come visible etc. Try all the controls and see what happens.

nvidia · 2008-11-04 13:52:58

Hi,

Thanks for your reply. I tried to add the labels but it unfortunately did not show up . I even made the font blue and size 20pt but it did not have any affect at all. I take it this is where the bug comes in, thanks OpenEV!! If i download something like OpenJump, will it allow me to add the names ontop of my shapefile? yes or no?

btw, i did check my dbf and yes it does contain the names.

Cheers

JRA · 2008-11-04 16:46:38

OpenJUMP and Quantum GIS will do it for sure for you. A brutal example can be found here

http://wiki.openstreetmap.org/index.php … enJUMP.jpg

emj · 2008-11-04 22:00:16

I think that how to is needed, there are so many utils and other stuff that needs explaining for OSM folks.

I don't use Windows at all so I can't write it, I could spend sometime to do it for MacOSX but I hae no idea what utils work on Mac. AxelM on the Swedish subforum uses Illustrator+commerical plugins to make maps on the mac, what he describe seems like a very manual process though.

Last edited by emj (2008-11-04 22:03:15)

nvidia · 2008-11-11 22:09:16

Hi,

I downloaded the openjump app, and found it quite appealing than the fwtools is. I'm reading this link here http://openjump.org/wiki/show/Index to show me how to use it, but i can't seem to find a way for me to display the road names on my shapefile. Although i can view the attributes like name being the road names ect, but can't seem to figure out how to display the road names. I thought about adding a new layer called road names, but at that point this would mean i would have to some how convert my db file into shapefile, then load that into the application which seems overcomplicating the matter. Any suggestions how i could see the names?

Cheers

emj · 2008-11-11 23:50:57

Search for "labels openjump" on google and you might find something. ( did find stuff but it's very hard to understand a program that I never used.)

Last edited by emj (2008-11-12 10:03:17)

JRA · 2008-11-12 09:14:58

Hi,

There is a menu icon that looks like an artists palette, press that and see what happens. Or select your layer on the layer list that is on left of the map window, right-click and take the Change styles option. OpenJUMP does have documentation that is not bad at all. Both a document an an on-line version, have a look at this, for example:
http://openjump.org/wiki/show/Layer+Menu

nvidia · 2008-11-15 22:20:26

Hi,

I managed to show the labels i.e road names on my map which is good. Now, is there any way for me to save the shape file BUT with the names already appearing on it so that next time i view it, i don't have to go and click Change Styles>Labels>Names because as i don't want to repeat this process again and again if u know what i mean.

Another reason for this is beacuse in Geotools website they have a section about displaying shapefiles, so i'd like to display both the map with the names already appearing without having to ADD it on the shapefile.

Cheers

Last edited by nvidia (2008-11-15 22:28:50)

JRA · 2008-11-16 15:35:43

The names are in the shapefile all the time, it is just if they are shown or not when you open the file. In OpenJUMP and many other similar programs you can save the "project", that is, all you have collected in the map window including styles. Next time you can open this saved project and you will see your map styled in the same way that it was when you saved it.
I am pretty sure this in written somewhere in the documentation, otherwise I wouldn't know it.

nvidia · 2008-11-17 14:43:58

Oh i see, i'll try it out. By the way, i don't know if this happens to u but when click on the open jump icon, it take a good while perhaps 1/2 mins for it to load up on my machine. Sometimes, when i try to open a file on C:\ drive for example, it takes some time for it to display my files. Does anybody know why it takes this amount of time?

or is it the case that i need more ram or something?

cheers

Announcement

#1 2008-10-28 13:02:39

Using OSM data to convert into shapefil format

#2 2008-10-28 13:24:27

Re: Using OSM data to convert into shapefil format

#3 2008-10-28 16:32:45

Re: Using OSM data to convert into shapefil format

#4 2008-10-28 20:26:17

Re: Using OSM data to convert into shapefil format

#5 2008-10-29 15:47:20

Re: Using OSM data to convert into shapefil format

#6 2008-10-29 21:19:23

Re: Using OSM data to convert into shapefil format

#7 2008-10-30 08:22:15

Re: Using OSM data to convert into shapefil format

#8 2008-10-30 13:23:18

Re: Using OSM data to convert into shapefil format

#9 2008-11-01 23:47:53

Re: Using OSM data to convert into shapefil format

#10 2008-11-03 01:41:59

Re: Using OSM data to convert into shapefil format

#11 2008-11-03 11:26:38

Re: Using OSM data to convert into shapefil format

#12 2008-11-03 15:07:35

Re: Using OSM data to convert into shapefil format

#13 2008-11-03 18:23:44

Re: Using OSM data to convert into shapefil format

#14 2008-11-04 00:00:00

Re: Using OSM data to convert into shapefil format

#15 2008-11-04 04:02:06

Re: Using OSM data to convert into shapefil format

#16 2008-11-04 11:00:25

Re: Using OSM data to convert into shapefil format

#17 2008-11-04 13:52:58

Re: Using OSM data to convert into shapefil format

#18 2008-11-04 16:46:38

Re: Using OSM data to convert into shapefil format

#19 2008-11-04 22:00:16

Re: Using OSM data to convert into shapefil format

#20 2008-11-11 22:09:16

Re: Using OSM data to convert into shapefil format

#21 2008-11-11 23:50:57

Re: Using OSM data to convert into shapefil format

#22 2008-11-12 09:14:58

Re: Using OSM data to convert into shapefil format

#23 2008-11-15 22:20:26

Re: Using OSM data to convert into shapefil format

#24 2008-11-16 15:35:43

Re: Using OSM data to convert into shapefil format

#25 2008-11-17 14:43:58

Re: Using OSM data to convert into shapefil format

Board footer