OpenStreetMap Forum

The Free Wiki World Map

You are not logged in.

#1 2012-10-14 21:00:00

Sajmonn
Member
From: Slovakia
Registered: 2012-10-11
Posts: 6

How to parse .OSM files

Sorry guys but i was searching long time and i don't have a clue ... still i don't know how to do it. Anyone can tell me how?

Goal: I need for my project parse .OSM files for getting information from maps as longtitude etc. I know about Osmosis that is written in Java(i prefer Java language) but i didn't find any javadoc or references with any information about "important" java classes and interfaces (for example i absolute don't know what is Sink class or RunnableSource interface) and i cannot programming when i know anything about classes. So any help will be appreciated.

[or also when someone use other framework for reach that goal]

Thanks you very much

Offline

#2 2012-10-14 23:37:05

Marqqs
Member
Registered: 2011-01-01
Posts: 724

Re: How to parse .OSM files

Sajmonn wrote:

Goal: I need for my project parse .OSM files for getting information from maps as longtitude etc.

Hello Sajmonn,

welcome to OSM forum!

Please tell us more about the output format you prefer. Maybe you could give us examples for a few datasets?

Markus

Offline

#3 2012-10-16 15:36:01

Tordanik
Moderator
From: Germany
Registered: 2008-06-17
Posts: 2,294
Website

Re: How to parse .OSM files

There unfortunately isn't much documentation for the Osmosis classes, but the program structure is a relatively straightforward pipeline architecture, so it's not impossible to figure out.

If you give us more details about what you are trying to do, we could help you better. Do you only need to parse the files and access them as objects in a Java program? Then this code snippet could get you started.

Offline

#4 2012-10-18 21:28:06

Sajmonn
Member
From: Slovakia
Registered: 2012-10-11
Posts: 6

Re: How to parse .OSM files

Thanks for answers. Tordanik your snippet of code i used but i don't like when i don't know how it works. Can you provide me some informations about Sink class for example etc. from your snippet of code because i don't know how classes works.

is a relatively straightforward pipeline architecture

I still did not work with pipeline architecture. Do you have some references about it?

If you give us more details about what you are trying to do, we could help you better.

So i will work for application offline Android-based address search on pre-processed OpenStreetMap data. So my opinion is that important logic and "basis of app" is to get correct and efficient solution for parsing data from OSM maps. So i need to know the best solution how to parse these maps because i do not have experience with that. At first when i will parse data i will save them for example to SQLite database and then i figure out some effective structure(maybe R-tree) where will be data for fast searching.

So im glad for any suggestion or help.

EDIT:

When i run your snippet of code, this exception was thown:

10-18 20:34:24.312: E/AndroidRuntime(553): FATAL EXCEPTION: main
10-18 20:34:24.312: E/AndroidRuntime(553): com.bretth.osmosis.core.OsmosisRuntimeException: Unable to instantiate xml datatype factory.
10-18 20:34:24.312: E/AndroidRuntime(553): 	at com.bretth.osmosis.core.xml.common.DateParser.<init>(DateParser.java:39)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at com.bretth.osmosis.core.xml.common.BaseElementProcessor.<init>(BaseElementProcessor.java:35)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at com.bretth.osmosis.core.xml.v0_5.impl.TagElementProcessor.<init>(TagElementProcessor.java:31)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at com.bretth.osmosis.core.xml.v0_5.impl.NodeElementProcessor.<init>(NodeElementProcessor.java:46)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at com.bretth.osmosis.core.xml.v0_5.impl.OsmElementProcessor.<init>(OsmElementProcessor.java:46)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at com.bretth.osmosis.core.xml.v0_5.impl.OsmHandler.<init>(OsmHandler.java:55)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at com.bretth.osmosis.core.xml.v0_5.XmlReader.run(XmlReader.java:101)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at com.sajmon.activities.MainActivity.onClick(MainActivity.java:55)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at android.view.View.performClick(View.java:3480)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at android.view.View$PerformClick.run(View.java:13983)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at android.os.Handler.handleCallback(Handler.java:605)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at android.os.Handler.dispatchMessage(Handler.java:92)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at android.os.Looper.loop(Looper.java:137)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at android.app.ActivityThread.main(ActivityThread.java:4340)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at java.lang.reflect.Method.invokeNative(Native Method)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at java.lang.reflect.Method.invoke(Method.java:511)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:784)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:551)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at dalvik.system.NativeStart.main(Native Method)
10-18 20:34:24.312: E/AndroidRuntime(553): Caused by: javax.xml.datatype.DatatypeConfigurationException: Provider org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl not found
10-18 20:34:24.312: E/AndroidRuntime(553): 	at javax.xml.datatype.DatatypeFactory.newInstance(DatatypeFactory.java:103)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at com.bretth.osmosis.core.xml.common.DateParser.<init>(DateParser.java:36)
10-18 20:34:24.312: E/AndroidRuntime(553): 	... 18 more
10-18 20:34:24.312: E/AndroidRuntime(553): Caused by: java.lang.ClassNotFoundException: org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl
10-18 20:34:24.312: E/AndroidRuntime(553): 	at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:61)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at java.lang.ClassLoader.loadClass(ClassLoader.java:501)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at java.lang.ClassLoader.loadClass(ClassLoader.java:461)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at javax.xml.datatype.FactoryFinder.newInstance(FactoryFinder.java:138)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at javax.xml.datatype.FactoryFinder.find(FactoryFinder.java:219)
10-18 20:34:24.312: E/AndroidRuntime(553): 	at javax.xml.datatype.DatatypeFactory.newInstance(DatatypeFactory.java:96)
10-18 20:34:24.312: E/AndroidRuntime(553): 	... 19 more

Last edited by Sajmonn (2012-10-18 21:36:23)

Offline

#5 2012-10-18 22:49:19

Marqqs
Member
Registered: 2011-01-01
Posts: 724

Re: How to parse .OSM files

Sajmonn wrote:

So i will work for application offline Android-based address search on pre-processed OpenStreetMap data. So my opinion is that important logic and "basis of app" is to get correct and efficient solution for parsing data from OSM maps.

Seems you are looking for some preprocessed data, maybe CSV-formatted?

Offline

#6 2012-10-18 23:00:47

Sajmonn
Member
From: Slovakia
Registered: 2012-10-11
Posts: 6

Re: How to parse .OSM files

Seems you are looking for some preprocessed data, maybe CSV-formatted?

no i start with normal .OSM file and i have to parse it and then make some "black magic" with them.:cool:

Offline

#7 2012-10-20 14:43:55

Tordanik
Moderator
From: Germany
Registered: 2008-06-17
Posts: 2,294
Website

Re: How to parse .OSM files

Sajmonn wrote:

I still did not work with pipeline architecture. Do you have some references about it?

Very roughly: The basic idea is that you have several Tasks. OSM data will flow through these tasks in a pre-defined sequence - the pipeline. A Task can be a Source (provider of OSM data), a Sink (recipient of OSM data), or both. Then you have some special variants - e.g. Tasks that allow multiple successors to allow branching of the pipeline.

The pipeline is constructed by creating an instance of each Task, and connecting them using setSink method calls. Then run() is called on the first Task to start pipeline execution.

I'm not an Osmosis developer myself, and specific questions will probably be best (though not necessarily quickly) answered on the osmosis-dev mailing list.

When i run your snippet of code, this exception was thown:

Does it work in a regular (non-Android) Java environment?

Offline

#8 2012-10-24 09:22:09

Sajmonn
Member
From: Slovakia
Registered: 2012-10-11
Posts: 6

Re: How to parse .OSM files

Tordanik - thanks for your very nice answer. Exception meant above now is solved by adding apache-xml-xerces.jar but now im getting this parse error:

10-24 08:06:57.656: E/AndroidRuntime(750): FATAL EXCEPTION: Thread-85
10-24 08:06:57.656: E/AndroidRuntime(750): com.bretth.osmosis.core.OsmosisRuntimeException: Unable to parse xml file data/data/com.sajmon.activities/za.osm.  publicId=(null), systemId=(null), lineNumber=1, columnNumber=0.
10-24 08:06:57.656: E/AndroidRuntime(750): 	at com.bretth.osmosis.core.xml.v0_5.XmlReader.run(XmlReader.java:106)
10-24 08:06:57.656: E/AndroidRuntime(750): 	at java.lang.Thread.run(Thread.java:856)
10-24 08:06:57.656: E/AndroidRuntime(750): Caused by: org.apache.harmony.xml.ExpatParser$ParseException: At line 1, column 0: no element found
10-24 08:06:57.656: E/AndroidRuntime(750): 	at org.apache.harmony.xml.ExpatParser.finish(ExpatParser.java:545)
10-24 08:06:57.656: E/AndroidRuntime(750): 	at org.apache.harmony.xml.ExpatParser.parseDocument(ExpatParser.java:475)
10-24 08:06:57.656: E/AndroidRuntime(750): 	at org.apache.harmony.xml.ExpatReader.parse(ExpatReader.java:321)
10-24 08:06:57.656: E/AndroidRuntime(750): 	at org.apache.harmony.xml.ExpatReader.parse(ExpatReader.java:279)
10-24 08:06:57.656: E/AndroidRuntime(750): 	at javax.xml.parsers.SAXParser.parse(SAXParser.java:390)
10-24 08:06:57.656: E/AndroidRuntime(750): 	at javax.xml.parsers.SAXParser.parse(SAXParser.java:187)
10-24 08:06:57.656: E/AndroidRuntime(750): 	at com.bretth.osmosis.core.xml.v0_5.XmlReader.run(XmlReader.java:101)

But i still do not know why

org.apache.harmony.xml.ExpatParser$ParseException: At line 1, column 0: no element found

is thown.

Here is XML file(only one node for test)

<node id="929367522" lat="49.2226157" lon="18.7392615" user="MiMiNo" uid="103618" visible="true" version="1" changeset="5906684" timestamp="2010-09-28T19:43:21Z">
  <tag k="addr:city" v="Žilina"/>
  <tag k="addr:country" v="SK"/>
  <tag k="addr:housenumber" v="2"/>
  <tag k="addr:postcode" v="01001"/>
  <tag k="addr:street" v="Štúrova"/>
  <tag k="amenity" v="restaurant"/>
  <tag k="name" v="Tosca Cafe Restaurant"/>
  <tag k="website" v="www.toscacafe.mns.sk"/>
 </node>

Offline

#9 2012-10-24 14:20:39

Tordanik
Moderator
From: Germany
Registered: 2008-06-17
Posts: 2,294
Website

Re: How to parse .OSM files

Sajmonn wrote:

Here is XML file(only one node for test)

<node id="929367522" lat="49.2226157" lon="18.7392615" user="MiMiNo" uid="103618" visible="true" version="1" changeset="5906684" timestamp="2010-09-28T19:43:21Z">
  <tag k="addr:city" v="Žilina"/>
  <tag k="addr:country" v="SK"/>
  <tag k="addr:housenumber" v="2"/>
  <tag k="addr:postcode" v="01001"/>
  <tag k="addr:street" v="Štúrova"/>
  <tag k="amenity" v="restaurant"/>
  <tag k="name" v="Tosca Cafe Restaurant"/>
  <tag k="website" v="www.toscacafe.mns.sk"/>
 </node>

Is this your entire XML file? A valid .osm file needs at least

<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Yourprogram">

at the beginning and

</osm>

at the end, as shown in the example on http://wiki.openstreetmap.org/wiki/OSM_XML

Offline

#10 2012-10-29 19:51:54

Sajmonn
Member
From: Slovakia
Registered: 2012-10-11
Posts: 6

Re: How to parse .OSM files

Tordanik - thanks man, now seems that it works. if i will have problem i will report it.

Offline

#11 2012-11-06 09:14:35

Sajmonn
Member
From: Slovakia
Registered: 2012-10-11
Posts: 6

Re: How to parse .OSM files

Men im here again. So i read some articles about OSM XML format, about basic elements such as Node, Way and Relation but now im little confused. So my question is when someone want to get information for example latitude and longtitude of some address name(input from user in application), how can achieve it?
thanks very much.

Offline

#12 2012-11-06 17:09:09

stephan75
Member
Registered: 2008-05-28
Posts: 2,711

Re: How to parse .OSM files

Sajmonn wrote:

Men im here again. So i read some articles about OSM XML format, about basic elements such as Node, Way and Relation but now im little confused. So my question is when someone want to get information for example latitude and longtitude of some address name(input from user in application), how can achieve it?
thanks very much.

Maybe you are looking for geocoding? http://wiki.openstreetmap.org/wiki/Geocoding -> Nominatim

Offline

Board footer

Powered by FluxBB