osmosis on Ubuntu „incorrect data check”

Hi all

I was using osmosis now for a while on Windows. Now I set up an Ubuntu Linux machine for processing the OSM data. But it seems that there is a problem with the osmosis setup on Ubuntu, which drives me nuts.

Here the installation on Windows:

Windows 7
Oracle Java JRE 1.7.0_17
Osmosis 0.40.1 (http://bretth.dev.openstreetmap.org/osmosis-build/osmosis-0.40.1.zip)

Installation on Ubuntu:

Ubuntu 12.04.2
Oracle Java JRE 1.7.0_21
Osmosis 0.40.1 (http://bretth.dev.openstreetmap.org/osmosis-build/osmosis-0.40.1.zip)

I’m running the same commands on both the Windows and the Ubuntu installation. On Windows they run with out any problems. But on Linux I get errors.

I downloaded the latest planet.osm.pbf file on both Machines. Then I cut out the data for Europe and write it to europe.osm.pbf witch seems to work on both Machines From the europe.osm.pbf file I will cut out different countries. On Windows this works fine. On Linux I get errors saying that the input file is corrupt.

May 09, 2013 1:50:48 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.40.1
May 09, 2013 1:50:49 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
May 09, 2013 1:50:49 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
May 09, 2013 1:50:49 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
May 09, 2013 1:50:52 AM crosby.binary.osmosis.OsmosisSerializer$Prim serializeMetadataDense
WARNING: Attention: Data being output lacks metadata. Please use omitmetadata=true
May 09, 2013 1:50:52 AM crosby.binary.osmosis.OsmosisSerializer$Prim serializeMetadataDense
WARNING: Attention: Data being output lacks metadata. Please use omitmetadata=true
May 09, 2013 1:50:54 AM crosby.binary.osmosis.OsmosisSerializer$Prim serializeMetadataDense
WARNING: Attention: Data being output lacks metadata. Please use omitmetadata=true
May 09, 2013 1:50:54 AM crosby.binary.osmosis.OsmosisSerializer$Prim serializeMetadataDense
WARNING: Attention: Data being output lacks metadata. Please use omitmetadata=true
java.util.zip.DataFormatException: incorrect data check
at java.util.zip.Inflater.inflateBytes(Native Method)
at java.util.zip.Inflater.inflate(Unknown Source)
at java.util.zip.Inflater.inflate(Unknown Source)
at org.openstreetmap.osmosis.osmbinary.file.FileBlockPosition.parseData(FileBlockPosition.java:57)
at org.openstreetmap.osmosis.osmbinary.file.FileBlockHead.readContents(FileBlockHead.java:95)
at org.openstreetmap.osmosis.osmbinary.file.FileBlock.process(FileBlock.java:135)
at org.openstreetmap.osmosis.osmbinary.file.BlockInputStream.process(BlockInputStream.java:34)
at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:45)
at java.lang.Thread.run(Unknown Source)
May 09, 2013 4:53:26 AM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion
SEVERE: Thread for task 1-rxc failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: An output error has occurred, aborting.
at org.openstreetmap.osmosis.core.store.DataPostbox.checkForOutputErrors(DataPostbox.java:160)
at org.openstreetmap.osmosis.core.store.DataPostbox.populateCentralQueue(DataPostbox.java:216)
at org.openstreetmap.osmosis.core.store.DataPostbox.put(DataPostbox.java:303)
at org.openstreetmap.osmosis.set.v0_6.impl.DataPostboxChangeSink.process(DataPostboxChangeSink.java:45)
at org.openstreetmap.osmosis.core.sort.v0_6.SortedDeltaChangePipeValidator.process(SortedDeltaChangePipeValidator.java:70)
at org.openstreetmap.osmosis.xml.v0_6.impl.ChangeSourceElementProcessor$ChangeSinkAdapter.process(ChangeSourceElementProcessor.java:144)
at org.openstreetmap.osmosis.xml.v0_6.impl.WayElementProcessor.end(WayElementProcessor.java:117)
at org.openstreetmap.osmosis.xml.v0_6.impl.OsmChangeHandler.endElement(OsmChangeHandler.java:94)
at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanEndElement(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)
at javax.xml.parsers.SAXParser.parse(Unknown Source)
at org.openstreetmap.osmosis.xml.v0_6.XmlChangeReader.run(XmlChangeReader.java:109)
at java.lang.Thread.run(Unknown Source)

I get the same problem if I will update the planet.osm.pbf file with the daily differential files. The first update works but on the second update I get a error that the input file is corrupt. Sometimes I get the “incorrect data check”, sometimes that there is a problem with the compressed data. It seems to me that osmosis is writing incorrect data to the pbf file on my Linux setup.

Can any one give me a hint what the problem could be? Do I have to use openjdk?

I am not familiar with everyday use of osmosis, but I assume that version 0.40.1 of osmosis is heavily outdated, or not?

At least when dealing with 64bit identifiers in the OSM data you should use recent releases.

See http://wiki.openstreetmap.org/wiki/64-bit_Identifiers

It is, but I’m bound to this version trout the plugin mapsforge-map-writer. I tested the same with the newer version 0.43.1 which generates the same errors.

Yesterday I had a new idea and I setup the same installation on a VirtualBox under Windows. And surprisingly it runs like expected. So I guess it’s a hardware problem, probably the aged RAM causing this problem.

A memory test showed me that the RAM is faulty. After I removed the faulty RAM the files are now fine. So it was a hardware problem, just hart to detect.