OpenStreetMap Forum

ag4wake

Member

Registered: 2009-09-07

Posts: 1

osmosis way-key-value filtering

Hello.  This is my first post! 

I'm fairly new to the openstreetmap world.  I have been playing around with the PostGIS version of the database using osmosis to convert the osm files to pgsql-dumps and then importing those.  The OSM database is importing fine.  I'm having a problem in filtering the data.  I thought I put together a filter list that would remove non highway nodes (removing random POIs), but have found the resulting database missing normal way nodes also. 

One sticky point is I also editted osmosis (0.31.2) to enable some easier way filtering.  I updated WayKeyValueFilter.java to support wildcards so I could include all values on a certain key.  It supports the highway.* syntax.  I ended up putting all the key.value pairs and then at the end used the new syntax for oneway.*,junction.*,maxspeed.*

WayKeyValueFilter.java
       /**
        * {@inheritDoc}
        */
       public void process(WayContainer container) {
               Way way = container.getEntity();

               boolean matchesFilter = false;
               for (Tag tag : way.getTags()) {
                       String keyValue = tag.getKey() + "." + tag.getValue();
                       if (allowedKeyValues.contains(tag.getKey()+"."+"*")) {
                               matchesFilter = true;
                               break;
                       }
                       if (allowedKeyValues.contains(keyValue)) {
                               matchesFilter = true;
                               break;
                       }
               }

               if (matchesFilter) {
                       sink.process(container);
               }
       }

The command I'm running (I've made it on multiple lines so it's easier to read)

time bzcat michigan.osm.bz2 | ~/osm/osmosis-0.31.2/bin/osmosis -v
--read-xml enableDateParsing=no file=/dev/stdin outPipe.0=rawosm
--way-key-value inPipe.0=rawosm outPipe=wayfilter keyValueList="highway.road,highway.unclassified,highway.residential,highway.tertiary_link,
highway.tertiary,highway.secondary_link,highway.secondary,highway.primary_link,highway.primary,
highway.trunk_link,highway.trunk,highway.motorway_link,highway.motorway,highway.traffic_signals,highway.turning_circle,
highway.motorway_junction,oneway.*,junction.*,maxspeed.*"
--remove-tags-0.6 inPipe.0=wayfilter outPipe.0=filtered keys="leisure,created_by,class,source,note,access,railway,bridge,layer,amenity,route,surface,level,shop,tunnel,
cycleway,natural,motorcar,boat,motorcycle,gnis,foot,abutters,waterway,bicycle,man_made,place,lanes,
aeroway,postal_code,is_in,statscan,attribution,ele,type,stream,building,landuse" keyPrefixes="tiger,geobase,nhd,nist"
--used-node idTrackerType=BitSet inPipe.0=filtered outPipe.0=usednodes
--write-pgsql-dump inPipe.0=usednodes directory=/home/alang/osm/dumps/mi enableBboxBuilder=yes enableLinestringBuilder=yes nodeLocationStoreType="TempFile"

So like I was saying, I wanted just nodes attached to highway tagged ways and I wanted to remove tags on nodes that were not useful to me.

the --way-key-value list should include highway.*, oneway.*,junction.*,maxspeed.*
--remove-tags-0.6 should remove some tags that I just don't need
--used-node should only include nodes that are used in ways

It's possible I only needed --remove-tags-0.6 and --used-node

I live in michigan.  One particular intersection which filtered incorrectly was at lat:42.335961 lon:-83.508024 (Beck/Warren).  There ended up being no nodes here. 

I'm going to try the --remote-tags-0.6 and --used-node now, but I was wondering if someone could help explain the issues with how I did it above?

Alan