Looking for an OSM building data service

Hi Tobias & Matthias,

Some info, what OSM Buildings is capable of right now and in short term.

backend now

  • data is very custom geo to pixel, pre-rendered @ some zoom, in JSON, just footprint polygons + height
  • data is served by PHP out of a MySQL (or PostGIS) database containing just geo polygons + height
  • served area is fully dynamic - thus heavier load
  • just some data is stored, updated manually, very complicated upload

backend asap

  • move pre-rendering geo to pixel onto the client, just serve geo polygons + height
  • add a special db index like ‘map tiles @ zoom 14’ in order to have quick lookups → tile-like referencing
  • enhance JSON format for next client’s requiremens (see below)
  • if it makes any sense, create JSON data tiles (files)
  • ideally: allow access to everyone who wants to embed OSM Buildings
    => as this is the key point for costs and scalability, I could imagine either to charge users
    OR to let everybody cerate his own tile set for his web server
    OR allow people just to clone the server
    Point is: as less obstacles as possible!
  • updates should happen very often, coverage global (maybe Overpass fits here)

frontend now

  • like it is now… doesn’t create much traffic on the data side. map tile usage is much heavier

frontend asap

  • support for individual building and roof colors
  • some color determination by building type (living, religious, other…)
  • support for cylinders, domes, spheres, still in cell shading style
  • handling buildung parts as one object (for selections)
  • handling multipolygons
  • maybe support for one very simple roof shape (Tobias, this is something I would need your help)
    => all these will cause more data and will make it more attractive, I hope

not required

  • Cool details like terrain, trees, streets etc. are not planned to be part of the engine.
  • No heavy infrastructure, unless demand exists and its paid.
  • No 3d streaming needed yet, although it’s cool and maybe worth it.
  • No new standard needed. It’s basically creating it’s own. Full process of estaiblishing through the community takes too long.

Polygonal building models are a bit aside. I’d like to have a simple version, but it needs lots of performance tuning.
Therefore I’d like to postpone this a bit. And Tobias, thats also a topic to discuss in detail.

State from OSM-3D is, that Dr.Zipf is too busy to handle it and Marcus Götz, who helped me a lot, left the University.
I’m now talking to an assistand who is raising security concerns and currently not responding anymore.
As Heidelberg has every infrastructure sufficient for the next steps, would be a natural fit.

As I understand at this step you need only polygons and height at client site, so overpass sound ideal for this. For big zooms you can even resign from you server site and only parse data directly from overpass in web-browser JavaScript client. This can made your application independent from database.

Hi kendzi,
I’m going to give the API a try! If it works as expected, then it’s the ideal solution for now.
Being independent and up to date… I just hope, the service is at comparable speed delivering data.
But in worst case there will be a small caching instance.
Good news, thanks!

And one more thing which I blindly ignored: I was always expecting to have data quality similar to Germany. Testing New York City yesterday, I just woke up.
Doesn’t make sense to have just ine data source and OSM is not an ideal choice everywhere.
But no worries, the library might be open to different sources. I just shouldnt try to server the whole world from one service :wink:

Hi all,

Just a few informations on what we are currently working on at Oslandia, which could be of interest for the discussion.
We are busy with PostGIS 3D implementation, and we have some progress done.

You can read more about it here :
http://www.oslandia.com/tech/?p=1235

Some more insights into PostGIS 2.x and 3D are in those presentations :
http://oslandia.github.com/presentations/

And find some code here :
https://github.com/Oslandia/postgis-sfcgal

Our aim is :

  • to build a full 3D support for PostGIS, with 3D object, and textures as a second step. It will allow for storage (already in) and processing of 3D objects, just as you can do it with 2D data.
  • to build on top of that a «streaming» server for 3D objects. It is not streaming per se, but it’s a 3D vectorial data access webservice. It will be implemented using PostGIS 3D features and TinyOWS (part of Mapserver) WFS-T fast server. It will probably output CityGML first, but collada (KMZ) and X3D are candidates we will for sure consider. REST webservices would be great too, even if our first target is OGC webservices compliancy (yes, I know…).
  • Have tools to import / export data to and from postgis 3D (from collada/KMZ, CityGML)

As for now, PostGIS 3D foundations seem good. The next steps are :

  • discuss integration of the 3D codebase into postgis core (at OSGeo Boston code sprint)
  • improve available 3D operations in the database
  • start implementing the webservice part

Among the operations to be found, we have extrusion, straight skeleton (for roof generation) and much more (see current CGAL features).

The following steps will be :

  • integration of 3D objects from webservices into 3D web clients (cesium, opengeoglobe, openlayers 3)
  • data edition

Note that 3D objects are in the Cesium Roadmap, and Openlayers 3 will probably use Cesium for 3D. As for data edition, it needs to be thought about…

We will work hard this year to have as much of this stack as possible implemented. We are open to any contribution and help, just follow the projects on their respective source code repository and send pull requests, or write to respective mailing lists.
If you are interested in collaborating, do not hesitate to send a direct email too, as we do not follow this forum closely.

We will have to think about scalability as well, but PostgreSQL/PostGIS has made great progress lately, and probably with decent hardware, sharding and replication we can handle quite large load. If it’s not enough, caching 3D objects in a NoSQL database (memcache, redis…) could be an option. This would depend on the exact use case and will probably need prototyping and testing.

All that do not answer right now to the OP needs, which are different, but I hope it gives you some matter to think about for the future, and answer !i! comments on a full-featured 3D object streaming stack.

This could be of interest to use as a 3d database: http://www.3dcitydb.net/index.php?id=1880

Interesting paper on preparing data to CityGML standard: http://3u3d.edpsciences.org/index.php?option=com_article&access=standard&Itemid=129&url=/articles/3u3d/pdf/2012/01/3u3d_03004.pdf

Hi guys. Tordanik alerted me of this thread. I’m the developer of WikiMiniAtlas (http://wiki.openstreetmap.org/wiki/WikiMiniAtlas) the slippymap gadget on Wikipedia. WikiMiniAtlas has displayed 3D buildings for a few months now, but until last week only as wireframes rendered on a 2D canvas overlay. I’ve just implemented nice 3D webgl-based buildings. It is live on Wikipedia. My tool needs almost no preprocessing of the OSM data. It accesses a copy of the mapnik rendering DB that we maintain on the wikimedia toolserver. I have a small PHP script that generates JSON tiles from the db (on demand, but cached). At high zoomlevels those are are rendered in the browser onto canvas elements. The JSON data contains a large selection of OSM tag data, among it everything that is necessary to render 3D buildings. The code is basically using raw WebGL to keep everything small and fast. I use poly2tri to triangulate the polygons, this way I can easily handle Buildings with courtyards etc (holes).

The whole 3d stuff was basically written in a week and builds on my client-side rendering infrastructure (which was written in a week last summer). WikiMiniAtlas gives priority to displaying Wikipedia articles on the map, OSM data gives context, but no OSM text labels are shown on the map (at high zoom levels you can click on roads, buildings etc. and the OSM name pops up).

The biggest problem I currently foresee with 3D building data is the unfortunate policy of tagging the main building=yes element with the maximum height of the building, while building:part=yes elements are tagged with actual (smaller) heights. If this is followed then every building will just render as a box, and the detail given by building:parts is covered up. Detecting if a building element contains building parts is not trivial (actually almost impossible if you only have the mapnik rendering database to work with). This is a bit frustrating.

The relation type=building is to make easy to detect buildings parts. All building with parts should have this relation. So if in this relation there is at least one element with building:part you should render only building parts and skip building outlines (building=yes).

the

is necessarily for the level of detail concept:
When You wish generalize easily the 3D objects to the cuboids without roofs, You need only one parameter, which is height. Every other techniquel like analyse of building geometry and calculation of total height is more complicated.

Level of detail is fine, but the current implementation is not. The height key is used inconsistently for both the maximum height and the minimum/true height. It would be simpler to introduce a key such as max_height. Also I wonder if this level of detail concept is used at all in any application that processes or visualizes OSM data. The current concept places undue burden on applications that want to extract building detail. The relation concept is flawed for two reasons: 1) relations are not consistently used in building tagging and the complicate the tagging process, 2) visualization applications may not have access to the full relation data, which is lost when using a mapnik rendering database. Accessing the original OSM data requires costly preprocessing to get something useful on the screen.

Executive summary: if you care about application developers and getting 3D buildings to the end-user then change the inconsistent usage policy of the height key!

Where is height being used for something else than the maximum height? On building parts, it refers to the distance from the lowest point of the building above the ground up to the highest point of the roof. Clearly this is the “maximum height”? I fail to see any inconsistency here.

Ok, Tordanik, I guess you can make a case for height being max height in every case. But my other point still stands:

  • it is unreasonably hard to detect if building element has parts associated

Is the level of detail concept leveraged at all in any application? If so let’s look at the application scenario. Is it performance critical, like displaying tons of 3D buildings on a map?

This thread started because a dev was looking for a data source for 3D buildings. Well, we have a simple one the osm2pgsql rendering database. It can be optimized for fast access with cheap queries, that is unless this level of detail policy is enforced. Then at best we get expensive queries involving relations, and in the worst case have to check for geometric containment which is even more expensive.

The 3D buildings standard is still new. It can still be fixed and made more application friendly.

Moin

Hmm, the building outline with building=yes is available, if you got no free ressources, use that. IF you got ressources, search for tags and apply them. Each building:part has a building=yes associated and has a closed polygon line. See each building:part as a single building.
And here the realation comes into play: it group these parts together into one building.
If you got the OSM data, you got the relation data, to. I do not know any service which has the OSM data but not the relations.
You wrote about map services - they do not have the tags with height, building parts,… also. The map services usual do only have building=yes outline and nothing more.

So for performance: use building=yes. For good results, use the relation.

But what is your idea about tagging buildings as they are (with all different parts) which will be application friendlier?

Amiga4000

I have access to a osm2pgsql mapnik rendering database. It does contain building and building:part data. I have a working 3D building display. But it currently only works because building elements are not tagged with max height. The rendering database does contain some relation data, but it is hardly usable (multipolygons are collapsed into new geometry objects and I’m not sure all the way number data is preserved) and would increase query complexity tremendously if it would work at all.

My idea is not using the height tag on the building elements as a low level of detail max height but use a separate key for that. This way it would be a straight forward process to paint all building and building:part elements in 3D and get the desired result. Buildings without parts would display fine, buildings with parts would not need the complicate suppression of the low level of detail volume, which would otherwise cover up the high detail from building:part elements.

If you want to get beautiful 3D buidings visualized for the users you’ll have to make it easy for application developers. And my tagging scheme would not complicate matters much for the taggers either.

Moin

So only the height on building=yes is a problem if building:part are available?
Currently I do not map any height on building=yes if building:part are available, as the building:part should cover all the building:yes area and should have all the tags, also.
In this case: drop all 3D information from building=yes and use only the building:part information, job done.
But others do tag some information on building=yes, as they keep the same for the whole building. One simple pre-process would be to copy over information from building=yes to the building:part in the same relation and not copy over the information present in building:part, afterward ignore building=yes. Looks quite simple and fast to me. Just instead one building, you result in 2,3,4 or even far more (a church could have 20 building:part).

About way to get a result: 2 ways, one is keep it simple for the DB, keep it simple for the user, keep it in the way of OSM. The other is: make it easy for the apps.
Currently the OSM way is the first one, as a lot of information could be gathered automatic by the apps, with a bit more work.
At least thats what I do see.

Amiga

That is perfectly fine and will automatically render fine.

Like roof:shape? This didn’t even occur to me yet! I was mainly concerned with the height tag. I’ve only just started implementing visualizing the roof shapes.

Now it starts to get more complicated. In the rendering database I use I cannot do that. We synchronize to a master copy via replication. I could only add a new table for augmented buildings, which i would somehow have to synchronize to the rest of the db. And I don;t think I have all the necessary data (relations).

The frustrating thing is that everything could work so easily and right now, without any postprocessing and qomplex db acrobatics, if only one tiny change would be made

*not tagging the main building element with height if parts are present
*or treating building just like building:part and code the actual height rather than the min height (unless parts are guaranteed to cover the entire area of the building! Are they?)

Moin

Ok, so you do not only limit it to the height, but to all tags which are set to building=yes AND to building:part=yes ?
Usual it is defined: if set on building=yes, it is for the whole building and should not be changed in building:part again. Which is the treat building like building:part (for that specific feature).
Why only limit it to the height?
Btw, if height is set in building:yes, it is the max height of the whole building (and all building:part) of the building, just some parts could have a min_height, aka free space under them.

Amiga4000

Why do you want to forbid that? What with software with don’t support building parts?

And you end up with one building created with multiple parts and all of them tagged building=yes. So how to easily count building in city?

building=yes – it is building outline mostly for 2d render.
building:part=yes – it is part of building and it is mostly for 3d render

There are many database shames. Some of them are optimized for some task like routing or 2d render. So if you want to add 3d you need to extend your database schema to support relations.

That’s true, and in my opinion it is an argument that a debate on omitting the height tag from outlines may be focusing on the wrong problem: Avoiding the need to associate building parts with their buildings.

A lot of these tags that are “inherited” from the outline may not matter for WikiMiniAtlas right now, or at all - such as roof shapes, colors or materials. But others may very well matter. As one example, if you ever want to highlight the entire building the current Wikipedia article is about - instead of just its 2D footprint - then you will need the wikipedia tag from the building outline for that.

I think as a short-term solution, Dschwen’s current approach to omit buildings with no height is fine because de facto it works most of the time. But to get high-quality results, it would be far preferable to find a solution for creating that association.

I agree with the conclusion that that a database designed for 2D rendering may not be really well suited for the use case.

However, the relation alone is not a perfect solution with real-world data: The majority of buildings mapped with building parts today does not use the relation. We have 23 274 building:part=* ways and only 2 790 members with “part” role or empty role (some of which are probably something else, but it may serve as an upper bound for the number of building parts in relations).

Because height currently is changed in building:part even though it is set on building=yes! I don’t know of any other property that is (ab)used in this way.

I don’t want to “forbid” that (well, I would if I could :wink: ), I’m suggesting to leave out the height tag on building=yes elements. What software does not support building parts anyways? What is there not to support?! How hard could it possibly be to consider building:part elements if you can already process building=yes elements?

No. What? No! Absolutely not. How did you read that into my comments? Why would I want to tag all building:parts with building=yes?

Yes, sure, ok. Perfect argument for dropping height from building=yes. No 2D rendering needs height!