osm-3d and (Google Earth) geomodeling

As many of you know, Google had besides their croud-sourced 2d MapMaker project also croud-sourced 3d modeling for Google Earth.

While Google strives to make users contribute primarily to MapMaker (instead of OSM), they discontinued the croud-sourced modeling in October 2013, leaving osm-3d as the single and more open alternative. There is not only a huge amount of technology and experience that dies with the end of the so-called “3d pipeline”, but also a huge modelers community that was hit at the hardest. It hurts me to see what is lost, and at the same time how beneficial it could have been to osm-3d.

I’m looking into ways to migrate the modelers community from SketchUp/3D Warehouse over to openstreetmaps, and find appropriate fields where we can contribute our skills. A challenge could be that our community – though still passionate about 3d – isn’t very active anymore and disperses into finding new hobbys related to 3d or geo. Though I’m enthusiastic about open source and open data, I’m myself new in openstreetmaps and have much to learn about it.

A main aspect is motivation. Our community was rather artistic (and cared less about the proprietary issues with Google), and it was a pleasure to see our work shown off to a wide audience (GE, 500mio installations). In addition to that, Google had professional community managers and generous modeling conferences on all continents. We can of course not expect to find exactly all of that here, but I see it as critical to identify commonalities and motivating aspects before trying to persuade people to get habituated to something new/unknown. Now it’s up to us to re-build our community.

I would be interested to learn more about the current status and roadmap of osm-3d (I’ve read what is in the wiki). My main questions are:
• The viewer on the main page is very attractive, easy to find and use, and professional (http://openstreetmap.org). I discovered that there is a plenty of powerful APIs, libraries, tools that exposed very advanced technology. However for 3d, the landscape appeared to me still early and fragmented. Since Openlayers 3 adds WebGL support, is there a roadmap to get 3d to the main page?
• OSM has rather a parametric and fact-centric approach. As for my understanding the tries to model buildings with ways and relations are good for low-detail mass models, but does it scale up to highly detailed and highly-accurate models? Are these models currently traced by hand, or is there a 3d modeler that is suitable? Are there converters or exporter plugins for more common modeling softwares?
• Are photo-textured models in the scope of osm-3d, and what format is used/preferred?
• Are there current efforts for a 3d model repository? openbuildingsmodels.org is at the moment not under active development.

Hi and welcome to OSM :slight_smile:

You posted a lot of question, maybe a few of them can be answered at this pages:

I have plans to start another approach http://wiki.openstreetmap.org/wiki/Open3DMap but this will take some time, but I’m happy that I can finally start this year :slight_smile:

I think Open3DMap goes into the right direction with linking real 3d files to osm points.
At the beginning Google had tried to do buildings with kml linestrings polygons (like osm primitives), but it had similar challenges: no textures, striking colors and no shading, bad performance. They ended up with Collada packaged in kmz, and compressed and streamed with an unknown adaptive 3d streaming technology. Since October, Google replaces step by step hand-made models with 3d mesh tiles (of currently aweful quality) to make it a more complete (with trees) and seamless experience with consistent shading.

There are tons of models readily available at 3D Warehouse. I know many modelers would agree to have their models be visible to the public again, only many people are not aware enough about licensing, and the upload process of 3D Warehouse did not provide a licensing scheme (only a “ask you lawyer” Terms of Service). The SketchUp and 3D Warehouse developers work now for Trimble but so far there were no significant changes. What types of data does osm include, would we have to persuade modelers to relicense their works? (then also the question of which license: Creative Commons, OpenData…?).

Here are examples of models: http://sketchup.google.com/3dwarehouse/doadvsearch?uq=1510783023098459660829802&isbestofgeo=true&start=0&scoring=r
and (selected) modelers: http://sketchup.google.com/3dwarehouse/cldetails?mid=bcb178508649bb8197f9872f3d808100&prevstart=0

I looked at OpenBuildingModels and uploaded some models. I couldn’t get the proprietary java plugin to work in my Firefox, and I had to experiment to convert models into the used format (vrml), but otherwise the upload process worked well. If this might be turned into a major osm project, we would definitely need a more file formats (dae) and 3d viewer, if possible without plugin, preferrably in WebGL, and a more streamlined upload website that appeals to a wider audience.

Hello Aerilius,

it’s interesting to get some insights into the state and motivations of Google’s former 3D community. When the original decision to give up this avenue of obtaining 3D data for Google Earth was published, there were already some discussions in this forum and elsewhere about the topic.

You are right about the parametric approach based on ways/relations, and indeed its main strength are relatively simple buildings. While more complex ones can be mapped (see e.g. this example), it’s not what you would get using photo-textured models. Right now, 3D data is usually added to OSM with the same tools we use to add other data, i.e. editors like JOSM or iD.

I believe that the 3D community in OpenStreetMap would be happy about the availability of free, photo-textured models of buildings and other things (I would be). However, this is a very different type of data than the ways etc. in the OSM database, so it would have to go into a separate database specialized in storing e.g. Collada files.

And that’s the main issue holding us back from using such models: Someone would have to program such a website for uploading models, along the lines of the “Open3DMap” idea. It’s not even that hard that service, but it would have to be maintained (unlike, unfortunately, openbuildingmodels.org).

However, if you believe that such a service could making open data attractive for former Google contributors, we could start another attempt to shanghai some developers into working on that project. :wink:

The only Creative Commons licenses that are (according to my understanding) definitely compatible with OpenStreetMap data are CC-BY and CC0. So if modelers could be persuaded to relicense their works under one of these licenses, then those works could legally be included into a 3D scene alongside OpenStreetMap data. However, it’s important to check whether they actually hold all rights to their creations - for example, if there are photo textures, then these have to be shot by the modeler himself or taken from a freely licensed source.

Currently there is not real 3d inside OSM. It is forbidden to put models inside OSM because they are to complex and database should be simple and human readable. Currently there is no working storage for models which connect them to osm data. Osm-3D and OpenBuildingModels repository are science project and I doubt if they target for supporting end users. OpenBuildingModels don’t provide any API for querying and downloading models, without it models can be used only for Osm-3D.

On other hand we have Open3DMap which start was planned in 2011…

From my point of view it is really hard to create service for handling world wide model streaming. It is quite complex from programming point of view to create application which provide streaming, automatic level of detail, model conversion and uploading. But much more challenging become administration. Service this kind will require really huge storage, bandwidth and maintain costs.

So we try talk about finding services for handling TB of data with specialized services. But I have really huge problems with finding working regular ftp for few MB required for kendzi3d and it seams impossible to find sever to put tomcat for some trivial back-end services…

I don’t think streaming is a requirement for the core service at all. Just like the OSM servers, the model repository itself would focus on handling uploads and providing “planet” dumps and diffs. If anyone wants more, they download these and provide regional extracts, filtering features and whatever.

The core service should be kept minimal - because if you plan too big you are afraid of even getting started. Things like that should be allowed to grow over time.

In my opinion, to get started we would just need enough “dumb” storage to last us for a year or two. Can anyone provide a guess how many MB we should expect per model and how many models we might get during the initial stage of such a project?

Hi Aerilius,

I like your idea to migrate the 3d model community to openstreetmap! And I’d really like to support this. I had several discussions with Tordanik about real 3D models to enhance the OSM2World-Slippymap but I always had the feeling that we’re facing the “chicken-egg-problem” here.

  1. We need modelers and models
  2. We need a platform to upload, view,… the models in an acceptable format and an acceptable license
  3. We need support in software like OSM2World (and ideally many others)

I think I could and would help provide part 2. A simple site for Collada-models, a simple (platform independant) viewer,… to start with. Part 3. is not so easy, but I guess Tordanik (and perhaps I) would be more encouraged to work on that (for OSM2World and the slippymap) if there are actual models that could be used :slight_smile: And as I understand it, you’d provide for part 1? You’d upload your models to kick-start the site? And you could help encourage other modelers to help OSM with even more models? If so, I’d give (2) a try.

Peda

Do we really want go into collada madness?
To be clear collada is not standard for files with models. Collada is template for creating new 3d file formats, not compatible with each other. To make more confusion with it, all that files are named collada. So one application which support collada can’t open correctly files generated by other. To make it even more messed up, many application store inside files additional data to mimic other application…

For me any file format is better then collada. For example OBJ are supported almost by all 3D editing application. They are easy to read and write and work everywhere the same.

But whatever you chose I will add support for that in kendzi3d as soon as possible.

Hm, I can not comment on that as a professional, however, I’ll add my humble opinion :wink:

I read the collada spec and the reason it was invented. I had the impression that it’s well specified and was thought to make a end to the mess with 3d file formats. Its main porpose was to have a file format that can be used to exchange 3d models between apps. Iit’s the format SketchUp supports :slight_smile: And by its XML nature it’s easy to extend, e.g. for meta data of the models.

On the other hand I have some experience with OBJ. It doesn’t support multi textures, it has some glitches how it’s interpreted (esp. textures) and it has some other shortcomings. Especially it doesn’t feature the extensible nature of Collada/XML: We need a way to store “anchor points” and “direction vectors” or something like that to integrate the models in our apps.

Anyways, I’m open minded to everything except VRML :slight_smile: I’d prefer something that’s expressive enough for nice models, open enough to be “platform independant” and free enough to not require any license fees or something like that :slight_smile:

I implemented exporter to collada and:

  • It is over-complicated
  • Only basic geometry can be portable between application
  • Everything beyond geometry like lighting, texturing is done by extension point, and extensions are application specific, undocumented, and made in strange way.
  • Collada supports multitexturing in theory, in practice I didn’t find any working or portable example of that, only some demo applications.
  • support for collada is really badly for 3d editing applications (except that which creating own extensions)
  • simply export collada from Blender, open it in notepad and search for tag all inside is not documented and application specific

As I know Google pack collada into zip package and add metadata as separate file, not inside file it self.

While the question about the right format is no doubt an important one, we should keep in mind that the model repository service itself will look 99% the same no matter whether the files it stores have .obj or .collada extensions. So first I’d like to focus on the decision whether to create it all. And to help with that decision, I’m also interested in Peda’s question to the thread starter:

Hi,
I will definitely provide all of my own models for free use if it can help make osm 3d happen (some are already to openbuildingmodels [203], [207], [209]).

I also plan to invite as many modelers as possible (I closely know the top ~80, and then there were 1000+ modelers whose work I know by sight). However there is the before mentioned chicken-egg problem: Now that so much time passed, I am not sure how many modelers I can reach and persuade. If we imagine Google would revive geomodeling, everyone would hear about it and modelers would not hesitate because of Google’s big brand and the familiar upload process. A comparable infrastructure does not yet exist for osm, so what could modelers do now (and see results?).
I’m thinking about an interim solution/activity, like adding 2d building foot prints to osm, generated from our 3d models.

About Collada: SketchUp say they have the industry’s most complete implementation (93+%) and considering that, the intercompatibility with other applications is really bad. There are even issues when re-importing into SketchUp (“projected textures”). And when importing into another application I have fear it does not support those advanced Collada features that SketchUp’s Collada makes use of.
Obj is (too?) simple, and it seems to support geometry groups (I don’t know about repeated geometry/components).

I did a collada importer to directX once and i will never go into this madness anymore i totally agree with Kendzi on this point :wink:
You can add to the list that .dae file are huge even for very simple geometries (that’s why we implemented a compressed format on F4Map)

Hi,

this is great news! I had a look at your warehouse site and this would be a huge start already!

I’d say this is enough encouragement to go and start a model site for OSM (and the like) for me/us.

However, there’s still the format problem and it’s not a good sign that 2 developers had issues with Collada and even you as a modeller had found issues. And I agree with Kendzi that metadata could also be simply put into a zip file. This way one could even distribute textures via a single file. Again, OBJ is simple, yes. But it lacks multitexturing and much of the lighting stuff. Aerillus, you used SketchUp for all your models, right? And that’s the main application other modellers use, too? For the 8.0 version that I managed to install via wine the only export format was kmz and dae. Do current version support more export formats out of the box in the free version? I’d guess this is an important point in decission making.

Before your objections, I would have chosen Collada. Now I’m not sure about the format any longer … more input and opinion welcome :slight_smile:

Aerilius:

You seem to be from Germany? Would you like to come to FOSSGIS conference in Berlin 19.03-22.03 to discuss the issue in person?

As far as I know, !i!, Tordanik and Peda will be there. I hope to be there as well.

Another issue is with modelling software.

Free version of Sketchup can’t be used for commercial work. Here is an excerpt from the license of the free version of Sketchup:

So I switched to Blender, though it definitely requires more time to learn than Sketchup.
Blender is more powerful tool than Sketchup. It’s rather a 3D platform than a software. It includes advanced modelling, rendering and animation parts. Everything can accessed via Python bindings.

At FOSSGIS-2014 I hope to talk how to georeference a 3D model with help of Blender and OpenStreetMap data.

Regarding formats.

Collada should be used only to exchange 3D models. It shouldn’t be used for production. There is some analogy with OSM file format. It’s normally used to excange data. Each OSM application uses optimized binary format for OSM data.

In my opinion pycollada library works quite reliably. It’s used in Blender to import/export COLLADA files.

I created a simple COLLADA->POV-Ray converter with the help of pycollada.

I’d suggest to have GUI for open 3D model repository similar to Sketchup 3D Warehouse.

Some time ago I wrote a simple application to explore a 3D model via prerendered sprites in the same way it’s done in Sketchup 3D Warehouse. Check a demo here.

I can contribute to a plugin for Blender that loads a model from and uploads it to the open 3D model repository.

SketchUp’s license aims to prevent professional architects to earn their living without paying the SketchUp developers to earn their living. Hobbyists can still use it for ever for free (that was probably a condition by Google). This is important to know, but I’m not sure how this could be a problem for open building models that are distributed under a free license. Even if distributed under public domain and someone else using it for commercial products (navigation software), I don’t think it would conflict with the license of the software.
I’d like if SketchUp itself was open source (or a comparable alternative existed), but we would probably better go the way of the least effort, use existing wide-spread tools/editors and concentrate on adding the missing infrastructure (repository and viewer).

I think it’s a weak point of Sketchup. Someone creates a model with a free version of Sketchup and publishes it in the repository with a free license. Then someone uses the model for commercial work. In my opinion this contradict to the license of the free version of Sketchup.

As it was mentioned already a much more powerful and open source alternative to Sketchup is Blender.