Hi everybody

Hi all,

At first : congratulations for all your work

I began a similar project, a few months ago, with exactly the same goals, but I did not searched so much on the net to see if that kind of project was already existing. Yours is well advanced!

I’m ready to start developping with you. Basically, what I developped (to give you my skills) is :

  • A php/mysql map data server that communicates with editor in XML
  • A C++ Qt editor, in openGL.

The thing is pretty robust and scalable, but pretty simple yet. It supports full server side editing history & undo.
Have a look at the editor:
http://www.galinette.org/Editor.jpg

So a suggestion for helping is developping a C++ Qt (Cross platform) opensource editor, maybe more optimized than java or online editor

One other question : When I developped my server, I tried several schemes for the database, and measured the query performances for retrieving the map in a rectangle region. My observations were:

  • Using the Mysql Spatial extensions is MUCH faster on large maps, as mysql optimizes the region queries with bidimensionnal trees on bounding box.
  • Using a single table for storing ways (each record is the full way object with node coordinates) avoids a lot of id lookups, which slow down the region request a lot. One single query provides all the data in the given rectangle, in a very short time, as all is properly indexed.

I tried this with a large random map and this improved MUCH the server performance. The overall table size (including index) was not higher.

So maybe I could try that scheme with your whole table data (blarger than mine) to compare performance. Maybe this could suggest database evolutions for the future.

And last question, is there a “map compilator” that will generate compact binary maps of a region for use in handheld devices, for instance route navigators on PDAs?

Any other ideas welcome!

Pleased to join,

Etienne

Hi again,

By reading further the forum, it seems that server request speed is really a concern.

I am very motivated in trying the scheme described in my previous post with your database.

So I should start by downloading a full recent dump of the base, and write a converter to my database scheme. I can try setting up a server at home with it and play with my editor to see if requests are fast. After this I open the connection and give the editor to a few of you to test it (I only have a DSL line so network will be the bottleneck if many users try it). Afterwards, if you find it interesting, I would be glad helping you porting the DB to this scheme.

Cheers,

Etienne

Welcome to OpenStreetMap Etienne!

Lots of questions and remarks that you’ve made are perfectly viable and have been asked and suggested before. One of the biggest ongoing discussion is using the special spatial additions to databases like MySQL or (some would rather suggest) PostgresSQL/PostGIS. So far it has proven to be difficult to get rid of the old, plain and simple MySQL database which has brought this project to where it is today by pure simplicity. Though we have gained some important speed enhancements by using a special quadtile index, we can still benefit from any improvements that make this project even better. Your input and skills are very welcome.

You will need to have strong reasons and the hard numbers to back them up to really change things because of the ‘proven’ state of the current server setup. Lots of capable people have looked at it so this might not be as easy as it might seem right now. Moving to another platform on a project this far developed as OpenStreetMap is not an easy task. It involves not only the DB or it’s schema but also the tools that work with it, making a big change is a humongous task. But don’t let this stand in your way to try to make things better.

About the editors: OpenStreetMap actually has three editors (and maybe more in development), Merkaator being the least well known but written in C++ AFAIK. Personally I would love to see a capable editor that runs decently on PDA’s or something like the Nokia N800/N810 internet tablets (I know others would love this too).

Some folks are working on a mobile (binary) map exchange format for OpenStreetMap. Others are working on route navigators like Gosmore and Pyroute.

The best way, currently, to go head to head with other developers is to join the dev mailing list and to join the irc channel for quick exchange of ideas.

Galinette, I hope you stay on and keep developing your editor to work with Openstreetmap. I like the way you render one-way streets.

Hi again,

I dropped my own map db developpment and started to adapt all my work to osm.

I currently spend a couple of hours a day working on this now. The good thing is I can try my code on the big osm data files, it behave quite well on country files. I will try soon the whole database but I am quite limited by my server capacity now… I only have 60GB left for it! I need a new harddisk.

I have succesfully imported some osm dumps into my database scheme, and started to modify the client to work with your tag system. It works and the map updates quite smoothly as the users scrolls and zooms. I am now working at more streaming for the map downloading, to increase speed and reactivity. I am also working at compressing the data transfers (XML compresses well!)

I will probably have something well advanced by end of february to let you test

Regards,

Etienne

There is already a c++ qt-based editor, Merkaartor. Maybe you could collaborate on development with the author?

Yes, I like their editor. It’s based on the same framework that the code I already wrote (Qt) so this is good.

Etienne

Hi again,

I have finished a test server using mysql/GIS and loaded a full planet.osm dump.
I have also written a simple map editing client with Qt (same framework as Merkaartor, compilable under Win/Linux/macOS). You can test the windows build by downloading it here:
http://www.galinette.org/Mapiki_Editor_OSM.zip

I will prepare a clean source package for people wanting to run it under linux or MacOS

If you want to submit map change (vie the synchronize function) you must register here:
http://montparnasse.galinette.org/mapiki_test/register.php
Then enter your email and password into the client network preferences

Feel free to test the app & server, just be soft with the server : it’s my home desktop PC behind my home DSL connection, and my poor 60€ hard disk :slight_smile:

Currently the client app is quite slow, especially when rendering large areas or large number of roads. I would be interested in adapting Merkaartor to this server scheme as its rendering is faster.

Regards,

Etienne

Nobody interested by this??

This is a possible answer to something proposed on the wiki :
“Database geospatial investigation”
in
http://wiki.openstreetmap.org/index.php/Student_projects

The database is working, as well as a simple client to test it!

I have also something well advanced dealing with the “Wiki-like control” request, are you still interested in this?

I can make a demo db+client integrating this if there is some interest.

Etienne

Seeing a working solution to wiki like history control is very interesting… :wink: ATM it’s very hard to get history for an area/user in an efficient way, and the dumps doesn’t contain history so it’s also hard to do it on your own server.

That’s only working on changes done on my server.
GIS extensions with spatial indexes in mysql are very helpful here , especially to extract an area history.

But really, do someone here really cares experimental developments on the server that are listed in the wiki? I’m not claiming to change your db code in two weeks, but I did some trials on the server structure which:
1)Experiment the geospatial extensions of mysql
2)Increase map download speed a lot (try it by yourself)
3)I give any info you want (code, scheme, …) and let it open so that it may be useful to osm one day
4)I have put a test server online for you to test

It seems that nobody cares, which seems discouraging.

About the wiki-like history, I really have something well advanced, believe it or not. So I really would like to know if somebody cares here, because adapting it on OSM data would still require some development time which I’m going to use at it only if there is some interest.

If not, what should be worth working on (I’m quite motivated if I can help)

Regards,

Etienne

I’m not the guy to tell you what to do, you would only do stuff that I want, it’s very tempting but… :wink:

The first thing to know is that the hackers all hang out on the OSM dev mailng list, so you won’t get much discussion about it here. These people are a lot more into OSM than I am atm, though I’ve touched the code back in 2005-2006 I’m far from proficient in the performance issues at this date… So to have nice discussions about speed, publish your code, and go there.

I’m not sure, where you should start, there is so much code that need to be written in openstreetmap. Just doing what you are doing atm might also help, showing people that you can solve things in other ways. I find that reimplementng stuff usually gives birth to new cool ideas, sometimes it might be enough to replace the existing project other times just gives some new ideas.

Or just helping out and improving the code that is already there…

I hope you continue with this, there is always need for more code and that you have…