Creating a Java Application

moj · 2008-04-07 18:59:36

Hi everybody,

i'm new here and very interested in the osm project and i need some advice:
i have to write a java application using osm. it's supposed to extract the coordinates from an *.osm file (e.g. a city) and display the roads
in a window. it should also have a simple routing function, e.g. a car (displayed as a dot) is going from one point to another on the streets.
i don't want any java code, i just would like to hear some ideas how this could be realized. please keep it simple, because i'm not quite familiar with osm xD

emj · 2008-04-07 22:41:44

The .osm format is pretty easy if you just want to display roads, you just need to read the section on how the WAY and NODE works. Just skip areas, coastlines and relations they are a bit cumbersome.

So you will easily be able to parse a file, get all "nodes", and one "way" and then draw a Line Vector from the nodes in that way. You only need to draw ways that have the tag k="highway". With that you should have a pretty nice application working.

You can get a pretty small example area from:
http://www.openstreetmap.org/api/0.5/ma … 18.0,59.31

Routing I know nothing about.

Was that too simple?

Coffeejolts · 2008-04-08 20:45:23

I am interested in doing much the same thing- real time offline rendering of OSM maps. I have lots of Java experience, but no experience with OSM.
Is it possible to do real time rendering using osmarender? What sort of strategy would I need to take in order to render less detail as the map is zoomed out (thus increasing speed)? And, even if I change the detail level, will osmarender be fast enough (single user)?

Ben · 2008-04-09 01:29:15

Osmarender is painfully slow, which is something mapnik has over it as it's lightning quick. Reducing detail in osmarender just involves opening the rule sheet and deleting each rule you don't want. Really simple..even I can do it. Sorry if I misunderstand the question though, but that's my quick answer.

emj · 2008-04-09 06:47:35

Osmarender is a great thing sadly, but what Ben says is very true it's slow as molasses molasses(which can be fast, but that's not important). Well to get it to render faster you are going to have to remove some data from the .osm files, simplifying vectors (ways) by removing some nodes from them. This can for example be done with a Douglas-Peucker polyline reduction, there is code in OSM implementing this.

I'm not sure you can ever make XSLT, that's the thing Osmarender uses, fast.

Coffeejolts · 2008-04-09 14:21:56

Great information, thanks for responding so quickly. It looks like I'll need to write my own rendering code to do this right. That's what I figured, but I didn't want to re-invent the wheel if there was already a fast cross platform way to accomplish it.

Lambertus · 2008-04-09 16:34:08

Have a look at Mapnik or Kosmos for fast rendering. Kosmos being my favorite when it comes to offline rendering of small areas.

Coffeejolts · 2008-04-09 16:48:00

yes, I have looked at both, but neither can be bundled inside a standalone java application. For a web app, I could use them.

Harry Wood · 2008-04-09 17:21:18

I've been pondering this myself. The Renderer wiki page should list such applications, but I'm not convinced that's a full list.

There's Mkgmap, which is pure java aparently. It's a renderer, although it's specifically for creating garmin image sets.

I like the configureability of Kosmos, and idea of sharing rendering rules on a wiki page is quite nifty. Just a shame it's written in .NET really. If it was in java, I think it would attract more community interest. Also it could be bundled into JOSM plugin, meaning you could render your current JOSM view with the click of a button. JKosmos anyone?

Speaking of JOSM, there's code in there which, used to be called the Mappaint plugin (before it got assimilated into JOSM core). This is kind of a renderer . It's not exactly a nice polished map, because the output is desgined to be clicked on and edited, rather than viewed as a map, but it's configureable (elemstyles.xml) and I guess its pure java drawing to the graphics object in realtime. You can see by zooming around, that it must be doing this pretty fast, though less fast if you're loading a lot of data in.

Perhaps with a bit of java development we could start to blur the lines between editor graphics -> rendered map image -> slippy map tile images so that rather than figuring out many seperate tools, you can start to do these things with a click of the mouse within JOSM.

I know a fair bit of java, so really I need to stop pondering and start coding.

emj · 2008-04-10 08:39:54

Coffeejolts wrote:

But I didn't want to re-invent the wheel if there was already a fast cross platform way to accomplish it.

I read your post as if you wanted to do it all yourself.. :-) Hmm.. Need to work on my reading comprehension.

I agree with Harry; have you looked at JOSM? It has OKish rendering, and has already solved all those problems with projections and all that. And it's a software that can probably use another hand.

IMHO, the best would be to be able to download the slippy map pngs and render blue routes over that...

Last edited by emj (2008-04-10 08:42:19)

Coffeejolts · 2008-04-10 20:41:40

emj wrote:

IMHO, the best would be to be able to download the slippy map pngs and render blue routes over that...

I've tinkered with that- I actually wrote a little java app that downloaded all the tiles within a bounding box. After a while, I started to get worried about slamming the server, so I stopped at zoom 8. It's certainly easier to implement- since there is already a Swing component for viewing OSM maps on swinglabs.org. Then, I could just draw routing information or POI on the map using java2d.

Does the OSM community have a problem with the tiles being downloaded in the way I described? I didn't want to chew up all your bandwidth if you were paying by the Gb or something.

emj · 2008-04-11 22:13:11

Well... I think we have free Bandwidth from VR.UCL at the moment, but if someone decides to slam the servers...

Be moderate, I'm mean you probably won't need to download more than a GB.. (I remember fretting over downloading 50KB) :-)

emj · 2008-04-11 23:25:40

if you want to do the slippy map download here are some posts from the maiinglist.

Jon Burgess wrote:

On Fri, 2008-04-11 at 16:09 +0200, Frederik Ramm wrote:
> Hi,
>
> > Hang on - I doubt you've got enough disk space to do that. That's
> > terabytes of space you'd need.
>
> I have a neat little Perl script that tells you some details about the
> various zoom levels. These are *theoretical* values, I'd be interested
> to hear from operations. Here's the output (values are colon separated):
Here are the stats from tile.openstreetmap.org for each zoom level. This
includes every tile which has ever been looked at on the Mapnik layer in
the last 6 months or so.
zoom    tile count      limit           % of limit
--------------------------------------------------
0       1               1               100
1       4               4               100
2       16              16              100
3       64              64              100
4       256             256             100
5       1024            1024            100
6       4096            4096            100
7       16384           16384           100
8       65536           65536           100
9       262144          262144          100
10      1048576         1048576         100
11      2190208         4194304         52.22
12      2521856         16777216        15.03
13      4391616         67108864        6.54
14      8242176         268435456       3.07
15      13675904        1073741824      1.27
16      12613952        4294967296      0.29
17      23811328        17179869184     0.14
18      14342784        68719476736     0.02
-------------------------------------------------
Total   83187925        91625968981     0.09
As you would probably expect, we have every tile for the low zooms and
then the percentage of tiles drops off rapidly past zoom 10. We
currently have about 1/1000th of all the possible tiles.
These are taking 120GB of disk space.

Frederik Ramm wrote:

Hi,
> Hang on - I doubt you've got enough disk space to do that. That's
> terabytes of space you'd need.
I have a neat little Perl script that tells you some details about the
various zoom levels. These are *theoretical* values, I'd be interested
to hear from operations. Here's the output (values are colon separated):
Zoom Tiles       Tiles cumul Bytes    Bytes cum Tile width  Pixel is
0    1           1            5 KB    5 KB      40075 km    157 km
1    4           5            20 KB   25 KB     20038 km    78 km
2    16          21           80 KB   105 KB    10019 km    39 km
3    64          85           320 KB  425 KB    5009 km     20 km
4    256         341          1 MB    2 MB      2505 km     10 km
5    1024        1365         5 MB    7 MB      252 km      5 km
6    4096        5461         20 MB   27 MB     626 km      2 km
7    16384       21845        82 MB   109 MB    313 km      1222 m
8    65536       87381        328 MB  437 MB    157 km      611 m
9    262144      349525       1 GB    2 GB      78 km       305 m
10   1 million   1 million    5 GB    7 GB      39 km       152 m
11   4 million   6 million    21 GB   28 GB     20 km       76 m
12   17 million  22 million   84 GB   112 GB    10 km       38 m
13   67 million  89 million   336 GB  447 GB    5 km        19 m
14   268 million 358 million  1 TB    2 TB      2 km        9 m
15   1 billion   1 billion    5 TB    7 TB      1222 m      4 m
16   4 billion   6 billion    21 TB   29 TB     611 m       2 m
17   17 billion  23 billion   86 TB   115 TB    305 m       1 m
18   69 billion  92 billion   344 TB  458 TB    152 m       0 m
I.e. even if you only go down to zoom level 16, you should expect to
store around 30 Terabytes of data.
The script is attached, for anyone who wants to tweak the assumptions
(most notably the assumption that a tile has 5 kb on average).
Bye
Frederik
-------------------------
#!/usr/bin/perl
print "Zoom level:Tiles:Tiles cumul.:Bytes:Bytes cumul.:Tile width
is:Pixel is\n";
# earth
$circum = 40075160;
# this number has the "70% water" figured in, i.e. of 100 possible
# tiles, only 30 will have non-water content, and if those average 16kb,
# then we have a total average of 5 kb per tile
$average_kb_per_tile = 5;
foreach my $zoom(0..18)
{
print "$zoom:";
$z=4**$zoom;
$let="";
if ($z>1000000) { $z=int(($z+500000)/1000000); $let = "million";
if ($z>1000) { $z=int(($z+500)/1000); $let = "billion"; }}
print "$z $let:";
$kum += 4**$zoom;
$z=$kum;
$let="";
if ($z>1000000) { $z=int(($z+500000)/1000000); $let = "million";
if ($z>1000) { $z=int(($z+500)/1000); $let = "billion"; }}
print "$z $let:";
$mem = (4**$zoom)*$average_kb_per_tile;
$kummem += $mem;
$let = "KB";
if ($mem>1000) { $mem=int(($mem+500)/1000); $let = "MB"; }
if ($mem>1000) { $mem=int(($mem+500)/1000); $let = "GB"; }
if ($mem>1000) { $mem=int(($mem+500)/1000); $let = "TB"; }
print "$mem $let:";
$let = "KB";
$mem=$kummem;
if ($mem>1000) { $mem=int(($mem+500)/1000); $let = "MB"; }
if ($mem>1000) { $mem=int(($mem+500)/1000); $let = "GB"; }
if ($mem>1000) { $mem=int(($mem+500)/1000); $let = "TB"; }
print "$mem $let:";
$cw = int($circum/(2**$zoom));
$let="m";
if ($cw>2000) { $cw=int(($cw+500)/1000); $let="km"; }
print "$cw $let:";
$cw = int($circum/(2**$zoom)/256);
$let="m";
if ($cw>2000) { $cw=int(($cw+500)/1000); $let="km"; }
print "$cw $let\n";

Last edited by emj (2008-08-21 00:23:21)

Ben · 2008-04-12 10:12:24

hmm, if we currently have about 1000th of all possible tiles taking 120GB, then I wonder what percentage in size those tiles are, since I imagine the first 0.1% of tiles which are done are where most people live, and vice versa. So tiles that will just be solid yellow in the sahara, or solid green from the Amazon will be a fraction of the file size, and huge in number. (Does Frederick's estimation already count for this?)

That leaves me thinking; I wonder, at some point do you think if there is just a huge area of thousands/millions of 'yellow' squares in a clump, isn't there a more efficient way of storing this than each tile?. Not only are they the same for all zooms, bar tile size, but also the same as all the neighbouring tiles.

Plus: this Z18 seems odd. I thought there were 17..?

emj · 2008-04-12 13:39:25

Btw there is a limit of how much you can download: http://trac.openstreetmap.org/browser/s … /limit.png

Ben: I don't think they do that optimization yet, but I might be wrong. You can always look at the mapnik rendering code if you want.. ;-

Last edited by emj (2008-04-12 14:03:17)

MDave · 2008-04-16 04:30:57

Hi Everybody,

New guy here. I am looking for an cross-platform, open source application that will render maps from the planet.osm file and compute routes. Kind of like Microsoft Streets and Trips, but free (as in freedom).

My take away from this thread is that this application does not exist. The closet thing is Kosmos which is written in .NET, is not GPL'd and seems to have a hard architectural limit due to its use of memory (apparently it was written to be a renderer to run beside JOSM).

An alternative starting point would be to use some of the internal code of JOSM.

Am I reading this right? BTW -- I found another application called RoadNav that gave me high hopes, but just didn't cut it...

Last edited by MDave (2008-04-16 04:31:30)

emj · 2008-04-16 19:30:37

Hi Dave, it's out there you just have to believe. (e.g. in Aliens or Pyroute (also on pyroute on the wiki) :-) I haven't tested any of the routing applications, I can't remember their names but there are a couple of options. Please do some research and tell us what works for you..

There are several problems with offline and derived use, mostly there is no good implementation of a killer app.. I think once we have someone who have built a good model, you are going to see a lot of other copy cats coming. Just handling the 1GB OSM map data that is Sweden is hard to do atm, and that's without displaying it.

Maybe there is a really easy way to do it, I just haven't seen it yet. But you gotta believe that it works, and just do it.

MDave · 2008-04-17 00:39:43

emj wrote:

Please do some research and tell us what works for you.

Thanks for the lead on pyroute, I'll check it out. I shall report back on the state of the tools out there...please stay tuned.

bruce89 · 2008-04-17 00:47:34

MDave wrote:

The closet thing is Kosmos which is written in .NET, is not GPL'd and seems to have a hard architectural limit due to its use of memory (apparently it was written to be a renderer to run beside JOSM).

The source is available under some kind of BSD style thing, but there is no VC access.

I'd write something nice in Vala, but I'm not good enough yet.

Last edited by bruce89 (2008-04-17 00:48:53)

emj · 2008-08-21 00:18:16

bump I like this thread. :-)

Announcement

#1 2008-04-07 18:59:36

Creating a Java Application

#2 2008-04-07 22:41:44

Re: Creating a Java Application

#3 2008-04-08 20:45:23

Re: Creating a Java Application

#4 2008-04-09 01:29:15

Re: Creating a Java Application

#5 2008-04-09 06:47:35

Re: Creating a Java Application

#6 2008-04-09 14:21:56

Re: Creating a Java Application

#7 2008-04-09 16:34:08

Re: Creating a Java Application

#8 2008-04-09 16:48:00

Re: Creating a Java Application

#9 2008-04-09 17:21:18

Re: Creating a Java Application

#10 2008-04-10 08:39:54

Re: Creating a Java Application

#11 2008-04-10 20:41:40

Re: Creating a Java Application

#12 2008-04-11 22:13:11

Re: Creating a Java Application

#13 2008-04-11 23:25:40

Re: Creating a Java Application

#14 2008-04-12 10:12:24

Re: Creating a Java Application

#15 2008-04-12 13:39:25

Re: Creating a Java Application

#16 2008-04-16 04:30:57

Re: Creating a Java Application

#17 2008-04-16 19:30:37

Re: Creating a Java Application

#18 2008-04-17 00:39:43

Re: Creating a Java Application

#19 2008-04-17 00:47:34

Re: Creating a Java Application

#20 2008-08-21 00:18:16

Re: Creating a Java Application

Board footer