Daily extract of Thailand

I have been away from mapping for some time and have recently got back into it, having acquired a new KLX-250 and some spare time :slight_smile:
I used to be able to do a WGET on the whole of Thailand and make my own maps with relative ease.
The API now seems to limit each request to a much smaller area, and is often offline/overloaded anyway.
This ofc makes it impossible to make my own up to date maps (by up to date, I mean at least daily)
I can’t find any site that offers daily updates for Thailand, best I have found is Cloudmade, which as of today was updated Aug-26th.

I could download all the daily diff files and use osmosis to update my local copy of thailand.osm, but that seems very wasteful.
I don’t really want to download the entire planet either, I’m only interested in TL at the moment.

Any have any suggestions about the best way to go about it?
Does anyone know of a daily updated thailand.osm hosted anywhere?
If not I may have to bite the bullet and do the daily diffs (or wait until cloudmade update theirs) and I could host that myself, once complete.

@Stephan
You must have gone through this process at some point for your bilingual site.
How did you get the inital data?

Cheers for now.
Dui

The non-availablity of an extract of Thailand is really a bad thing. I use to download asia.osm.bz2 from Geofabrik:
http://download.geofabrik.de/osm/
It’s 1.9 Gigabytes…

Thanks Bernhard.
In the end I downloaded all the daily diffs and brought my local copy from Cloudmade up to date.
It’s here if any one is interested, I will try to update daily.
www.mahleeproperty.com/osm/thailand.zip

Cheers for now.
Dui.

All the Cloudmade extracts I have ever downloaded had all lines missing that cross the map boundary. That means that on district maps practically all the major roads have at least pieces missing. I told them about it and was told that it had to do with relations.

Have you checked the files you have downloaded?

Haven’t noticed any problems, but rarely go out of Chiang Mai.
Did do a visa run recently to Mae Sai (which is still probably within the main boundary) and didn’t notice any problems.

Thanks for the heads up though, will check it now.

Dui.

Cloudmade has boundaries that seam to be far away from the real boundaries. And they update extracts very sporadic.

The bilingual maps contain a lot more than Thailand. They cover full Laos, Cambodia and Vietnam. It’s a database for rendering. So not the data you want to have.

If you need updates just from time to time I might be able to upload the extracts I randomly extract from the full planet. This is done manually when I feel the need of an update.
For 2011 I have 107 extracts. That’s one extract about every 3rd day with gaps during the times I’m in Thailand.

We might ask Frederik to include it in his extracts. If his server has the capacity the would be probably the most convenient option.

Let me know if I can help with extracts. I uploaded one for you here:
http://downloads.osm-tools.org/thailand-20111121.7z

Stephan

Thanks Stephan, I have been too busy lately to update my local copy.
I think it’s really odd that there is no way to download just the area you need rather than the whole planet, but I guess someone somewhere made that decision for a good reason.
PITA though!

Is there any concrete reason why geofabrik.de has still no download for raw OSM data from Thailand? and maybe more missing countries?

Has anyone already asked Frederik from geofabrik.de?

I ust the ‘alternative downloads’ plugin in JOSM. Draw a box any size you want and hit download. The resulting .osm I just drag and drop onto a bat file, which creates my map set. Great when I do a lot of mapping in the same area. I can see in MS what I need to cut up and add and what I have already done.

Where is that plug-in available from?
It is not in my list within JOSM of available plug-ins…

On mine it is called mirrored_download and once installed you invoke it with shift-alt-down.

Got it, thanks, seems to work a treat :slight_smile:

Hmmm I got excited too quickly.
If I use the plug-in to download a small chunk (but still bigger than would normally be possible) everything is fine.
But if I d/l a large area, lots of data is missing, notably all roads and ways.
Any ideas?

Yes, I had also such a problem. It seems to exist a limit - I guess 1048576 objects - and the plugin downloads all nodes first and then later the ways etc.
So when you try to download too many objects, the process aborts and your result in JOSM maybe will contain only nodes.
I propose you should split your download into partial chunks. You can do it eighter with the longitude/latitude range.
Or you could use search conditions. Maybe download unclassified streets first, then all other streets, then landuse, then …
http://wiki.openstreetmap.org/wiki/Overpass_API/Language_Guide .

An explicit size limit does not exist, but the download is aborted by the server after three minutes. You can prolongate the limit: Please see
http://wiki.openstreetmap.org/wiki/Overpass_API/Overpass_QL#timeout
However, JOSM usually has crashed far earlier than the three minute limit, at about 100 MB of raw data. For that reason the plugin doesn’t extend the timeout.

But you can download the whole Thailand directly

wget -O thailand.osm "http://overpass-api.de/api/interpreter?data=area['name:en'='Thailand'];(node(area);<<;node(w););out meta;"

This should take, depeding on your bandwidth, about a minute and produce a file of about 700 MB size.

With extended timeout (3600 seconds = one hour), for slower connections:

wget -O thailand.osm "http://overpass-api.de/api/interpreter?data=[timeout:3600];area['name:en'='Thailand'];(node(area);<<;node(w););out meta;"

Thanks for the replies.
I get the same problem using either of the wget command lines you posted, will play further.
My file was around 390Mb, will try a longer timeout…

I think I offered it before: I can provide extracts of Thailand in xml or pbf format. Just not (yet) automated on a server.

The consumers of that data are that rare that this overhead is not justified.

May I ask what the purpose of the data download is? Maybe there is a better way to get the same result…

Stephan

For me personally I just download the parts I’m working on, but I now just use smaller parts and that works fine. When I travel in Thailand again I will just use Lambertus’ server.

Stephan,
I use the data to create my own up-to-the-minute maps.
I often go off-road biking where some of the trails are not mapped at all.
Lambertus doesn’t (or didn’t, I haven’t checked lately) update Thailand all that frequently.
Good news that geo offers TL now, will check that out at the weekend.
Hopefully that’ll be a daily, or better.

Cheers
Dui