Multi-lingual rendering of Thailand

stephankn · 2010-08-02 23:10:39

Hi,

some saw it already in the past, I thought it might be useful for all.

I have a map of Thailand that renders street names either in Thai script, in English or in a combination of both.

This could be useful for visitors to Thailand not being able to read Thai as well for mappers trying to complete the name:en and name:th tags.

http://thaimap.osm-tools.org/

Enjoy,

Stephan

Willi2006 · 2010-08-03 06:49:41

Looks great. Especially in Khon Kaen where almost all is mapped using name and name:en.

Some suggestions

Call the layers "Local language", "English" and "Bilingual".
This works also in other countries, e.g. Laos, especially Vientiane, which is perfect.
In the bilingual map:
- There shouldn't be parenthesis around the english name.
- Names of nodes and areas should be on top of each other.
- Names of ways follow already each other along the way but should be separated by a gap (not parenthesis).
- If name:en is the same as name it shouldn't be repeated, e.g. Malaysia.
May be a country using roman letters get rendered too, where only some major places are tagged with "name:en"

Thanks to webrian Laos provides also switching languages: Laos project map, under construction, use Firefox browser.

Hopefully this feature will be introduced for the map at openstreetmap.org.
This would motivate mappers to tag worldwide "name" in local language and also "name:en" if the local language doesn't use roman letters.
Google maps are already bilingual: Bejing, Google Earth partially. Being able to switch the language OSM would be a step further.

Last edited by Willi2006 (2010-08-03 06:53:12)

stephankn · 2010-08-03 23:32:40

Willi2006 wrote:

Call the layers "Local language", "English" and "Bilingual".
There shouldn't be parenthesis around the english name.
Names of ways follow already each other along the way but should be separated by a gap (not parenthesis).
If name:en is the same as name it shouldn't be repeated

Have done that. Hope that Mapnik does a good job in placing the label right. I also changed the rendering in a way that names that are not in local language (name=name:en) are not repeated in bilingual maps. Thanks for the hint. You might need to fore a reload to see the changes.

Willi2006 wrote:

Names of nodes and areas should be on top of each other.

This is an issue with the underlying data or with some Mapnik rules. Can you give an example? Often this is caused by people "mapping for the renderer". Eg by placing a node inside an building area because mapnik did not render some icon. As mapnik is later changed this results in some duplicate rendering.
Having an example should help identifying the problem.

Willi2006 wrote:

May be a country using roman letters get rendered too, where only some major places are tagged with "name:en"

I don't understand that point. Are you referring to Thailand? I don't think so. Please give an example.

Stephan

Willi2006 · 2010-08-04 15:47:23

stephankn wrote:

Have done that.

Thanks for the improvements. Looks much better now.

Willi2006 wrote:

Names of nodes and areas should be on top of each other.

Sorry, wasn't clear enough. I didn't mean duplicate names. In my opinion for nodes and areas it would look better to write in two instead of one line, e.g.:

 ขอนแก่น              โรงแรมพูลแมน
              and                             instead of   "ขอนแก่น  Khon Kaen" and  "โรงแรมโรมา  Roma Hotel"
Khon Kaen            pullman Hotel

stephankn wrote:

Willi2006 wrote:
May be a country using roman letters get rendered too, where only some major places are tagged with "name:en"
I don't understand that point. Are you referring to Thailand? I don't think so. Please give an example.

As far as I see only Thailand and parts of the neighbour countries are rendered. I think it would be interesting to see how e.g. the map of multilanguage countries like Switzerland or Luxembourg looks like having the same language layers.

stephankn · 2010-08-05 02:28:06

Willi2006 wrote:

Sorry, wasn't clear enough. I didn't mean duplicate names. In my opinion for nodes and areas it would look better to write in two instead of one line

This is a problem with Mapnik, that's the software used to render the map. I opened a ticked and talked a bit with the developers. I hope they get it working, then this can be done.

http://trac.mapnik.org/ticket/584

Willi2006 wrote:

As far as I see only Thailand and parts of the neighbour countries are rendered. I think it would be interesting to see how e.g. the map of multilanguage countries like Switzerland or Luxembourg looks like having the same language layers.

I can extend the bounding box. Actually it was only thailand at first. The bounding box is applied when updating the database. This can only handle rectangular bounds. I would have to clip everything outside the Thailand polygon from time to time.

I wanted to re-use the data to do some special rendering for Thailand especially missing names. In case I include too much of Laos, etc this will be rendered as missin Thai as well.

Please let me know if there is a need to see laos, etc as well.

Regarding European countries: The basic principle is the same. But you would not use name:en as the second language.

Stephan

stephankn · 2010-08-05 23:03:49

Good news, everyone!

After a longer chat session the Mapnik issue is resolved. The bilingual maps render the Thai names in one line, then the English names. Always assumed there are both languages and there is actually space on the map for the label.

Stephan

Willi2006 · 2010-08-06 05:07:37

Wow, that's really great.

Looks good at all zoom levels I tried in Khon Kaen. Many thanks to all who contributed to this enhancement. Only here and at zoom level 15 I found nodes rendered the old way, may be still old tiles: e.g. the nodes "Khon Kaen Railway Station", "Ban Ru Plaeng Mo", "Ban Su Pacha Ri", "Piman Buri", "Piman Place".

I'm looking forward to see language switching at openstreetmap.org too.

Could be extended to other areas using non-latin characters, i.e. most of Asia and also Europe including the large countries China and Russia. May be it's better to label the choices "Local names", "Romanized names" and "Local and romanized names" or "Combined names". It's not about language really. It's about letters. This would be even more accurate. It's only sometimes a translation and the romanization isn't done always according to English rules. In Thailand it was but is changing now. Making available and announcing this great feature to the international community would lead to contributions of mappers of other "non-latin countries" and motivate them even more to tag "name" and "name:en". "name:en" should be better called name:romanized. But obviously this discussion is stuck and didn't come to a result yet.

Last edited by Willi2006 (2010-08-06 05:10:54)

stephankn · 2010-08-06 09:51:47

I deleted all old tiles on the server. In case there is somewhere still the old style please fore a reload. They might still be in your browser cache.

In case there is a demand for it I can render more of SEA. For Laos there seams to be already a server providing maps. So maybe not needed. I'm open to requests.

name:en is the English name.

Fortunately most times this is the same as the romanized form of the Thai name.

I think on the naming discussion we agreed on that. Maybe it was not agreed but we just stopped talking

As the romanized form of a name can be generated by a program it's not needed to store this in the database.
My server is using linux. So I would need a program that can do the romanization on linux. I only have a windows tool. I will open another thread as this a different topic.

Stephan

stephankn · 2010-12-27 03:07:37

Christmas gifts for the community.

http://thaimap.osm-tools.org/ received some improvements:

The rendering style was updated to the most recent version available on the osm main site.
The area covered now includes not only Thailand but also complete coverage of Laos, Vietnam and Cambodia.

The bilingual map received further improvements:

It now uses a different font to improve readability of Thai names.
It is possible to zoom in one more level (Z=19) than the official map. This gives enough space on the map to show names of POIs in densely mapped areas.

Enjoy mapping...

Willi2006 · 2010-12-28 13:55:25

Thanks for the improvements.
With zoom=19 more housenumbers are shown, e.g. here.

amai · 2011-02-09 22:24:06

>Thank you< also from here!

Are there any plans to get that integrated in official sites (*.osm.org)? Somehow that should not be "hidden" but visible for anybody.
And that would make ppl much more relaxed about the famous question of what to write in name, name:en, int_name, etc. ;-)

amai

stephankn · 2011-02-09 23:36:36

amai wrote:

Are there any plans to get that integrated in official sites (*.osm.org)?
Somehow that should not be "hidden" but visible for anybody.

Please feel free to advertise this Thai map.

It will not be implemented on osm.org now or mid-term.
One of the tickets that suggested this and got rejected: http://trac.openstreetmap.org/ticket/2739

Wikimedia runs the toolserver that renders multiple language layers as an alternative source fur multi-lingual rendering. Slightly different approach than my site.

So right now we have our own rendering that even provides additional benefits over the osm site by extending to zoom 19

Stephan

Willi2006 · 2011-08-21 14:40:55

To check my mapping now I only use the bilingual map. It shows the names in two languages and the font is much better. Usually changes are rendered within minutes after submitted. I noticed two points.

At this location water on the left and wetland on the right side which are of about the same size are rendered at zoom level 13. At the zoom levels 12 to 9 only the water is rendered. Is this a fault or intentional?
Water and wetland, another example.

At that location I deleted a node "temple" at a wrong place in the lake and replaced it by a multipolygon "temple". The deleted node is still on the thaimap but not on openstreetmap anymore. I made several times a "dirty" request and the "status" got updated.
Temple on thaimap and on openstreetmap.

Last edited by Willi2006 (2011-08-21 14:45:04)

stephankn · 2011-08-21 15:43:16

Thank you for the notice. The server is using the osmosis minutely-replication system. Usually changes should appear within five minutes on the map. If it does not it could be that your browser or proxy server still has a cached copy of the old image. Try to open only the image and do a "shift-reload" to force a reload.

In case this does not help and also marking the tile as dirty on the server is not helping there is a problem on server side.

I will have a close look at your example to track down the problem.

Stephan

stephankn · 2011-08-21 17:59:55

Hi Willi,

I can confirm the database contained wrong data. That node was deleted Sun, 24 Jul 2011 20:53:37. According to the munin stats there had been an issue with the database update during that time. In theory it should not affect data consistency. Looks like practice proofs theory wrong.

I'm currently re-applying the updates of last month. Database should be up-to-date in a few hours.
Please let me know if you discover more problems. And don't hesitate to mention them earlier

Stephan

Willi2006 · 2011-09-10 03:10:01

There still seems to be a problem. I deleted the duplicate housenumber 555/95 and about one hour later I corrected the housenumber 555/101. At thaimap.osm-tools.org the correction is reflected but the deletion isn't. At openstreetmap.org both are reflected. Several "/dirty" up to 40 minutes after upload didn't cure the situation.

stephankn · 2011-09-11 13:47:40

Willi2006 wrote:

the correction is reflected but the deletion isn't.

I was able to reproduce this. It's all because of a bug introduced in osm2pgsql in end of may. When using it together with a bounding box (I'm doing it) this leads to deleted nodes not correctly replicated.

I have now two options. One would be to replay all deletions from June onwards, the other a complete re-import of the database. The replay could work and be faster but it would miss nodes being deleted and later restored. Not sure how many of these would be in Thailand.

So best option certainly is a re-import. With a re-import I can also switch to the database scheme that works without intarray. This is the system used by the main branch of osm2pgsql.

Please expect a stop in tile-rendering the next days. I'll announce it on the main page once I start the re-import and once I'm done with it.

Sorry for any inconvenience and thanks to Willi for reporting this problem.

Stephan

Willi2006 · 2011-11-10 15:25:23

This node landuse=commercial is shown on openstreetmap.org at zoom 15 to 18.
On thaimap.osm-tools.org it is shown at zoom 15, 16, 18 and 19, but not at zoom 17. The status was okay: "Tile is clean. Last rendered at Thu Nov 10 14:00:44 2011". A "dirty" 19 minutes later didn't cure it.

Last edited by Willi2006 (2011-11-10 15:26:51)

stephankn · 2011-11-24 23:26:01

Willi2006 wrote:

This node landuse=commercial is shown on openstreetmap.org at zoom 15 to 18.
On thaimap.osm-tools.org it is shown at zoom 15, 16, 18 and 19, but not at zoom 17. The status was okay: "Tile is clean. Last rendered at Thu Nov 10 14:00:44 2011". A "dirty" 19 minutes later didn't cure it.

It's a bit strange. I can only guess what is going wrong here.

I suppose it's related to mapnik's painter algorithm. It placed already the highway shield. So not enough space to render the name of the node. Using landuse on single nodes is uncommon, but unrelated to the problem.

Mapnik could be clever to move the shield a bit, but I think that is not yet implemented. At SOTM-eu there was a talk from someone of the mapnik team who is working on such smarter label placement.

So sorry, you have to live with things like this until mapnik is improved.

Stephan

keng · 2012-02-23 11:54:22

I see some wrong rendering with thai vowels here: http://thaimap.osm-tools.org/?zoom=18&l … ayers=B00T

"สีวลี 01 Siwalee 01" can be a reference for correct rendering, but the rest from "สีวลี 03 Siwalee 03" to "สีวลี 10 Siwalee 10" vowel _ี (Sara Ee) is misplaced. _ี (Sara Ee) should be normally placed above ส (So Suae) and ล (Lo Ling) to make "สีวลี" readable.

Keng.

stephankn · 2012-02-23 15:21:30

Hi Keng,

i know this is unsatisfactory. But unfortunately this is a known problem with mapnik, the software usually used to render maps with OSM data. Besides the osmarender maps (TAH) all OSM maps I am aware of suffer from this problem.
so on my map there is nothing I can do to fix it.

If you know someone who is familiar with C++ programming and could work on this topic it could speed up a bugfix.
https://github.com/mapnik/mapnik/issues/112

Stephan

Willi2006 · 2012-02-23 15:34:30

Hello keng,

welcome to the OSM community and the forum. Glad to see one mapper more mapping names in Thai.

The "_ี" below the consonant instead above is quite strange. I mapped already quite a lot and never saw this. Hopefully not to tough for Stephan to fix.

I see you prefer another method of romanization: e.g. "_ี" becomes "ee". As described on the page WikiProject Thailand I'm using RTGS and "_ี" becomes "i". This comes closer to my mother language German and to IPA (International Phonetic Alphabet). On signs I saw both versions even on different signs of the same village or street.

In addition I noticed the tagging of name:th and int_name. I stopped using name:th where it's just a duplication of name. It adds to the database size. I learned that it's quite easy for programs to recognize Thai due to the different charcter set. See e.g. mkgmap. For Thailand there's an OSM map available with weekly updates on Free routable maps for Garmin brand GPS devices which I use on a Garmin phone and with the Windows program MapSource.

"int_name" I'm tagging only if it's different from "name:en", e.g. "int_name=Bangkok" for "name=กรุงเทพ".

Happy mapping
Willi

Last edited by Willi2006 (2012-02-23 15:38:07)

stephankn · 2012-02-23 16:24:56

Hi Willi,

Database size is nothing to really worry about. We just have a few thousand names. That impact is neglectable.
I still use name:th as from time to time some "clever" people optimize the map on osm.org by replacing Thai script with English names.
with name:th i still have an easy way to automatically detect these cases.

If we agree someday in the future we no longer need it it just takes me 5 minutes to get rid of them by using an automated edit.
Currently it does not harm.

int_name is deprecated. Better use name:en. Only for special cases it might make sense as Willi already explained.

Some people are discussing to distinguish English names and transliterations. Maybe this is too much for now.
I hope my Thai will be better some day so I can read Thai names without problems. Until then the current name:en is helping a lot already.

For a map of Thailand having a Thai script name should be the primary goal.

If a tourist map is showing "Sukhumvit Road" or "Thanon Sukhumvit" is just a small issue without any practical impact on a tourist. They could use both to find their way.

Stephan

Willi2006 · 2012-09-27 15:34:25

The map on openstreetmap.org shows the actual section of the border between the provinces Khon Kaen and Chaiyaphum which is the Choen River, changed on September 15th. I recognised today that the Thaimap still shows the old version. But the render status is of today.

Last edited by Willi2006 (2012-09-27 15:35:13)

stephankn · 2012-09-29 20:24:52

Willi2006 wrote:

The map on openstreetmap.org shows the actual section of the border between the provinces Khon Kaen and Chaiyaphum which is the Choen River, changed on September 15th. I recognised today that the Thaimap still shows the old version. But the render status is of today.

As noted on the main page I'm in the process of updating all software involved to current versions as well as reimport ODbL to meet legal requirements. There had been software crashes of the upgrade path due to mismatches of the involved tools. I thought I had fixed it, but obviously something stayed broken. I now re-imported all changesets from 14th onwards.
That should have repaired most.
In the near future I hope to be able to upgrade the database as well. It's also needed because we will run out of node ids that fit in 32bit in the next months. After this upgrade all tiles will be re-rendered to comply with legal requirements.

Sorry for inconvenience. Please report any further inconsistencies after the completion of the upgrade.

Stephan

OpenStreetMap Forum

#1 2010-08-02 23:10:39

Multi-lingual rendering of Thailand

#2 2010-08-03 06:49:41

Re: Multi-lingual rendering of Thailand

#3 2010-08-03 23:32:40

Re: Multi-lingual rendering of Thailand

#4 2010-08-04 15:47:23

Re: Multi-lingual rendering of Thailand

#5 2010-08-05 02:28:06

Re: Multi-lingual rendering of Thailand

#6 2010-08-05 23:03:49

Re: Multi-lingual rendering of Thailand

#7 2010-08-06 05:07:37

Re: Multi-lingual rendering of Thailand

#8 2010-08-06 09:51:47

Re: Multi-lingual rendering of Thailand

#9 2010-12-27 03:07:37

Re: Multi-lingual rendering of Thailand

#10 2010-12-28 13:55:25

Re: Multi-lingual rendering of Thailand

#11 2011-02-09 22:24:06

Re: Multi-lingual rendering of Thailand

#12 2011-02-09 23:36:36

Re: Multi-lingual rendering of Thailand

#13 2011-08-21 14:40:55

Re: Multi-lingual rendering of Thailand

#14 2011-08-21 15:43:16

Re: Multi-lingual rendering of Thailand

#15 2011-08-21 17:59:55

Re: Multi-lingual rendering of Thailand

#16 2011-09-10 03:10:01

Re: Multi-lingual rendering of Thailand

#17 2011-09-11 13:47:40

Re: Multi-lingual rendering of Thailand

#18 2011-11-10 15:25:23

Re: Multi-lingual rendering of Thailand

#19 2011-11-24 23:26:01

Re: Multi-lingual rendering of Thailand

#20 2012-02-23 11:54:22

Re: Multi-lingual rendering of Thailand

#21 2012-02-23 15:21:30

Re: Multi-lingual rendering of Thailand

#22 2012-02-23 15:34:30

Re: Multi-lingual rendering of Thailand

#23 2012-02-23 16:24:56

Re: Multi-lingual rendering of Thailand

#24 2012-09-27 15:34:25

Re: Multi-lingual rendering of Thailand

#25 2012-09-29 20:24:52

Re: Multi-lingual rendering of Thailand

Board footer