what is mod_tile / renderd / PostgreSQL optimal configuration?

General talk
1

Hi. I’m installing the OSM server for my organization - not big deal, less than 500 users. Nominatim is incredibly fast, but the tiles service is killing me.

I’m a developer, not a DBA, but I noticed things got better after configuring PostgreSQL as recommended here: https://wiki.openstreetmap.org/wiki/PostgreSQL

I’m making cache of tiles with

render_list

, up to some level for whole world and deeper for known bounding box of interest in my organization.

My virtual server: 16 vCPUs , 64GB RAM and fast disk with 4TB, half of it occupied by Tiles and Nominatim databases . All the world was loaded into the tiles and nominatim databases.

So, this is the scenario. But the problem is: whenever a tile is asked to be rendered, it can take a lot of time. I configured renderd to, in practice, never re-render old tiles, until I discover how to generate the tiles faster.

Is there a recommended configuration for better perfomance for this server (16vCPU and 64GB RAM) ? Is it not enough, maybe?

Is there any other tuning for faster rendering of the tiles? mode_tile ? renderd.conf ? apache conf? :rage:

I can’t find how many threads renderd should be set, for example. I realized that 32GB RAM wasn’t enough (renderd frequently was killed after demand too much memory).

An excerpt from /var/log/messages :

Sep 10 13:48:49 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 1 0-1 0-1 in 850.027 seconds
Sep 10 13:51:36 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 0 0-0 0-0 in 1000.030 seconds
Sep 10 13:55:39 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 2 0-3 0-3 in 1242.598 seconds
Sep 10 13:56:31 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 6 56-63 16-23 in 52.494 seconds
Sep 10 14:04:06 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 6 56-63 24-31 in 750.124 seconds
Sep 10 14:11:50 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 3 0-7 0-7 in 2176.555 seconds
Sep 10 14:17:32 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 5 24-31 8-15 in 1722.331 seconds
Sep 10 14:17:38 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 14 5504-5511 9488-9495 in 6.622 seconds
Sep 10 14:26:44 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 6 48-55 24-31 in 1812.681 seconds
Sep 10 14:33:52 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 5 16-23 16-23 in 1321.952 seconds
Sep 10 14:35:15 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 6 32-39 24-31 in 1868.531 seconds
Sep 10 14:47:19 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 4 0-7 8-15 in 1780.822 seconds
Sep 10 14:48:47 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 5 8-15 16-23 in 88.151 seconds
Sep 10 14:49:24 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 6 24-31 32-39 in 37.125 seconds
Sep 10 14:50:25 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 6 16-23 32-39 in 60.110 seconds
Sep 10 14:50:32 rhosmhmin01 renderd[80442]: renderd[80442]: DEBUG: DONE TILE default 7 48-55 72-79 in 7.527 seconds

Here is /usr/local/etc/renderd.conf :

[renderd]
num_threads=4
tile_dir=/var/lib/mod_tile
stats_file=/var/run/renderd/renderd.stats

[mapnik]
plugins_dir=/usr/lib/mapnik/input/
font_dir=/usr/share/fonts
font_dir_recurse=1

[default]
URI=/openstreetmap/osm_tiles/
TILEDIR=/var/lib/mod_tile
XML=/var/www/openstreetmap-carto-4.25.0/style.xml
HOST=localhost
TILESIZE=256
MINZOOM=0
MAXZOOM=18
TYPE=png image/png
DESCRIPTION=OSM
ATTRIBUTION=&copy;<a href=\"http://www.openstreetmap.org/\">OpenStreetMap</a>

And Apache conf / mod_tile module:


    LoadModule tile_module modules/mod_tile.so
    ModTileTileDir /var/lib/mod_tile
    LoadTileConfigFile /usr/local/etc/renderd.conf
    ModTileEnableStats On
    ModTileBulkMode Off
    ModTileRequestTimeout 0
    ModTileMissingRequestTimeout 30
    ModTileMaxLoadOld 50
    ModTileMaxLoadMissing 80
    ModTileVeryOldThreshold 31536000000000
    ModTileRenderdSocketName /var/run/renderd/renderd.sock
    ModTileCacheExtendedDuration 2592000
    ModTileEnableTileThrottling Off
    LogLevel notice

Thanks in advance,

2

I can’t answer the specifics in the question (you haven’t said anything about what map style you’re using so I’m guessing OSM Carto), but when tuning any other system, the first question to ask is what resources are being exhausted?

Is the system waiting for disk? Are all CPUs busy? Is it excessively swapping? Have you done any analysis of what the database is taking a long time to do?

Did the import process complete successfully? Are all indexes in place? Are there any recommended “extra” indices for a worldwide OSM Carto server, and if so are they in place?

I can’t comment on the time taken to rerender low zoom tiles on a worldwide server, but for mine covering UK+Ireland it’s just taken 9 seconds to rerender https://map.atownsend.org.uk/hot/2/1/1.png .

3

Thanks for you reply. I see… I really need to focus on some of the subjects you’ve pointed me.

I’m using openstreetmap-carto-4.25.0 .

I belived, and I’ll duckduckgo for it, that osm2pgsql created the indexes . I don’t know if I need to run some commands to guarantee that.

It seems that that the resources are not being exhausted anymore, after I doubled the amount of RAM (32 to 64GB) and applied the configuration from OSM Wiki to pgsql.

Well, I see it is to vague . You’re not GraphHopper, but thanks for the direction!