Some TileCache deficiencies (and workarounds).

Posted by – Sunday 2010-11-14

TileCache, as every piece of software complex enough, is not free of deficiencies. In this post I write about some TileCache features (or lack of) which, in my opinion, are deficiencies. These are: (a) a TileCache layer will only work with one set of map resolutions, (b) poor logging, (c) the TileCache seeder always stops on error, (d) the TileCache seeder cannot be restarted from a given point and (e) the lack of support for distributed computing. In some cases a workaround to partially overcome limitations is given.

TileCache version 2.11 is covered in this post.

1. Only one set of map resolutions per TileCache layer.

By nature WMS caches work with finite sets of map resolutions. In my opinion, one deficiency of TileCache is that a layer will only work with one set of map resolutions. For each layer the set of map resolutions is set in the configuration file by using the parameter ‘resolutions’ or by giving both ‘maxResolution’ and ‘zoomLevels’ parameters.

The problem comes when you have to work with more than one set of map resolutions. Say you have a WMS layer named ‘municipalities-boundaries’ that it is cached and must be shown at two different sets of map resolutions, say one with maximum map resolution 100 and the another one with map maximum resolution 200. In this case there would be two possible workarounds. One of them is creating two TileCache layers, say ‘municipalities-boundaries-200’ and ‘municipalities-boundaries-100’, as show in the following TileCache configuration file snippet:

[municipalities-boundaries-100]
debug=yes
type=WMSLayer
url=http://...
extension=png
size=256,256
bbox=580000,4688000,680000,4850000
layers=municipios
srs=EPSG:23029
extent_type=loose
maxResolution=100
levels=10

[municipalities-boundaries-200]
debug=yes
type=WMSLayer
url=http://...
extension=png
size=256,256
bbox=580000,4688000,680000,4850000
layers=municipios
srs=EPSG:23029
extent_type=loose
maxResolution=200
levels=10

The another workaround would be running two instaces of TileCache [note 1], one for each set of map resolutions.

2. No layer name in TileCache log files.

In the TileCache log files, the name of the layer is not shown. Cache hit, cache miss, time measurements… but what layer? The problem gets serious when two or more instances of TileCache are running. Since TileCache seeder output messages also lack layer name, this problem gets daunting, particularly if running in parallel several instances of it.

3. Tilecache seeder: unpredictable stops on error.

To ensure that responses to requests received by the WMS cache are fast from the first moment, we can seed it by simulating all possible client requests. Typically, seeding is done before the system is released to public. TileCache has an utility called tilecache_seed.py that will feed the WMS cache.

Seeding a WMS cache is a lengthy proccess, specially at low map resolutions or, equivalently, high zoom levels [note 2]. Say you have an area of 10,000 square kilometers, a map resolution of 0.55 meters per pixel and tiles whose size is 256×256 pixels. If a WMS layer covering this area is cached and seeded, near 504,425 tiles will have to be generated for this map resolution. In case it takes 1.2 seconds (average time) for the WMS server to render a tile, the total time is 605,310 seconds, or about 7 days. In 7 days many things can happen, like temporary network failures or power outages.

During a temporary network failure a spatial database might not be available. Thus, a WMS server retrieving data from this spatial database to render tiles will get nothing but an error message, and this failure will raise a TileCache seeder error. Since the TileCache seeder aborts on error, every temporary network failure – even one lasting fractions of second – has the potential of unpredictably stoping the seeding process. This behaviour forces monitoring the TileCache seeder and restart it when needed. On events like this, a more desirable behaviour would be doing nothig – that is, on error skipping tile rendering – instead of aborting the process. A small number of tiles would remain uncached for every temporary network failure like this one. Unless there are a high number of network failures during the seeding proccess, this problem will not be severe because the remaining tiles would be rendered and cached the first time a human client requests them.

I am not claiming that skipping tile rendering on error is always the best solution. It seems that there different opinions about this issue, which depends on the circumstances of each particular case [note 3]. My critique is that the TileCache seeder does not allow choosing what to do on error.

4. TileCache seeder: lack of restarting options.

Another flaw of the TileCache seeder is it not possible to restart it from the tile it was asking for when the process aborted, or, actually, from any given point but the init.

Let’s suposse that the WMS cache layer being feeded is the one given in the example given in previuos section (more than half million tiles) and that the TileCache seeder process aborts – no matter why – in a moment such 200,000 tiles have already been rendered and stored. The area that is being cached is logically divided in 256×256 pixel tiles (that is, there is a logical grid), all of them identified by an unique set of x, y and z coordinates (horizontal, vertical and zoom level, respectively). For any zoom level the x = 0, y = 0 [note 4] tile will be the one whose top left corner point is equal to the (xMin, yMin) point of the bounding box of the area.

For a given zoom level, the cache seeder scans this grid sequentally, looking in the cache for the tiles (x, y, z), (x, y + 1, z), (x, y + 2, z)… (x + 1, y, z), (x + 1, y + 1, z), (x + 1, y + 2, z)… and sending a GetMap request to the WMS server on cache miss [note 5]. As said before, it is not possible to restart the TileCache seeder from a given (x, y, z) tile; only the initial zoom level can be set. In the given scenario, before the TileCache seeder actually works again 200,000 cache hits will happen and despite cache hits are some orders of magnitude faster than cache misses, the wasted time can sum hours.

A workaround for this problem consists on (a) dividing the area to be cached in subareas that do not overlap and (b) run two or more instances of the TileCache seeder so each subarea is assigned to one and only one of these instances. To make these assignments, we will use the tilecache_seed.py parameter ‘–bbox'[1], which restricts the TileCache seeder to the specified bounding box. We will not avoid the problem of wasting time because unneccesary cache hits on TileCache seeder restart, but make it smaller. Running several instances of the TileCache seeder should not imply a noticeably overall performance penalty.

5. TileCache seeder: no support for distributed computing.

Distributed computing can be used to get the WMS cache seeded faster. If, for example, we have clones of a WMS server, it would be great that on cache miss the TileCache seeder could distribute the GetMap requests to the WMS servers. While is possible cloning a WMS server we have previously set up, we cannot make the TileCache seeder take advantage of it because for each layer, TileCache only accepts one URL (parameter ‘url’) [2] pointing to a WMS server.

A workaround for this deficiency could be making the URL point to a proxy script which would distribute the WMS GetMap requests to the WMS servers and its clones, as shown in the following TileCache configuration file snippet:

[municipalities-boundaries]
debug=yes
type=WMSLayer
url=http://url/to/proxy/script
extension=png
size=256,256
bbox=580000,4688000,680000,4850000
layers=municipios
srs=EPSG:23029
extent_type=loose
maxResolution=200
levels=10


notes

[note 1] There is a post in this blog titled “Running two (or more) instances of TileCache” explaining how to do it.

[note 2] Obviuosly the most important variable is the speed of the WMS server, something that it is not possible to control from TileCache.

[note 3] In the TileCache mailing list at osgeo.org there is a dicussion about this issue that can be found at http://goo.gl/iLAkP.

[note 4] This statement is true because this tile service is a Tile Map Service one. Other technologies like Google Maps or Microsoft’s Bing Maps Tile System follow different tile indexing conventions, so the statement mentioned before would be wrong in these particular cases. TileCache is aware of this and, in fact, in the configuration file there is a parameter named tms_type that can be set to ‘google’.

[note 5] The TileCache seeder has another seeding pattern called “center point/radius mode” in which the are seeded is circular and given by a center point and radius. In this mode start and zoom end level parameters are not accepted, so seeding is done in all the existing zoom levels.


references

[1] http://tilecache.org/docs/README.html#options

[2] http://tilecache.org/docs/README.html#configuration

0 Comments on Some TileCache deficiencies (and workarounds).

Respond

Respond

Comments

Comments