Dimensions du jeu de données

Author:Thomas Bonfort
Contact:tbonfort at terriscope.fr
Author:Seth Girvin
Contact:sgirvin at compass.ie
Author:Jerome Boue
Last Updated:2018/05/04
Version:1.6.1

Introduction

WMS layers in MapServer can support dimensions, see WMS Dimension for details. MapCache also supports dimensions for tilesets.

Storing separate tile caches for different dimensions has several practical use cases such as creating caches for different spatial boundaries, elevations, or time periods. It also provides a mechanism for dynamically switching Mapfiles based on a dimension value.

MapCache uses Dimensions to:

  • create the directory structure of disk and SQLite caches, using the template structure with {dim} replacement entry, see Template Structure in Caches disque and Using Multiple SQLite Database Files in SQLite Caches.
  • create the database structure of SQLite caches, using custom schema with :dim replacement entry, see Custom Schema in SQLite Caches.
  • build requests to WMS sources: any dimensions specified for a tileset will be forwarded on to their WMS source whenever a cache miss occurs.

Enabling Dimension Support in MapCache

Dimensions are enabled in the configuration file at the Jeu de tuiles level. Each tileset may define its own dimensions with a <dimensions> section which includes one or several <dimension> elements.

Dimension Attributes

Several attributes are defined for the <dimension> element.

  • name (required): The name by which the dimension is refered to in a WMS request (minus the dim_ prefix as specified in WMS 1.1.1 Specification).
  • default (required): The default value for the dimension if none is explicitly provided in the WMS request.
  • type (required): MapCache defines several ways for specifying a dimension: values, regex, sqlite and time. They are described in the following sections.
  • unit (optional): dimension unit. This information is used only in the GetCapabilities response.

Dimension Types

Several types of dimensions are supported by MapCache: Values, Regex, SQLite and Time. They can be classified into two groups depending on how dimension values are defined:

  • First level dimension types: Values and Regex. Dimensions values are statically specified in the configuration file.
  • Second level dimensions types: SQLite and Time. Dimensions values are stored in a dynamic backend (a SQLite file) so that an update in the dimension values does not involve a server restart.

First Level Dimensions

First level dimensions define values in the configuration file.

Values Dimensions

A Values type dimension lists all possible dimension values. The name of the dimension is used as the key in a query string when accessing through a WMS or WMTS service, for example &DIM1=foobarbaz.

If a client does not provide the dimension as a key value pair the default will be used (in this example foobar).

<tileset name="LayerName">
    <source>LayerSource</source>
    <cache>sqlite</cache>
    <grid>GoogleMapsCompatible</grid>
    <format>PNG</format>
    <metatile>5 5</metatile>
    <metabuffer>10</metabuffer>
    <dimensions>
        <dimension type="values" name="DIM1" default="foobar">
            <value>foobar</value>
            <value>foobarbaz</value>
            <value>foo</value>
            <value>bar</value>
        </dimension>
    </dimensions>
</tileset>

Regex Dimensions

An alternative to the hard-coded list of values is to use regular expressions.

The following example creates a MAPFILE dimension, for using the same MapCache tileset with different MapServer Mapfiles. The name of the Mapfiles need not be known to MapCache, and can therefore be created even after MapCache has been started.

When a user passes a MAPFILE=/path/to/mapfile, the string /path/to/mapfile is validated against the supplied (PCRE) regular expression. The one in this example allows a name composed of alphanumeric characters separated by slashes (/) and finishing with ”.map” ( [a-zA-Z0-9./]*.map$ ) , but will fail if there are two consecutive dots (..) in the path, to prevent file-system traversal.

<dimension type="regex" name="MAPFILE" default="/path/to/mapfile.map">
        <regex>^(?!.*\.\.)[a-zA-Z0-9\./]*\.map$</regex>
</dimension>

Second Level Dimensions

Second level dimensions store values in a dynamic back-end, initially a SQLite file. This allows for adding new dimension values without modifying MapCache configuration file, which otherwise would need to restart the server.

Time Dimensions

MapCache supports WMTS and WMS requests that include a TIME parameter, for both timestamps and time intervals. See MS RFC 96: Time Dimension Support in MapCache Tilesets initial proposal for further details.

Defining a Time dimension in MapCache involves a SQLite back-end. Two elements must be provided:

  • <dbfile>: Location of the SQLite file implementing the time dimension
  • <query>: SQL query returning the list of available timestamp values within a given time interval. In that query, start time and end time are represented by the respective placeholders :start_timestamp and :end_time_stamp. Returned values shall be in yyyy-mm-ddThh:mm:ssZ time format, as specified in RFC 3339, but restricted to UTC.

Time values in the SQLite back-end are expressed in Unix time, i.e. the number of seconds that have elapsed since January 1, 1970 (midnight UTC).

Querying the back-end with an interval may result in returning multiple timestamp values, each of which referencing distinct tiles. Assembling these tiles is described in Tile Assembly Policies section below.

<!-- "time" dimension

     This example defines a "time" dimension whose possible values are
     stored in the /path/to/mapcache-time.sqlite SQLite file. The default
     value is "2010".

     A WMS request with that dimension may contain, e.g.
     "... &time=1999-08-11T11:03:07Z/2001-09-21T08:17:56Z& ..."
-->
<dimension type="time" name="time" default="2010">
    <dbfile>/path/to/mapcache-time.sqlite</dbfile>
    <query>
         SELECT strftime("%Y-%m-%dT%H:%M:%SZ",ts) FROM time
          WHERE ts &gt;= datetime(:start_timestamp,"unixepoch")
            AND ts &lt;= datetime(:end_timestamp,"unixepoch")
       ORDER BY ts DESC
    </query>
</dimension>

SQLite Dimensions

In a SQLite dimension, dimension values are stored in a SQLite file. Moreover, each value references a list of internal, or sub-dimension values which are used to query cache.

For example, let’s define a “sensor” dimension whose allowed values represent various Earth observation spacecrafts. A WMS request with that dimension may contain, e.g.: ...&dim_sensor=spot&... In this example, each sensor references several data along an internal “product” sub-dimension. An example of the “sensor” dimension back-end contents is represented in the following table:

Sensor dimension Product sub-dimension
spot spot-img1
spot spot-img2
phr phr-img1
phr phr-img2
phr phr-img3

The WMS request occurs at the “sensor” dimension level. The cache query is performed at the “product” sub-dimension level. In case of a cache miss, the “product” sub-dimension values are used to query the data source.

Querying the back-end may then result in returning multiple sub-dimension values, each of which referencing distinct tiles. Assembling these tiles is described in Tile Assembly Policies section below.

<!-- "sqlite" dimension

     This example defines a "sensor" dimension whose possible values are
     stored in the /data/sensor.sqlite SQLite file. The default value is
     "phr".

     A WMS request with that dimension may contain, e.g.
     "... &dim_sensor=spot& ..."
-->
<dimension type="sqlite" name="sensor" default="phr">
   <dbfile>/data/sensor.sqlite</dbfile>
   <list_query>
      SELECT distinct(product) FROM dim
   </list_query>
   <validate_query>
      SELECT product FROM dim
       WHERE sensor=:dim
   </validate_query>
</dimension>

Tile Assembly Policies

When several time or sub-dimension values are returned from a dimension back-end query, the resulting tile is an assembly of all simple tiles matching each value. The way simple tiles are assembled is set with the <assembly_type> tag of the <dimensions> element. Currently only two values are implemented in MapCache:

  • none: No assembly is performed, the resulting tile is the first retrieved simple tile. This is the default value.

  • stack: Every pixel of the resulting tile is filled with the first opaque pixel found in all simple tiles in their retrieval order, e.g.:

    Assembly type: stack
    ../_images/mapcache-assembly-stack-1.png ../_images/mapcache-assembly-stack-2.png ../_images/mapcache-assembly-stack-3.png
    Product #1 Product #2 stack assembly of Products #1 & #2

The resulting tile may or may not be cached depending on the value of the <store_assemblies> tag of the <dimensions> element. The two possible values are true and false. Default is true.

The <subdimensions_read_only> tag of the <dimensions> element indicates whether data source shall be queried in case of cache miss. The two possible values are true and false. Default is false.

<dimensions>
    <assembly_type>stack</assembly_type>
    <store_assemblies>true</store_assemblies>
    <subdimensions_read_only>false</subdimensions_read_only>
    <dimension >...</dimension>
</dimensions>

Storing Dimensions

When using a disk based cache tiles will be stored in a folder structure similar to base/gridname/DIM1/value/xx/xx/xx/xx/xx/xx.png (where DIM1 is the dimension value).

The order of the <dimension> tags inside the <dimensions> tag is important as it is used to create the directory structure for the disk cache. If you change the order of these values, any tiles that have been previously cached are invalidated (they will be unavailable to MapCache but not deleted).

Templates can be used to change the folder structure for example:

<cache name="tmpl" type="disk">
    <template>/tmp/template-test/{tileset}#{grid}#{dim}/{z}/{x}/{y}.{ext}</template>
</cache>

Templates can also be used to change Sqlite database names e.g.

<!-- the following will create databases named TilesetName-#DimensionValue.db -->
<cache name="sqlite" type="sqlite3">
   <dbfile>{tileset}-{dim}.db</dbfile>
   <detect_blank />
</cache>

Accessing Tile Caches with Dimensions

To retrieve a tile with a specific dimension, add the dimension name and value to the request query string e.g. &DIM1=value

Only WMS and WMTS clients support passing dimensions to MapCache. Other tile services such as gmaps (GoogleMaps), tms, and KML do not support dimensions. WMTS however does allow access using a x,y,z addressing scheme, so by using URL rewriting on the web server tile-based clients (for example an OpenLayers ol.source.XYZ layer) can load a dimension-based tileset.

# a URL such as the following
http://servername/tiles/13/3914/2687.png
# can be rewritten into a WMTS request - note WMTS uses {z}/{y}/{x} rather than {z}/{x}/{y}
# so these values need to be swapped
http://servername/mapcache/wmts/1.0.0/layername/default/dimension/GoogleMapsCompatible/13/2687/3914.png

Seeding Tile Caches with Dimensions

The MapCache seeding tool allows specific dimensions to be seeded with the following syntax:

mapcache_seed -c mapcache_config.xml -t tileset1 -e -1187000,6695000,-605000,7450000 -z 7,10 --dimension DIMENSION_NAME=DIMENSION_VALUE