MapServer banner Home | Products | Issue Tracker | FAQ | Download
en it es zh_cn de el fr id sq tr

OGR

Author:Jeff McKenna
Contact:jmckenna at gatewaygeomatics.com
Last Updated:2010-10-16

Introduction

Starting with version 3.5, MapServer included the ability to access vector data sets in formats other than Shapefile in their native format using the OGR library. The following document describes the process for implementing OGR support within MapServer applications.

Note

Experimental OGR support was included in MapServer version 3.4 but this initial implementation had some limitations and is not covered in this document.

This document assumes that you are already familiar with certain aspects of MapServer:

  • MapServer application development and especially setting up .map files.
  • Some compilation skills if you don’t have ready access to a pre-compiled installation and need to compile your own copy of MapServer with OGR support.
  • access to OGR utilities, such as ogrinfo, which are available in the FWTools and MS4W packages.

Readers should also check out the Vector Data Access Guide, which has lots of examples of how to access specific vector formats.

What is OGR?

The OGR Simple Features Library is a C++ open source library (and command-line tools) providing read (and sometimes write) access to a variety of vector file formats including ESRI Shapefiles, and MapInfo mid/mif and TAB formats.

OGR is actually part of the GDAL library, so you will notice that some references point to GDAL (such as the mailing list).

What Does OGR Add to MapServer?

The OGR Simple Features Library allows MapServer users to display several types of vector data files in their native formats. For example, MapInfo Mid/Mif and TAB data do not need to be converted to ESRI shapefiles when using OGR support with MapServer.

What Data Formats are Supported?

See http://www.gdal.org/ogr/ogr_formats.html for the latest list of supported formats. At the date this document was written, the following formats were supported:

Note

Some of the above formats (e.g. OGDI) have external dependencies and are not always included in the pre-compiled binary distributions of MapServer with OGR support.*

Note

Some of the above formats are not well suited for random access by nature, that’s the case of MapInfo MIF/MID files which is a TEXT format and will give very poor performance for a web application. On the other hand, some binary formats such as MapInfo TAB are better suited for random access and will give performance comparable to native shapefile access in MapServer.*

How to Get More Information on the OGR Project

  • More information on the OGR Simple Features Project can be found at http://www.gdal.org/ogr/.
  • The GDAL mailing list can be used for OGR related questions. Always search the list archives before sending new questions.
  • The GDAL Wiki has lots of good information for users and developers.
  • The #gdal IRC channel on irc.freenode.net might also be of help. For info on IRC see the MapServer IRC page.

The main developer of the OGR library is Frank Warmerdam and the integration of OGR within MapServer was done by Daniel Morissette.

Obtaining and Compiling MapServer with OGR Support

  • Follow the instructions on the OGR page to compile/install OGR/GDAL.
  • Obtain the MapServer source.

For UNIX users, see the README.CONFIGURE file in the MapServer source, or see the UNIX Compilation and Installation. If GDAL/OGR is normally installed it should be sufficient to add –with-ogr to the configure line before (re)building MapServer. Linux users might want to try FGS, a Linux installer for MapServer.

For Windows users, it is recommended to look for a pre-compiled binary on the MapServer site (MS4W is recommended). If you want to compile your own then see the README.WIN32 file in the MapServer source.

Integrating OGR Support with MapServer Applications

The only change that is needed to integrate OGR support with a MapServer application is with the .map file. The LAYER’s DATA parameter is expanded to three parameters (CONNECTIONTYPE OGR, CONNECTION and DATA).

The syntax for this differs depending on the type of data being used (the Vector Data Access Guide is an excellent resource for this). In OGR, a data source can be either a set of files that share a common basename (e.g. .shp/.shx/.dbf for ArcView Shapefiles, or .tab/.map/.dat/.ind/.id for MapInfo TAB files) or a whole directory of files (e.g. TIGER).

Let’s call the former “File-based data sources” and the later “Directory-based data sources”. When accessing a file-based data source you specify the filename of one of the files in the set (e.g. roads.shp or roads.tab) and when accessing a directory-based data source you specify the directory name and OGR reads all the files in the directory as a single data source with potentially several layers (e.g. TIGER files).

Some OGR drivers (e.g. SHP, TAB) can have dual behaviors, that is if they’re pointed to a single file then they behave as a file-based data source and if they’re pointed to a directory then they will behave as a directory-based data source and then every file in the directory becomes a new layer in the data source.

See the OGR formats page for more info on the specific file format you’re using. (Click on the format name for more specific driver info on that format)

Using OGR Data Sources in the Map File

The .map file LAYER definition for file-based sources is as follows:

LAYER
  ...
  CONNECTIONTYPE OGR
  CONNECTION "<datasource_name>"
  DATA "<layer_definition>"
  ...
END

<datasource_name> is the name of the datasource to read from and is prefixed by the CONNECTION keyword. The exact organization depends on the format driver in use. The format driver to use is automatically selected by OGR based on the nature of the string passed as the datasource, and/or the format of the file referenced by it.

  • For file based datasources this is the name of the file, including the extension, using an absolute path, or a relative path. Relative paths are interpreted relative to the SHAPEPATH first, if not found then we try again relative to the .map file location.

    Note

    Before version 4.1 the SHAPEPATH was ignored for OGR datasources.

  • For directory based datasources, such as TIGER/Line, or Arc/Info Binary Coverages this is the name of the directory containing the files. If the path is relative it is interpreted relative to the .map file.

  • For virtual datasources such as database systems, and OGDI this is the service connection string and is generally not related to the filesystem. For instance, for Oracle Spatial this might be “OCI:warmerda/Password@gdal800.velocet.ca”.

<layer_definition> is the name, number or SQL definition of the layer to use from the datasource. It is indicated via the DATA keyword in the map file.

  • Layer Name: The (case insenstive) layer name may be used to select a layer.
  • Layer Number: The layer number (starting from 0 for the first layer) may be used to select a layer. Generally the layer name is preferred to this since it is more self describing.
  • Omitted: If no DATA keyword is provided, this is equivalent to selecting layer 0.
  • SQL SELECT: If an SQL SELECT statement is used, it is interpreted in a driver specific manner to try and generate a temporary pseudo-layer. For some formats this a restricted subset of SQL is interpreted within OGR. For RDBMS based drivers (such as PostGIS and Oracle) this is passed through to the underlying database.

The OGRINFO utility can be used to find out the list of layers and their names in a data source.

Examples of Layer Definitions Using OGR

Please see the Vector Data Access Guide for details and examples of each data format supported.

Example 1. MapInfo TAB file

LAYER
  NAME "Builtup_Areas_tab"
  TYPE POLYGON
  CONNECTIONTYPE OGR
  CONNECTION "data/tab/092b06_builtup_a.tab"
  STATUS ON
  CLASS
    ...
  END
...
END

Example 2. Microstation DGN file using <layer_index>

The entire DGN file is represented in OGR as one layer (see the DGN driver page for more details):

LAYER
  NAME "dgn"
  TYPE LINE
  CONNECTIONTYPE OGR
  CONNECTION "dgn/santabarbara02.dgn"
  DATA "0"
  STATUS ON
  STYLEITEM "AUTO"
  CLASS
    ...
  END
END # Layer

Example 3. TIGER/Line file using <layer_name>

LAYER
  NAME "Roads_tig"
  TYPE line
  CONNECTIONTYPE OGR
  CONNECTION "full/path/to/tiger/TGR25001"
  DATA "CompleteChain"
  STATUS ON
  CLASS
    ...
  END
END

Example 4. Directory of Shapefiles using SQL JOIN

LAYER
  NAME "Parks_cov"
  TYPE POLYGON
  CONNECTIONTYPE OGR
  CONNECTION "data/shppoly"
  DATA "SELECT eas_id, idl.Name FROM pol LEFT JOIN idl ON pol.eas_id = idl.eas_id"
  STATUS ON
  CLASSITEM "idlink.Name"
  CLASS
    ...
  END
END

How to Use “OGRINFO”

OGRINFO is part of the GDAL/OGR distribution (it is also included in FWTools and MS4W). It is an executable that can be used to obtain layer information about OGR supported files. The parameters are:

ogrinfo [-ro] [-q] datasource_name [layer [layer...]]

  • -ro opens the file as read only (optional)
  • -q executes in quiet mode, only the layer idex line will be returned (optional)
  • datasource_name is the filename including extension (eg. roads.tab); for TIGER/Line files, datasource_name is the directory containing the TIGER files (eg. ogrinfo TGR25001)

Example 5. To get the list of layers in a file:

$ ogrinfo  popplace.tab

Had to open data source read-only.
INFO: Open of `popplace.tab'
using driver `MapInfo File' successful.
1: popplace (Point)

which shows that there is one point layer in the popplace.tab file.

Example 6. To get a dump of a specific layer, including field names, projection, etc:

$ ogrinfo popplace.tab popplace

Had to open data source read-only.
INFO: Open of `popplace.tab'
using driver `MapInfo File' successful.

Layer name: popplace
Geometry: Point
Feature Count: 497
Layer SRS WKT: PROJCS["unnamed",GEOGCS["unnamed",DATUM["North ...snipped...
AREA: Real (15.3)
PERIMETER: Real (15.3)
POPPLACE_: Real (11.0)
POPPLACE_I: Real (15.0)
NAME: String (50.0)
OGRFeature(popplace):1
  AREA (Real) =           0.000
  PERIMETER (Real) =           0.000
  POPPLACE_ (Real) =           1
  POPPLACE_I (Real) =               1
  NAME (String) = Port Hope Simpson
  POINT (2437287.249 1153656.751)

OGRFeature(popplace):2
  AREA (Real) =           0.000
  PERIMETER (Real) =           0.000
  POPPLACE_ (Real) =           2
  POPPLACE_I (Real) =               1
  NAME (String) = Hopedale

...
...

Example 7. To get a list of layers in a TIGER/Line Directory:

$ ogrinfo TGR25001

Had to open data source read-only.
INFO: Open of `TGR25001'
using driver `TIGER' successful.
1: CompleteChain (Line String)
2: AltName (None)
3: FeatureIds (None)
4: ZipCodes (None)
5: Landmarks (Point)
6: AreaLandmarks (None)
7: KeyFeatures (None)
8: Polygon (None)
9: EntityNames (Point)
10: IDHistory (None)
11: PolyChainLink (None)
12: PIP (Point)
13: TLIDRange (None)
14: ZipPlus4 (None)

The above example shows that there are 14 layers in the TGR25001 directory.

Example 8. To get a summary of a specific TIGER layer, including only field names, projection, and extent

$ ogrinfo TGR25001 Landmarks -summary

Had to open data source read-only.
INFO: Open of `TGR25001'
using driver `TIGER' successful.

Layer name: Landmarks
Geometry: Point
Feature Count: 777
Extent: (-70.674324, 41.519817) - (-69.969211, 42.046868)
Layer SRS WKT: GEOGCS["NAD83",DATUM["North_American_Datum_1983",
SPHEROID["GRS 1980",6378137,298.257222101]],PRIMEM["Greenwich",0],
      UNIT["degree",0.0174532925199433]]
MODULE: String (8.0)
FILE: String (5.0)
STATE: Integer (2.0)
COUNTY: Integer (3.0)
LAND: Integer (10.0)
SOURCE: String (1.0)
CFCC: String (3.0)
LANAME: String (30.0)

Queries Through OGR Format

OGR layers can be queried the same way as regular shapefiles in MapServer.

TILEINDEX with OGR

OGR layers can utilize tile indexes in a similar fashion to Shapefile based layers. The TILEINDEX keyword should contain the connection string for the tile index file. The tile index file may be any supported OGR format, including shapefiles.

The TILEITEM keyword in the LAYER definition indicates what attribute from the tile index file should be used as the datasource location. If omitted, the default TILEITEM value is “location”. The value in the location field should be a connection string the same as would have been used in the CONNECTION field for OGR layers. The CONNECTION keyword is not needed (and will be ignored) for layers using the OGR connection type and having the TILEINDEX keyword.

Tileindex files can be prepared in an external GIS, or using the OGR utility ogrtindex. Details can be found on the OGR Utilities Page.

The following is a simple example of a point layer using a tile index.

LAYER
  NAME "ogr_points"
  TYPE POINT
  CONNECTIONTYPE OGR
  TILEINDEX "PIP_ogr_tiles.shp,0"
  STATUS ON
  CLASS
    NAME "points"
    STYLE
      SYMBOL "default-circle"
      COLOR 255 0 0
      SIZE 6
    END
  END
END

OGR tileindex layers should support all normal query and attribute fetching mechanisms, including from MapScript; however, this has not been heavily tested as of April/2002. Please report problems via the MapServer Trac. If auto projection support is used for tileindexed OGR layers, the tileindex is read for the projection (not the component tiles). Problems may (or may not) be encountered if the component tiles have differing schemas (different sets of attributes).

Connection Pooling

For some OGR supported formats, connecting to the dataset is quite expensive in terms of CPU use and amount of disk IO. For instance, establishing access to an S-57 dataset results in a complete read into memory of the data files. Connection pooling control aims at reducing this overhead in situations where the same file is used for several different map layers.

To ensure that an OGR supported dataset is only opened once per map render (instead of separately for each map LAYER referencing the dataset, use the CLOSE_CONNECTION PROCESSING option. The default value is for CLOSE_CONNECTION is NORMAL, but if set to DEFER the dataset will be kept open till the map render is complete. It will be reused by any other layers with using the same datasource.

Example 9. Preserve S-57 connection for two layers

In this example, we are using the same dataset (NO410810.000) for two layers. To avoid re-reading the dataset, we mark the first layer to defer closing the connection till layer. In the second (or last) layer we request NORMAL connection handling (though this could have been left out as normal handling is the default).

LAYER
  NAME "AdminAreas"
  TYPE POLYGON
  CONNECTIONTYPE OGR
  CONNECTION "NO410810.000"
  DATA "ADMARE"
  PROCESSING "CLOSE_CONNECTION=DEFER"
  STATUS ON
  ...
END
LAYER
  NAME "Land Areas"
  TYPE POLYGON
  CONNECTIONTYPE OGR
  CONNECTION "NO410810.000"
  DATA "LNDARE"
  PROCESSING "CLOSE_CONNECTION=NORMAL"
  STATUS ON
  ...
END
  1. The text of the CONNECTION keyword must match exactly between layers for the connection to be reused.
  2. Some dataset connections are quite memory expensive, and keeping them open may result in increased memory use.
  3. If all layers rendered for a particular connection defer closing the connection, it will remain open till MapServer terminates. For normal cgi or MapScript use this is likely OK.
  4. This use of CLOSE_CONNECTION handling is unique to OGR layers, and may be changed at some point in the future as part of a broader implementation of connection pooling in MapServer.

STYLEITEM “AUTO” - Rendering Layers Using Style Information from the OGR File

Note

This feature is only supported with MapInfo TAB and Microstation DGN files at the moment, but eventually other formats that carry colors and styles at the shape-level may also be supported through OGR.*

In MapServer, ArcView, and other shapefile-based applications, colors and styles are usually defined at the layer level. This means that all the shapes in a given layer are usually rendered using the same color and styles.

On the other hand, some formats supported by OGR such as MapInfo TAB do have color and style information attached to each shape. OGR adds support for the ‘STYLEITEM “AUTO”’ layer parameter which allows you to request that the shapes in a layer be rendered using colors and styles coming from the data source instead of being driven by CLASSes as was traditionally done with MapServer.

How to Implement

In order to have a layer rendered using colours and styles coming from the OGR data source, your must do the following:

  • Your layer definition must contain the STYLEITEM “AUTO” parameter.
  • Your layer definition needs to contain at least one CLASS (which may be empty) and optionally a CLASSITEM to match the expressions if your CLASS contains an expression. The empty CLASS in the layer will be updated dynamically at runtime to contain colours and styles coming from the data source for each shape.

Examples

Example 10. Layer Definition Using STYLEITEM “AUTO” without a CLASSITEM

 LAYER
   NAME "test_dgn"
   STATUS ON
   TYPE POLYGON
   CONNECTIONTYPE OGR
   CONNECTION "../data/dgn/test.dgn"

   # This enables use of colors and styles from the source file.
   STYLEITEM "AUTO"

   # Define an empty class that will be filled at runtime from the
   # color and styles read on each shape in the source file.
   CLASS
   END
END  # layer

Example 11. Layer Definition Using STYLEITEM “AUTO” with a CLASSITEM

LAYER
 NAME "Builtup_Areas_tab"
 TYPE POLYGON
 CONNECTIONTYPE OGR
 CONNECTION "data/tab/092b06_builtup_a.tab"
 STATUS ON

 # This enables use of colors and styles from the source file.
 STYLEITEM "AUTO"

 # Define an empty class that will be filled at runtime from the
 # color and styles read on each shape in the source file.
 CLASSITEM "CATEGORY"
 CLASS
    EXPRESSION "1"
 END
END

Please Note:

CLASS EXPRESSIONs are still working, so it is still possible to query and classify layers that are using STYLEITEM “AUTO”. The only difference is that instead of using static class definitions, the colors and style will be read from the data file.

Important Notes

NOTE 1:Even though MapInfo and other OGR data sources may support layers with mixed geometry types (e.g. points, lines and polygons in the same file) this is not yet supported in MapServer. So you still have to define a layer ‘TYPE’ and make sure that all the shapes in the OGR data source are compatible with that layer type, otherwise MapServer may produce an error about incompatible geometry types at runtime.
NOTE 2:Due to the dynamic nature of this feature, it is not compatible with the labelcache, so the labelcache is automatically disabled for layers that make use of ‘STYLEITEM “AUTO”’.
NOTE 3:When you use STYLEITEM AUTO, MapServer tries to match symbol names returned by OGR to names in your symbol file. For a quick solution, try using the following symbol file:

http://demo.mapserver.org/ogr-demos/yk_demo/etc/symbols_mapinfo.txt

The name of the symbols returned by OGR to MapServer depends on the file format. In the case of MapInfo files, it will be:

  • For “old-style” symbols (default MapInfo 3.0 symbols numbered 32 to 67) the symbol name will be ‘mapinfo-sym-##’ where ‘##’ is the symbol number, e.g. ‘mapinfo-sym-32’.
  • For “Font Symbols”, the symbol name is also ‘mapinfo-sym-##’ where ‘##’ is the symbol number in the font. In this case, the name of the font itself is ignored by MapServer.
  • MapInfo also supports “custom symbols” (bitmap symbols)... I’m not sure what you would get from OGR for this, but I’m pretty sure that MapServer doesn’t do anything useful with them.

The OGRINFO utility can be used to find out exactly which symbol names OGR will return to MapServer. Look at the “Style” string in the ogrinfo output for each shape that is read.

Mapping of OGR Style Info to the MapServer CLASS Members

Here is the list of style parameters that are currently supported from OGR data sources and how they’re mapped in MapServer:

Line color
The line colour is mapped to CLASS.COLOR
Line thickness
The line thickness is mapped to CLASS.STYLE.WIDTH. The default will be 1 pixel line (as it always is with MapServer).
Polygon fill color

Polygon fill color is mapped directly to CLASS.COLOR

Note that at this time, transparent polygons are not supported (they’re always opaque).

Polygon outline
If a polygon has an outline color and thickness defined in the data source then the same rule as for line color and thickness above will apply, except that the outline color is mapped to CLASS.OUTLINECOLOR
Point symbols

Point symbol color is directly mapped to CLASS.COLOR. Point symbol size is directly mapped to CLASS.SIZE.

If your symbolset contains a symbol called “default-marker” then this symbol will be used, otherwise the default will be CLASS.SYMBOL=0 (i.e. a 1 pixel dot)

It is also possible (with a bit of work) to control which symbol gets used in rendering point symbols. OGR provides MapServer with symbol names, and if the symbol name returned by OGR to MapServer matches the name of one of the symbols in your symbolset then this symbol will be used.

For MapInfo point symbols (numbered 32 to 67 in the MapInfo MIF spec), the name returned by OGR is “mapinfo-sym-X” where X should be replaced with the MapInfo symbol number (e.g. “mapinfo-sym-35” is the star symbol).

If the OGR symbol id is a web reference (http://.../mysymbol.png), the symbol will be downloaded and a new symbol entry will be created referring to it.

Text labels

The text string is mapped to CLASS.TEXT

Text color is mapped to CLASS.LABEL.COLOR

Text background color is mapped to CLASS.LABEL.BACKGROUNDCOLOR

Text height is mapped to CLASS.LABEL.SIZE

Text angle is mapped to CLASS.LABEL.ANGLE

Text font mapping follows the following rules:

  1. If TTF fonts are supported:
    1. If the native font name (e.g. “Arial”) is found in your fontset then this font will be used. The font styles bold and italic are supported as follows: Arial bold fontname maps to arial-bold. Arial italic fontname maps to arial-italic. Arial bold italic fontname maps to arial-bold-italic. If the styles are not available, arial will be used.
    2. If 1a. failed and a font called “default” is present in your fontset then this “default” font will be used.
  2. If TTF fonts are not supported or if all above cases failed, then BITMAP MEDIUM font will be used.
Transparency
If the color parameter from the OGR style contains an alpha value, the value will be used to set the OPACITY parameter in the STYLE object.

Accessing OGR STYLEITEMAUTO Label Styles Through MapScript

OGR STYLEITEMAUTO label styles can be accessed through MapScript, such as PHP/MapScript’s getshape() or getvalue() methods, by setting the LAYER’s PROCESSING parameter to “GETSHAPE_STYLE_ITEMS=all”. Therefore, the LAYER may contain:

LAYER
  ...
  PROCESSING "GETSHAPE_STYLE_ITEMS=all"
  ...
END

The following label styles are supported:

Label Style Description MapServer Version Implemented
OGR:LabelFont Comma-delimited list of fonts names 5.4
OGR:LabelSize Numeric value with units 5.2.0
OGR:LabelText Label text string 5.2.0
OGR:LabelAngle Rotation angle (in degrees) 5.2.0
OGR:LabelFColor Foreground color 5.4
OGR:LabelBColor Background color 5.4
OGR:LabelPlacement How is the text drawn relative to the feature’s geometry 5.4
OGR:LabelAnchor A value from 1 to 12 defining the label’s position relative to the point to which it is attached. 5.4
OGR:LabelDx X offset 5.4
OGR:LabelDy Y offset 5.4
OGR:LabelPerp Perpendicular offset 5.4
OGR:LabelBold Bold text 5.4
OGR:LabelItalic Italic text 5.4
OGR:LabelUnderline Underlined text 5.4
OGR:LabelPriority Numeric value defining the order in which style parts should be drawn. 5.4
OGR:LabelStrikeout Strike out text (gdal >= 1.4.0) 5.4
OGR:LabelStretch Stretch factor changes the width of all characters in the font by factor percent. (gdal >= 1.4.0) 5.4
OGR:LabelAdjHor Horizontally adjacent text (gdal >= 1.4.0) 5.4
OGR:LabelAdjVert Vertically adjacent text (gdal >= 1.4.0) 5.4
OGR:LabelHColor Shadow color (gdal >= 1.4.0) 5.4
OGR:LabelOColor Outline color (gdal > 1.6.0) 5.4

Please see the OGR Feature Style Specification document for more details on those specific styles.

Sample Sites Using OGR/MapServer

The following sites use OGR’s STYLEITEM “AUTO” feature:

The following site uses OGR, as well as MapInfo’s ‘Seamless Map Layers’ feature:

The following site uses OGR to display TIGER 2000 files:

FAQ / Common Problems

Q:What Does “OGR” Stand For?
A:Basically, OGR does not stand for anything. For a detailed explanation of how OGR was named, see GDAL’s FAQ at http://trac.osgeo.org/gdal/wiki/FAQ.

Q:

When using STYLEITEM AUTO, what should I have in my .sym symbols file?

A:

When you use STYLEITEM AUTO, MapServer tries to match symbol names returned by OGR to names in your symbol file. For a quick solution, try using the following symbol file:

http://demo.mapserver.org/ogr-demos/yk_demo/etc/symbols_mapinfo.txt

The name of the symbols returned by OGR to MapServer depends on the file format. In the case of MapInfo files, it will be:

  • For “old-style” symbols (default MapInfo 3.0 symbols numbered 32 to 67) the symbol name will be ‘mapinfo-sym-##’ where ‘##’ is the symbol number, e.g. ‘mapinfo-sym-32’.
  • For “Font Symbols”, the symbol name is also ‘mapinfo-sym-##’ where ‘##’ is the symbol number in the font. In this case, the name of the font itself is ignored by MapServer.
  • MapInfo also supports “custom symbols” (bitmap symbols)... I’m not sure what you would get from OGR for this, but I’m pretty sure that MapServer doesn’t do anything useful with them.

The OGRINFO utility can be used to find out exactly which symbol names OGR will return to MapServer. Look at the “Style” string in the ogrinfo output for each shape that is read.