MapServer banner Home | Products | Issue Tracker | FAQ | Download
en it es zh_cn de el fr id sq tr

Documentation Development Guide

Author:Howard Butler
Contact:hobu.inc at gmail.com
Author:Jeff McKenna
Contact:jmckenna at gatewaygeomatics.com
Last Updated:2014-04-16

Background

The current structure of the MapServer documentation process is for developers to maintain their documents in reStructuredText format, and submit their changes through GitHub pull requests; you will first need to create a free personal account at GitHub. The Sphinx documentation generator is used to convert the reStructuredText files to html, and the live website is then automatically updated once every 2 hours, on the hour.

GitHub Notes

  • Documentation files are stored in the /docs sub-project on GitHub at: https://github.com/mapserver/docs

  • Always submit pull requests on the latest branch (such as branch-6-4), not on master

  • See GitHub’s help guidelines to install Git locally

  • Useful git commands:

    • clone branch locally:

      git clone -b branch-6-4 git@github.com:mapserver/docs.git mapserver-docs-git-branch-6-4
    • update local files:

      git pull
    • add file to repository (need to commit after this):

      git add <file>
    • commit file to repository:

      git commit -m "my message" <files>

General Writing Guidelines

  • MapServer instead of mapserver, map server, Map Server, mapServer or map Server.

  • MapScript instead of mapscript, Mapscript, or map script.

  • PostGIS instead of postgis.

  • HowTo instead of howto or HOWTO.

  • Email addresses should be manually spam protected:

    hobu.inc at gmail.com instead of hobu.inc@gmail.com

reStructuredText Formatting

  • All text should be hard breaks at or around the 80 column mark, just as the source code.

  • No .. sectnum:: in the contents directives

  • All external links should live at the bottom of your document, under the heading:

    .. #### rST Link Section ####
  • Always include the :Revision: and :Date: lines (as-is) at the top of your document, such as:

    :Revision: $Revision$
    :Date: $Date$

Installing and Using Sphinx for rst-html Generation

Note

As of 2013-10-06 the MapServer site requires Sphinx 1.2b3. You can browse the versions of the Sphinx packages here, and then install the exact version such as:

easy_install Sphinx==1.2b3

On Windows:

  1. install Python 2.X

  2. download setuptools

  3. make sure that the ‘C:/Python2X/Scripts’ directory is your path

  4. execute the following at commandline:

    easy_install Sphinx

    ...you should see message: “Finished processing dependencies for Sphinx”

    Note

    Make sure you install Sphinx 1.2b3 or more recent. See note above.

  5. inside the /docs directory, execute:

    make html

    or

    make latex

    the HTML output will be written to the _build/html sub-directory.

On Linux:

  1. make sure you have the Python dev and setuptools packages installed. On Ubuntu:

    sudo apt-get install python-dev
    sudo apt-get install python-setuptools
  2. install sphinx using easy_install:

    sudo easy_install Sphinx

    Note

    Make sure you install Sphinx 1.2b3 or more recent. See note above.

  3. to process the docs, from the MapServer /docs directory, run:

    make html
    make html BUILD_LANGUAGES=

    or

    make latex

    the HTML output will be written to the build/html sub-directory.

Note

If there are more than one translation, the above commands will automatically build all translations, unless you add a “BUILD_LANGUAGES=” to the make arguments

On OS X:

  1. install sphinx using easy_install:

    sudo easy_install Sphinx

    Note

    Make sure you install Sphinx 1.2b3 or more recent. See note above.

# install MacTex from http://www.tug.org/mactex/2009/ if you want to build pdfs

  1. to process the docs, from the MapServer /docs directory, run:

    make compile_messages
    
    make html

    or

    make compile_messages
    
    make latex

    the HTML output will be written to the build/html sub-directory.

Note

If there are more than one translation, the above commands will automatically build all translations.

How translations are handled

Nowadays, MapServer have two coexisting systems to manage translations process. Both rely on Sphinx software.

The first one was the answer to the absence of internationalization support in first days of the Sphinx software. The second use the new system of internationalization and is currently only used for the French translation.

The “old” way for translations

Although we think that nowadays the new system is better, we do not wanted to enforce every language community to change their habits.

  • All translations are organized in subdirectories in MapServer /docs directory
  • The directories are named using ISO3166-1 alpha-2 country codes, which will also reference to the corresponding flag icon
  • Translations are based on the English docs
  • The directory structure and filenames must be kept, they are used to generate links between the different translations

Note

To start a new translation, copy the directories images and include from docs/en to docs/<lang>, where <lang> is one of the country codes. You also should copy the docs/en/documentation.txt and docs/en/index.txt files into your <lang> directory (the build process requires these files...you are free to edit them as you wish for your own language).

  • Only translated files are kept in the <lang> directories and the repository.
  • The build script (Makefile and make.bat) have an option (init) to preprocess the <lang> directories. That means that each not translated English file is copied to the target <lang> directory. You don’t have to do this to build html files locally. If you do this, you have to clean up you directories afterwards.
  • To keep the translations in sync with the English docs, the translators can monitor commits to the repository.

Note

One way to monitor changes is to subscribe the RSS feed through Github: in the docs repository, click on the “Commits” tab and look for a subscription link for these commit changes, for example: see “Commit History” at https://github.com/mapserver/docs/commits/branch-6-2

  • You have to define which languages are available by setting TRANSLATIONS in the Makefile or make.bat:

    TRANSLATIONS     = en de

The build script will then process the subdirectories en and de. If they are not accessible, an error message will be returned.

The standard internationalization system (i18n) for translations

Initial translation process for a new language

It follows steps below:

  • Generate pot file from original English documentation using Sphinx (except if already here)

    make gettext
  • copy new version of *.pot files from _build/gettext/en to translated/pot

    make gettext_copy
  • Copy *.pot file => *.po file (a *.po file is the same as *.pot, only extension change and if you translate three languages, you will get three *.po file and each of them will be into a language dir)

    make generate_po_from_tmpl -e TRANSLATIONI18N=fr
  • translate your *.po file(s)

    You can do it with software like PoEdit but we may switch to Transifex, an hosted translation service to make contribution more easy, follow translation advance and have a review process for translation.

  • compile *.po to *.mo (sphinx need *.mo file to make substitution from original version to the language translated)

    make compile_messages -e TRANSLATIONI18N=fr
  • Generate doc for the language

You just need to set in the file Makefile or make.bat depending on your OS the following parameter before building the documentation:

TRANSLATIONI18N=fr

You need to change as well the script/build_docs.sh script to add your langage at the top of the file:

LANGUAGES= fr

Please for both parameter, keep the alphabetical order to kept them readable.

Structure of the i18n files in the docs root directory

The structure you need to make all the transaltion process is like below (an excerpt to give you the main required structure)

translated/
├── fr
│   ├── about.po
│   ├── announcements_all.po
│   ├── cgi.po
│   ├── community.po
│   ├── copyright.po
│   ├── development.po
│   ├── documentation.po
│   └── LC_MESSAGES
│       ├── about.mo
│       ├── announcements_all.mo
│       ├── cgi.mo
│       ├── community.mo
│       ├── copyright.mo
│       ├── development.mo
│       └── documentation.mo
└── pot
    ├── about.pot
    ├── announcements_all.pot
    ├── cgi.pot
    ├── community.pot
    ├── copyright.pot
    ├── development.pot
    └── documentation.pot

Moreover, you only need to commit the *.pot and *.po files. Be cautious, you must have an LC_MESSAGES directory to receive *.mo files. To keep this directory in Git, it’s recommended to create and empty file into it.

Note

When adding a new langage, take care of the ISO code used. Famfamfam flag icons is using ISO Country code, and transifex is using ISO Langage code which can be different.

The usual workflow for already existing translation

The previous process we explained is only for the language creation. After, the workflow consists of:

  • translating po files for your language via transifex,
  • pull po files from transifex and commit them
  • generating the .mo files
  • building again

Here are the command line:

  • Getting po files from transifex (you can use -l flags to filter the langage):

    tx pull -a
  • Build mo files from po files:

    make compile_messages  -e TRANSLATIONI18N=fr
  • Build html page:

    make clean html

See below how to setup your transifex account to use transifex client to push, pull translated files and build html files.

Sometimes, when a change occurs in the main english documentation, you will need to generate again the pot files. Then, you will have to merge your translated files with the new pot files using for example the command line utility (from gettext) below:

make gettext
make gettext_copy

-U mean update po file with pot file and -N mean don’t use fuzzy but exact match. Then push po files to transifex (you can use -l flags to filter the langage):

tx push -s

Knonw issues:

  • when downloading po files to local dir, if the file exists it won’t be updated. At this moment we should remove all po files before downloading.

  • when building html doc cache system doesn’t allow to update a page with up to date translation. We should use the clean target with make.

  • This command line throw an error:

    tx push -s -t

How to use Transifex client to manage translation process

Note

Transifex is described more in depth in the document: How to Help Translate the Documentation

  • Install transifex client:

    sudo apt-get install transifex-client
  • Edit ~/.transifexrc and add the appropriate info in there:

    vim ~/.transifexrc
    • With the following content:

      [https://www.transifex.com/]
      username = yjacolin
      token =
      password = passw0rd!
      hostname = https://www.transifex.com/

Reference Labels

:ref: reference labels
Label Title
about About
agg AGG Rendering Specifics
antialias AntiAliasing with MapServer
arcinfo ArcInfo
arcsde ArcSDE
autotest Regression Testing
background Tutorial Timeframe
batch_utilities Batch Scripting
bugs Bug Submission
cgi CGI
cgi_controls MapServer CGI Controls
cgi_introduction MapServer CGI Introduction
class CLASS
community Community Activities
development Development
dgn DGN
documentation MapServer 5.2.2 Documentation
documentation_development Documentation Development Guide
dotnet_compile .NET MapScript Compilation
download Download
dynamic_charting Dynamic Charting
editing Mapfile Editing
errors Errors
example1-1 Example 1.1
example1-1-map Example1-1.map
example1-2 Example 1.2
example1-2-map Example1-2.map
example1-3 Example 1.3
example1-3-map Example1-3.map
example1-4 Example 1.4
example1-4-map Example1-4.map
example1-5 Example 1.5
example1-5-map Example1-5.map
example1-6 Example 1.6
example1-6-map Example1-6.map
example1-7 Example 1.7
example1-7-map Example1-7.map
example1-8 Example 1.8
example1-8-map Example1-8.map
expressions Expressions
faq FAQ
fastcgi FastCGI
feature FEATURE
filter_encoding WFS Filter Encoding
flash Flash Output
fontset FONTSET
format_types Data Format Types
genindex Index
git GitHub
gloss Glossary
gml GML
gpx GPS Exchange Format (GPX)
grid GRID
html_legend HTML Legends with MapServer
iis IIS Setup for MapServer
imagemaps HTML Imagemaps
include INCLUDE
inline Inline
input Data Input
input_postgis PostGIS/PostgreSQL
installation Installation
introduction An Introduction to MapServer
irc IRC
join JOIN
kml KML - Keyhole Markup Language
label LABEL
layer LAYER
legend LEGEND
legend_utility legend
license License
linux Linux
lists Mailing Lists
management File Management
map MAP
map_context Map Context
mapcache_formats MapCache Image Formats
mapcache_jpeg_format MapCache JPEG Format
mapcache_png_format MapCache PNG Format
mapcache_mixed_format MapCache Mixed PNG+JPEG Format
mapcache_caches MapCache Cache Backends
mapcache_grids MapCache Tile Grids
mapcache_cache_sqlite MapCache Sqlite Caches
mapcache_cache_disk MapCache Disk Caches
mapcache_cache_memcache MapCache Memcache Caches
mapcache_cache_tiff MapCache TIFF Caches
mapcache_services MapCache Tile Services
mapcache_ve MapCache VirtualEarth Quadkey Tile Service
mapcache_tms MapCache Tile Map Service
mapcache_kml MapCache Super-Overlay KML Services
mapcache_wmts MapCache OGC WMTS Service
mapcache_wms MapCache OGC WMS Service
mapcache_gmaps MapCache GoogleMaps Tile Service
mapcache_proxying MapCache Proxying Support
mapcache_dimensions MapCache Tileset Dimensions
mapcache_featureinfo MapCache FeatureInfo Requests
mapcache_tile_assembling MapCache Tile Assembling
mapcache_sources MapCache Input Sources
mapcache_source_wms MapCache WMS Source
mapcache_source_mapfile MapCache Native MapServer Sources
mapcache_http MapCache HTTP Request
mapcontext_cgi Map Context Files
mapfile Mapfile
mapfile_tuning Mapfile
mapinfo MapInfo
mapscript MapScript
mapscript_introduction Introduction
mapscript_ows MapScript Wrappers for WxS Services
mapscript_tests MapScript Unit Testing
mapserv mapserv
modindex Module Index
msencrypt msencrypt
mysql MySQL
ntf NTF
oci Oracle Spatial
oci_install Oracle Installation
ogc OGC Support and Configuration
ogc_support OGC Support
ogr OGR
online_resource_wms More About the Online Resource URL
optimization Optimization
osx Mac OS X
output Output Generation
outputformat OUTPUTFORMAT
pdf PDF Output
pgeo ESRI Personal Geodatabase (MDB)
php PHP MapScript
php_example By Example
php_install PHP MapScript Installation
php_install_example_steps Example Steps of a Full Windows Installation
projection PROJECTION
python Python MapScript Appendix
querying Querying
querymap QUERYMAP
raster Raster Data
raster_optimization Raster
reference REFERENCE
rfc1 MS RFC 1: Technical Steering Committee Guidelines
rfc10 MS RFC 10: Joining the Open Source Geospatial Foundation
rfc11 MS RFC 11: Support for Curved Labels
rfc12 MS RFC 12: C code Unit tests
rfc13 MS RFC 13: Support of Sensor Observation Service in MapServer
rfc14 MS RFC 14: Relative Coordinates for INLINE features
rfc15 MS RFC 15: Support for thread neutral operation of MapServer/MapScript
rfc16 MS RFC 16: MapScript WxS Services
rfc17 MS RFC 17: Dynamic Allocation of layers, styles, classes and symbols
rfc18 MS RFC 18: Encryption of passwords in mapfiles
rfc19 MS RFC 19: Style & Label attribute binding
rfc2 MS RFC 2: Creating line features and/or shapes using WKT
rfc21 MS RFC 21: MapServer Raster Color Correction
rfc22a MS RFC 22a: Feature cache for long running processes and query processing
rfc23 MS RFC 23: Technical Steering Committee Guidelines
rfc24 MS RFC 24: Mapscript memory management
rfc24first MS RFC 24: Mapscript memory management
rfc25 MS RFC 25: Align MapServer pixel and extent models with OGC models
rfc26 MS RFC 26: Version 5 Terminology Cleanup
rfc27 MS RFC 27: Label Priority
rfc28 MS RFC 28: Redesign of LOG/DEBUG output mechanisms
rfc29 MS RFC 29: Dynamic Charting Capability
rfc3 MS RFC 3: Feature Layer Plug-in Architecture
rfc30 MS RFC 30: Support for WMS 1.3.0
rfc31 MS RFC 31: Loading MapServer Objects from Strings
rfc32 MS RFC 32: Support for Anti-Grain Geometry (AGG) Rendering Engine
rfc33 MS RFC 33: Removing msLayerWhichItems() from maplayer.c
rfc39 MS RFC 39: Support of WMS/SLD Named Styles
rfc4 MS RFC 4: MapServer Raster Resampling
rfc40 MS RFC 40: Support Label Text Transformations
rfc41 MS RFC 41: Support of WCS 1.1.x Protocol
rfc42 MS RFC 42: Support of Cookies Forwarding
rfc43 MS RFC 43: Direct tile generation for Google Maps and Virtual Earth API
rfc44 MS RFC 44: Restore URL modification of mapfiles to pre-5.0 levels
rfc45 MS RFC 45: Symbology, Labeling, and Cartography Improvements
rfc46 MS RFC 46: Migrate Website to OSGeo
rfc47 MS RFC 47: Move IGNORE_MISSING_DATA to run-time configuration
rfc48 MS RFC 48: GEOTRANSFORM Geometry operations
rfc49 MS RFC 49: Symbology, Labeling, and Cartography Improvements
rfc5 MS RFC 5: MapServer Horizon Reprojection Improvements
rfc50 MS RFC 50: OpenGL Rendering Support
rfc51 MS RFC 51: XML Mapfile Format
rfc53 MS RFC 53: Guidelines for MapScript method return values
rfc54 MS RFC 54: Rendering Interface API
rfc55 MS RFC 55: Improve control of output resolution
rfc56 MS RFC 56: Tighten control of access to mapfiles and templates
rfc6 MS RFC 6: Color Range Mapping of Continuous Feature Values
rfc7 MS RFC 7: MapServer CVS Commit Management
rfc7.1 MS RFC 7.1: MapServer SVN Commit Management
rfc8 MS RFC 8: Pluggable External Feature Layer Providers
rfc9 MS RFC 9: Item tag for query templates
rfcs Request for Comments
runsub Run-time Substitution
s57 S57
scalebar SCALEBAR
scalebar_utility scalebar
sdts SDTS
search Search Page
section1 Section 1: Static Maps and the MapFile
section2 Section 2: CGI variables and the User Interface
shapefiles ESRI Shapefiles (SHP)
shp2img shp2img
shptree shptree
shptreevis shptreevis
sld SLD
sortshp sortshp
sos_server SOS Server
source Source
sponsors Sponsors
style STYLE
styleitemauto-label-styles Accessing OGR STYLEITEMAUTO Label Styles Through MapScript
svg SVG
svn Subversion
swig SWIG MapScript API Reference
sym2img sym2img
sym_construction Cartographic Symbol Construction with MapServer
sym_examples Symbology Examples
symbol SYMBOL
template Templating
testing Testing
tiger USGS TIGER
tile4ms tile4ms
tile_mode Tile Mode
tileindex Tile Indexes
tutorial MapServer Tutorial
unix Compiling on Unix
utilities Utilities
variable_sub Variable Substitution
vector Vector Data
vector_optimization Vector
vim VIM Syntax
virtual_vector Virtual Spatial Data
wcs_format WCS Use Cases
wcs_server WCS Server
web WEB
wfs WFS
wfs_client WFS Client
wfs_server WFS Server
win32 Compiling on Win32
windows Windows
wms_capabilities Validate the Capabilities Metadata
wms_client WMS Client
wms_server WMS Server
wms_time WMS Time
wrapper A Simple CGI Wrapper Script

Regenerating the reference labels

You can regenerate the reference labels by issuing:

make labels

from the docs directory like when you are building the html or latex versions