Extracting Administrative Boundaries from OpenStreetMap – Part 1

"Boundary Stone 1" taken by Tim Green (CC-BY-licensed from Flickr)

This is the first part of a two-part blog post about some of our work on making it easier to deploy FixMyStreet and MapIt in new countries.  This part describes how to generate KML for a given boundary in OpenStreetMap. Update: the second part is now available.

As mentioned in a previous post, we’re looking at ways of making it smoother to deploy FixMyStreet for a new country, city or use-case.  Essentially there are two fundamental bits of data that you need for this:

1. a mapping between a latitude and longitude (or postcode) to all the adminstrative areas that cover that point.
2. a mapping between problem type and adminstrative areas to an appropriate email address for reporting that problem.

The first of those is typically provided by a service called MapIt, an open source GeoDjango web application written by Matthew Somerville.  In the UK we are fortunate that official boundary data and the postcode database have now been released under an open data license from the Ordnance Survey.  However, in other many other countries similar data is unavailable, or not available under reasonable licensing conditions.  In such cases, though, all is not lost thanks to the extraordinary work of contributors to the OpenStreetMap project.  OpenStreetMap contains high-quality administrative boundary data for many countries of the world, and we know that data submitted to the project is available under the Creative Commons Attribution-ShareAlike license, so we can reuse it in a web service like MapIt.

The first step towards being able to build an instance of MapIt based on OpenStreetMap boundary data is to be able to generate a shapefile that represents a boundary in the project.  (In OpenStreetMap’s data model, boundaries are represented as either ways or relations, and those that we are interested in are tagged with boundary=administrative.)  Matthew had previously written code to generate KML files for boundaries in Norway in order to help to set up an instance of MapIt for Norway, which is used by FiksGataMi.  However, that script was quite specific to the organization of boundaries in that country, and it did not deal with more complex boundary topologies (e.g. enclaves), or different representations of boundaries (e.g. multiply nested relations).

So, Matthew and I wrote a new version of the code to extract a boundary from OpenStreetMap and generate a KML representation of it. The new version uses the Overpass API instead of XAPI, since it allows us to specify multiple predicates in the query and recursively fetch the ways and nodes that are contained in a relation.  Once all the ways that make up a relation have been fetched (ignoring those with roles like “defaults” or “subarea”), the script tries to join each unclosed way to any other with which it shares an endpoint.  We should end up with a series of closed polygons – the script exits in error if there are any unclosed ways left.  We can then directly create KML from these polygons, the only subtlety being that we need to mark certain boundaries as being an inner boundary (i.e., creating a hole in a boundary) if they had the role “enclave” or “inner” in an OpenStreetMap relation. For example, the South Cambridgeshire District Council boundary has a Cambridge City Council-shaped hole in it:

Similarly, the script has to cope with multiple distinct polygons, such as the boundary of Orkney.

If you want to use this code to generate a KML representation of a closed way or boundary relation from OSM, just clone the MapIt repository and run bin/boundaries.py:

$ bin/boundaries.py --help
Usage: boundaries.py [options]

Options:
  -h, --help            show this help message and exit
  --test                Run all doctests in this file
  --relation=<RELATION_ID>
                        Output KML for the OSM relation <RELATION_ID>
  --way=<WAY_ID>        Output KML for the OSM way <WAY_ID>

For example, to generate a KML boundary for the Hottingen area of Zürich, you can do:

$ bin/boundaries.py --relation=1701449 > hottingen.kml

In the next blog post in this series, we will discuss extracting such boundaries en masse and creating a service based on them.