Converting a regular carto project to vector tiles

OSM-carto case study

Rory McCann www.technomancy.org

Geofabrik geofabrik.de

OpenStreetMap Carto style

• Main main on openstreetmap.org
• Style has been grown for years
• Ported to CartoCSS by Andy Allen in 2012
• Probably one of the most complicated CartoCSS styles

Goal

Same output (raster images over HTTP), but served behind with vector tiles

Not client side JS / Open GL rendering

To allow one to make minor tweaks to the style without having to host a whole new style

Let's go!

How do we render it?

Kosmtik can render a vector tiles project into images-over-HTTP

tilelive - collection of libraries for rendering & serving tiles

tessera - command line tilelive-based tool for serving tiles.

Split the project into two

OSM-carto is just one project with a project.yaml

Vector tiles are 2 projects, source and style

osm-carto.tm2 and osm-carto.tm2source

Source project - osm-carto.tm2source

One data.yml (NB: .yml not .yaml)

Remove the top level source and styles keys

Set maxzoom: 14

Style project - osm-carto.tm2

One project.yml (NB: .yml not .yaml)

All .mss style files.

source is URLish to you tm2source project

All Layers only have the id

Source project - osm-carto.tm2source

One data.yml (NB: .yml not .yaml)

Remove the top level source and styles keys

Set maxzoom: 14

Style project - osm-carto.tm2

One project.yml (NB: .yml not .yaml)

All .mss style files.

source is URLish to you tm2source project

All Layers only have the id

Done?

ha!

sadface

Results

Results

No carto classes - ergo no roads

No carto classes

You can't use carto classes - only ids - to filter

Rewrite style to only use #id instead of .class filters

Not so bad for osm-carto

Lots of classes map 1:1 to layer ids anyway

Solved!

Buffer

Common problem in vector tiles

Each vector tile data has a clipped polygon/line for each object

Can only see data inside your tile

Except sometimes you need to see outside your tile

Or you can't render properly

Buffer

Buffer

It's a polygon (green). We want to draw a border.

It is spread across many vtiles (black)

Buffer

Polygon is cut. 4 new polygons

Buffer

Border of the polygon is rendered at tile borders

Solution - buffer-size

Add buffer-size: $PIXELS to each data layer properties

Value can be small (~4) for polygon boundaries

Large (~128) for label placement

Solution - buffer-size

Polygon now goes outside the image, border still draw.

But results are clipped out of final image

Problem: Centroid of a polygon

e.g. Labels for names, addresses, icons.

In raster tiles, using meta tiles, mapnik has the polygon, and can place the centroid once

Vector tiles: Each tile is separate render, each tile has its own polygon

Problem: Each tile will have a label in the middle

Solution: Change SQL to return a point, not a polygon

i.e. Have PostgreSQL, not Mapnik, convert the polygon to a point

You now have a point layer, and mapnik puts the label there

Only one point per polygon, regardless of how many v. tiles the polygon is on

SQL

Problem: Indexes

"Give me buildings in this bbox" → PostgreSQL can use geom index

"Give me ST_Centroid(way) in this bbox" → PostgreSQL cannot use index

Solution: Use an Index

Add AND way && !bbox! to WHERE clauses

Makes PostgreSQL filter based on the way column, and use the geom index

Or you can CREATE INDEX on ST_Centroid(way)

Zoom level

Data is being simplified

Fine for normal zoom level, but z15+ uses z14 data

Simplification is too much

Solution - maxzoom

maxzoom: 14 in tm2source

mapnik won't do simplication for z14 tiles

Initially I tried to use symlinks as a hack to vector-tile-ify this and use the same project.yaml for both the source & style

But source and style need different maxzoom.

Source needs maxzoom: 14 to prevent simplification

Style with maxzoom: 14 means no z15+ tiles

All data needs to be at z14

Each layer in the source has a minzoom

Needs to be minimum of 14

Otherwise data won't be included in z14 tile, which is uses to render z15+ tiles

Style can still filter on zoom for display purposes ([zoom>16])

Minor: way polygon type on planet_osm_polygon

With default osm2pgsql database, node-mapnik will do something strange and silly

Tries to autodetect geometry type by running the query and looking at first few rows

No bbox filtering, runs query globally

SELECT all buildings ORDER BY way_area DESC LIMIT 5

This is slow

osm2pgsql issue #573

Solution - Change geometry type

ALTER TABLE planet_osm_polygon ALTER COLUMN way TYPE geometry(MultiPolygon, 900913) USING ST_Multi(way);

Have a osm2pgsql fork which imports as MULITPOLYGON

Code has been released on GitHub

Will try to keep it updated with upstream style (no guarantees)

github.com/geofabrik/openstreetmap-carto-vector-tiles