Generate vector tilesets from OpenStreetMap data on Ubuntu 20.04 using OpenMapTiles
I’ve been playing around some more with OpenStreetMap data. In style with the last few tech-related posts, this is just a personal memo on how to get the OpenMapTiles pipeline running on a basic AWS box to generate MVT tiles.
This was tested on a c5.4xlarge box with ami-0a3a4169ad7cb0d77.
Install docker and docker-compose
sudo apt-get -y update
sudo apt-get -y dist-upgrade
sudo apt-get install -y apt-transport-https ca-certificates curl gnupg-agent software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io
sudo usermod -aG docker $(whoami)
sudo apt-get install -y jq
COMPOSE_VERSION=$(curl -s https://api.github.com/repos/docker/compose/releases/latest | jq -r .tag_name)
sudo curl -L "https://github.com/docker/compose/releases/download/$COMPOSE_VERSION/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
Now log out and back in and you should be able to run docker ps.
Generate the tiles
You again need to get an Interline token, then do
export TOKEN="..."
# required for openmaptiles
sudo apt-get install -y make
# get openmaptiles
git clone https://github.com/openmaptiles/openmaptiles.git
cd ./openmaptiles
sed -i "/MAX_ZOOM=/c\MAX_ZOOM=14" .env
mkdir -p data
wget -O data/data.osm.pbf "https://app.interline.io/osm_extracts/download_latest?string_id=new-york_new-york&data_format=pbf&api_token=$TOKEN"
make generate-bbox-file
sudo chown -R $(whoami) .
./quickstart.sh data
You now have a tiles.mbtiles in data.
(Optional) Extract the .pbf tile files
cd ~/openmaptiles/data/
sudo apt-get install -y python
git clone https://github.com/mapbox/mbutil.git
./mbutil/mb-util --image_format=pbf tiles.mbtiles tiles
cd tiles/
# un-gzip them
gzip -d -r -S .pbf *
# rename them to a better filename
find . -type f -exec mv '{}' '{}'.pbf \;
# lazy
mv metadata.json{.pbf,}
You’ll want to add a tiles.json to the root here, something like the linked file. Be sure to change the example.com to your own domain.
(Optional) Upload to S3
# install aws cli
sudo apt-get install -y unzip
mkdir -p /tmp/awscli
pushd /tmp/awscli
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
popd
# make sure your credentials are set up
aws s3 cp . s3://your-bucket/ --recursive
Make sure to fix up the your-bucket above.