London Datastore Pipelines & Cool Spaces Infrastructure
Python ETL and open-data publishing for the London Datastore, alongside spatial infrastructure for the separate Cool Spaces initiative.
- Scope
- Greater London Open Data
- Infrastructure
- ArcGIS Enterprise & AWS
- Database
- PostgreSQL & PostGIS
- python
- postgresql
- postgis
- arcgis-enterprise
- aws
- london-datastore
- rest-apis
Evidence
Overview
At the Greater London Authority in 2021, I worked as a Data Engineer & Analyst on London Datastore pipelines, spatial services and Cool Spaces infrastructure. These responsibilities supported Greater London open data and civic mapping, with coverage determined by each dataset and application.
London Datastore: Publishing & Spatial Services
I developed Python ETL pipelines to ingest, clean and validate municipal and demographic data for publication on the London Datastore. The work included data enrichment and checks to support predictable updates for downstream users.
I also administered cloud-hosted ArcGIS Enterprise software supporting interactive maps, spatial REST APIs and data visualisations. Infrastructure responsibilities included PostgreSQL/PostGIS databases, AWS deployments, API documentation and automated testing routines.
The London Datastore publishes open datasets, dashboards and spatial information for Greater London.
Cool Spaces: Heat-Refuge Mapping
Separately, I engineered spatial data infrastructure and analytics for Cool Spaces, supporting the identification and cataloguing of public refuge locations during heatwaves.
The Mayor’s 2021–2022 annual report records registered Cool Spaces sites across 19 boroughs in 2021. That is programme context, rather than a count of boroughs covered by my individual pipelines or a claim about every London Datastore service.
Data Stewardship
Publishing open data requires clear schemas, provenance and validation so users can interpret updates consistently. The practical focus was maintaining the data pipelines, databases and services needed for those releases, with the scope of each output described separately.