Architecture and infrastructure
Design a new platform or improve the one you have, choosing managed cloud, open source or a hybrid around your data, team and operating budget
Data Engineering
Data platforms
from design to daily use
I design and build data platforms, connect new sources and modernise existing pipelines, from the first architecture decisions to day-to-day operation
Where I can help
Pipelines and data models
Connect databases, SaaS APIs, files and events, then build the ingestion, orchestration and models that turn them into useful data
Migration and reliable operation
Move workloads in stages and check the results, with improvements to testing, access, monitoring, performance and cost where they matter
Examples from my work
Moving legacy reporting into dbt
Migrated analytics workloads from legacy SQL pipelines to dbt, with comparisons against existing outputs and staged changes to downstream reporting
A managed workflow on dbt Core
Built scheduled runs, CI/CD and centralised logs around dbt Core, giving the team a managed workflow without dbt Cloud
Technology
Python · SQL · dbt · Dataform · Airflow · Airbyte · Terraform
Data APIs, internal tools and integrations can be included in the same scope when your team needs to use data outside reporting
From the current stack to a working change
Map the requirements
Review sources, consumers, existing infrastructure, budget and the team who will operate the platform
Decide what to tackle first
Agree the first piece of work and how we will check it, whether that is a new pipeline, a migration or a change to the existing platform
Build and hand over
Implement and validate the change, with documentation, operating guidance and any ongoing support agreed separately
Work can start with a technical assessment, a focused build or ongoing engineering alongside your team
Let’s talk
What needs to work better
in your data platform?
Share the current stack, the problem and what depends on the data, so we can define a useful first stage
Discuss your data platform