At a glance
- 49dbt models
- 37Sources
- 7Serving tables
- 91Lineage edges
- 3Markets
- WeeklyCadence
The problem
Finance planning lives in spreadsheets; actuals live in the warehouse. Reconciling the two into a weekly view was a legacy Excel process run by an analyst: manual, slow, easy to get subtly wrong, and because it was assembled by hand nobody could say where a number came from.
The rebuild treats the spreadsheets as real sources: ingested, staged and tested like warehouse tables, so every figure has a traceable path back to its origin.
How it runs
Orchestration
Once a week. Quality checks bracket the transform, and every task reports its status.
How it is modelled
Model lineage
One market's subtree, simplified. All three markets repeat the same source → staging → intermediate → serving shape, read by a Tableau dashboard and an emailed Excel report.
Models are filled; what consumes them is outlined. Names are simplified placeholders.
Design decisions
Data quality gates on both sides of the transform
Checks run after ingestion and again after transformation. The ingestion checks are deliberately non-blocking: a failed expectation on a hand-maintained spreadsheet should raise a signal, not stop the whole run. Post-transformation checks guard the models the business actually reads.
Two very different kinds of source, one standard
Planning inputs live in spreadsheets maintained by Finance; actuals come from several warehouse tables on the lakehouse. Both are ingested as first-class sources, staged the same way and held to the same tests, so a hand-typed target and a warehouse fact are equally traceable.
Delivered where people already are
The pipeline ends in a serving layer, with two things on top: a Tableau dashboard for anyone who wants to explore, and a formatted Excel report generated on Databricks and emailed to stakeholders who were never going to open a BI tool.
Failures surface before users find them
Run status is emitted to Datadog and a failed task raises a Slack alert. The weekly cadence makes this matter more, not less: with only one scheduled run a week, a silent failure would mean stale numbers in front of Finance for seven days.
One shape, many markets
Each market has its own subtree following the same source → staging → intermediate → serving pattern, and all three run in production. Because the shape is shared, a change to how a metric is derived lands the same way everywhere instead of being reimplemented per country.