Analytics Engineering
Analytics engineering applies software engineering practices to the transformation layer between raw data and analytics. It uses version-controlled SQL or Python to build tested, documented data models that analysts and data scientists can trust without needing to understand every source system.
itData engineering and analytics | OpenSkills.info
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Don't Panic
Don't Panic — Analytics Engineering
Somewhere between the database that runs your app and the dashboard your CFO stares at during a board meeting, somebody has to translate. Raw systems record transactions: a signup, a charge, a status change. Executives ask about customers, revenue, and trends. Those are not the same language, and translating between them by hand, one query at a time, is how this discipline got invented.
Before it had a name, the translation happened anyway, badly. An analyst wrote a query. Someone copied its logic into a dashboard. Someone else copied a slightly different version into another one. Eventually finance and the executive team were staring at two numbers for the same thing, trusting neither, and nobody had decided any of this should happen.
The whole discipline rests on one idea: grain, the exact meaning of one row. Join a monthly total onto a daily table and nothing errors. The query runs, the number is wrong, and it looks exactly as confident as the right one would have. Most mistakes in this field trace back to somebody skipping the sentence "one row represents ______" and going straight to the SQL.
Transformations then move through layers, and the names are less exotic than they sound. Staging renames and standardizes one source and nothing more. Intermediate models do the joining nobody downstream should have to see. Marts are the finished, business-shaped tables people actually query. Ten layers of unclear SQL are not more rigorous than one clear layer — the point is boundaries, not ceremony, and stacking them for their own sake is how a simple translation grows a cathedral nobody asked for.
Here is the part that catches people out: the product of all this work is not the transformation. It is trust. A dashboard reporting 107,102 visits next to one reporting 106,988 does not just cost you 114 visits' worth of accuracy. It costs you the reader's confidence in every number after it, which is why they quietly go build their own spreadsheet instead. Tests and documentation exist to keep that trust intact, not to prove the code compiles. A passing test proves the platform ran your query. It does not prove anyone should believe the answer.
None of this requires a particular tool. Whatever runs the transformations, the same four moves apply: declare the grain before writing anything, keep each piece of logic in exactly one layer, test the assumption rather than the syntax, and write down what a model means for the person who cannot read the code that built it.
Read the Intro next for the full architecture and where the role's edges actually sit. The Cheatsheet holds the reference tables for when you are mid-build and need an answer, not a lecture. Field Notes covers what goes wrong in practice, including why the fix for one kind of drift quietly creates another.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://www.getdbt.com/blog/what-is-analytics-engineering
Supports
- Analytics engineers transform, test, deploy, and document data for end users
- The role applies software engineering practices to analytics code
- The role bridges data engineering and data analysis responsibilities
- https://docs.getdbt.com/docs/introduction
Supports
- Modular warehouse transformations and data products
- Version control, testing, modularity, continuous integration, deployment, and documentation in analytics workflows
- Project context including lineage, tests, contracts, metrics, and governance
- https://docs.getdbt.com/best-practices/how-we-structure/1-guide-overview
Supports
- Transformation from source-conformed to business-conformed data
- Modular layers that apply each transformation in one place
- Consistent project structure as an explicit design choice
- https://docs.getdbt.com/docs/build/data-tests
Supports
- Data tests as assertions about models and other project resources
- Built-in checks for non-null, unique, relationship, and accepted-value conditions
- Custom SQL assertions for organization-specific business logic
- https://docs.getdbt.com/docs/build/documentation
Supports
- Documentation for consumer discovery and understanding
- Generated project information including model code, tests, and dependency graphs
- Warehouse metadata and human-authored descriptions for models, columns, and sources
- https://docs.getdbt.com/docs/deploy/source-freshness
Supports
- Freshness checks against organization-defined service expectations
- Source freshness as a distinct job concern rather than an implication of model execution
- https://www.getdbt.com/resources/the-analytics-development-lifecycle
Supports
- Software engineering practices across an analytical workflow
- Reusable data assets produced for downstream business value
- Collaboration among engineering, analysis, and decision-making responsibilities
- https://www.getdbt.com/product/semantic-layer
Supports
- Central definition of governed metrics and business logic
- Reuse of metric definitions across analytical consumers
- https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/
Supports
- Facts, dimensions, star schemas, conformed dimensions, and fact table patterns
- Dimensional modeling techniques for analytical data
- https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/four-4-step-design-process/
Supports
- Selection of business process, declaration of grain, identification of dimensions, and identification of facts
- Business needs and source realities as inputs to dimensional design
- https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/grain/
Supports
- Grain as the exact meaning of one fact table row
- The requirement that dimensions and facts remain consistent with grain
- Separation of different grains into different fact tables
- https://www.gov.uk/government/publications/the-government-data-quality-framework/the-government-data-quality-framework
Supports
- Data quality as fitness for purpose
- Completeness, uniqueness, consistency, timeliness, validity, and accuracy as distinct dimensions
- Quality trade-offs based on user and business needs
- Ongoing monitoring, reporting, metadata, and treatment of issues at source
- https://bitol-io.github.io/open-data-contract-standard/v3.0.0/home/
Supports
- Data contracts as agreements between producers and consumers
- Contract sections for schema, quality, service levels, ownership, support, roles, and infrastructure
- https://docs.getdbt.com/docs/cloud/about-cloud
Supports
- dbt Cloud provides managed development and deployment workflows for dbt projects
- dbt Cloud supports collaboration, testing, scheduling, and observable production jobs
- https://docs.cloud.google.com/dataform/docs/overview
Supports
- Dataform develops, tests, versions, schedules, and documents SQL transformation workflows in BigQuery
- Dataform resolves workflow dependencies and runs assertion queries against BigQuery tables and views
- https://docs.coalesce.io/docs
Supports
- Coalesce Transform combines a visual DAG and code for governed data transformation pipelines
- Coalesce includes reusable patterns, testing, contracts, lineage, deployment, and refresh workflows
- https://sqlmesh.readthedocs.io/en/stable/concepts/overview/
Supports
- SQLMesh is an open-source SQL and Python data transformation framework
- SQLMesh plans identify affected models and computations, and its tests and audits check transformation code and results
- https://docs.cube.dev/docs/data-modeling/overview
Supports
- Cube provides a semantic layer that turns warehouse tables into governed metrics and dimensions for multiple consumers
- Cube models measures and dimensions as reusable data definitions
- https://benn.substack.com/p/data-is-for-dashboards
Supports
- Small discrepancies between two dashboards reporting the same metric undermine trust in the whole reporting layer, not only in the two conflicting numbers
- Dashboard consistency work is frequently undervalued relative to exploratory or strategic analysis, despite being foundational to decision-making
- https://benn.substack.com/p/the-data-config
Supports
- Hardcoded business logic such as session windows, ad-attribution lookback periods, and customer segmentation thresholds gets duplicated across queries because centralizing it is slower than shipping the query at hand, and the duplicated definitions drift
- Centralizing a definition into one governed model can make it inaccessible to the non-technical stakeholders who most need to check it, since they cannot read SQL or query the model directly
- https://docs.getdbt.com/blog/deprecating-dbt-metrics
Supports
- dbt Labs deprecated its Jinja-macro-based dbt_metrics package because the templating approach could not support constructing metrics from measures across multiple semantic models
- The replacement, MetricFlow, generates SQL through a dedicated query planner and dataflow plan instead of templated macro substitution
