dbt Fundamentals

dbt transforms data that is already in a data platform into trusted datasets for analytics, operations, and AI. You write transformation logic as SQL select statements. dbt compiles that logic, runs it in the data platform, and records how the pieces depend on each other.

That position matters. Extract and load tools move source data into a warehouse or another supported data platform. Business intelligence tools query prepared data and present it to people. dbt sits between them. It turns loaded data into documented, tested data products without becoming the system that extracts data or the dashboard that displays it.

The mental model

Think of a dbt project as a software project whose output is reliable data.

Each model contains transformation logic. A SQL model is usually one select statement in one file. When dbt runs the model, a materialization controls how the result is represented in the data platform, such as a view or table.

Models rarely stand alone. The ref function names another model as an input. dbt uses those references to build a directed acyclic graph, or DAG. The graph captures dependency order. If an orders summary depends on cleaned orders, dbt knows to build the cleaned model first.

A source names data loaded by an extract and load tool. The source function connects a model to that input and adds the relationship to the DAG. This distinction keeps external inputs separate from models your project owns.

YAML property files add descriptions and data tests to project resources. A data test is an assertion about the resulting data. Built-in generic data tests cover common expectations such as non-null values, uniqueness, accepted values, and relationships between resources. A passing data test returns no failing rows.

The result is more than a folder of SQL. The project carries transformation logic, dependencies, tests, documentation, and configuration together. Version control and code review can then govern changes to analytics logic.

A small flow

Imagine an application database copied into a warehouse. The loaded tables contain customers, orders, and payments.

1. Declare the loaded tables as sources.
2. Build staging models that rename fields and normalize types.
3. Reference those staging models from intermediate models that join or aggregate data.
4. Build a customer-level model for reporting.
5. Add data tests for keys, required values, and relationships.
6. Add descriptions so downstream users can understand the result.
7. Run a build and inspect failures before merging the change.

This layered design makes business logic visible. A revenue definition no longer hides inside one dashboard. It becomes reviewed code that other models can reference.

The dbt framework and ways to run it

The dbt framework includes a language and an engine. The language covers SQL, Jinja templates, YAML configuration, tests, and related project resources. An engine parses the project, compiles its logic, executes the graph, and produces metadata.

Current dbt documentation describes the dbt Core engine and the dbt Fusion engine. dbt Core version one is the open-source Python engine familiar to many existing projects. Fusion is a Rust-based engine with native SQL analysis and editor features. The managed dbt platform adds development, scheduling, continuous integration, documentation hosting, monitoring, and other operational features around project execution.

The framework concepts in this course apply across those choices. Product availability and supported features change, so check the current official documentation before selecting an engine or deployment path.

Core commands and feedback loops

dbt run builds models. dbt test runs data tests. dbt build runs selected models, tests, seeds, snapshots, and supported user-defined functions in DAG order. During a build, an upstream test failure can cause dependent resources to skip. That behavior prevents dbt from knowingly building downstream results on a failed dependency.

Selection syntax lets you narrow work to part of the graph. This is useful during development, but a narrow selection can miss effects elsewhere. Continuous integration should validate the relevant change against its dependencies and downstream impact.

dbt also produces artifacts that describe the project and execution results. Generated documentation can combine resource descriptions, model code, tests, warehouse metadata, and the project DAG. These outputs help both people and tools inspect what happened.

Where dbt helps

dbt is a strong fit when your data already lands in a supported data platform and SQL is the main transformation language. It helps teams centralize business logic, reuse models, review changes, test assumptions, and expose lineage.

It does not replace ingestion, storage, access control, warehouse administration, or business intelligence. It also does not make incorrect business logic correct. Tests only enforce the assertions you write, while documentation only helps when descriptions remain accurate.

The data platform still executes most transformation work. Your SQL dialect, permissions, compute costs, and performance behavior therefore depend on that platform. A successful dbt command proves that declared work completed. It does not prove that every business requirement was modeled.

Your learning path

Start with one official quickstart for your data platform. Build a small model and inspect the result. Then learn ref, sources, materializations, data tests, documentation, and resource selection in that order. After those foundations, study deployment, continuous integration, incremental models, project architecture, packages, macros, contracts, and governance.