Data Quality

Data quality tells you whether data is fit for a specific purpose. The same dataset can support one decision and fail another. Quality starts with the user, the intended use, and the harm caused by a wrong result.

That makes data quality more than cleaning. Cleaning changes data. Data quality management defines expectations, measures conformance, communicates limitations, and improves the processes that create defects.

Why data quality matters

Data moves through collection, storage, transformation, analysis, sharing, and retirement. A defect introduced early can travel through every later stage. A polished dashboard cannot repair an incorrect source value.

Poor quality can weaken decisions, interrupt services, increase manual work, and reduce trust. Unknown quality creates a second problem: users cannot tell which decisions the data can safely support.

You manage this risk by making quality observable. Define rules for critical data, measure them, assign owners, record issues, and show users the results.

The six core dimensions

The UK Government Data Quality Framework uses six core dimensions. Treat them as lenses, not a universal scorecard.

• Completeness: required records and critical values are present.
• Uniqueness: a record appears only once when the model requires one record per entity or event.
• Consistency: values do not contradict one another within or across data assets.
• Timeliness: data is current and available soon enough for its intended use.
• Validity: values follow the expected type, format, range, or permitted set.
• Accuracy: values match the real-world facts they represent.

These dimensions are independent. A date can use a valid format yet describe the wrong day. A complete customer table can contain an inaccurate address for every customer.

Different uses also create trade-offs. A rapid operational feed may favor timeliness over completeness. A regulated report may require a later cutoff so more records can be reconciled. State the trade-off instead of hiding it inside one average score.

From a need to a measurement

Start with a critical decision or process. Identify the data items it depends on and the people who use them. Then turn each important expectation into a data quality rule.

A rule names what you measure, the scope, the method, and the acceptance target. For example: “At least 99 percent of active orders have a recognized customer identifier by the daily reporting cutoff.” This rule connects completeness, business context, and time.

A metric defines how you calculate a result. A measurement is the result for a particular dataset or run. W3C's Data Quality Vocabulary separates dimensions, metrics, and measurements so quality evidence can travel with published data.

Targets express risk tolerance. They should reflect user needs and business impact, not an arbitrary desire for perfect data. A failed target becomes a quality issue that you can prioritize and investigate.

The operating loop

Use a repeatable loop:

1. Identify critical data and its intended use.
2. Define rules and targets with business and technical experts.
3. Profile and measure a baseline.
4. Record failures with their scope, impact, and evidence.
5. Find the root cause and improve the process near the source.
6. Communicate results and known limitations to users.
7. Repeat measurements and watch for change.

Tests at ingestion and transformation boundaries can detect many failures early. They cannot prove every value is true. Accuracy often needs comparison with an authoritative reference, direct observation, or confirmation from a responsible source.

People and accountability

Data quality crosses business and technical boundaries. Data owners set priorities and approve standards. Data stewards translate needs into definitions and rules. Data custodians implement technical controls and checks. Analysts profile results and look for patterns.

Job titles vary. The essential point is explicit responsibility for the rule, the data, the response, and communication with users.

Where data quality fits

Data governance supplies authority, policy, and decision rights. Metadata explains meaning, origin, and limitations. Data engineering implements controls in pipelines and platforms. Observability shows changes and failures over time.

Data quality connects these practices around evidence of fitness for purpose. A tool can automate checks, but it cannot choose the purpose, acceptable risk, or correct business meaning for you.

Limits and next steps

No fixed set of dimensions makes every dataset good. Quality depends on context, and context changes. New users, source systems, policies, and transformations can make yesterday's target inadequate.

Begin with one important data asset. Write a few rules tied to real decisions. Measure them consistently. Then study action planning, issue prioritization, root cause analysis, and quality metadata through the official resources in this course.