Data Quality
Data quality measures and improves the accuracy, completeness, consistency, timeliness, and validity of data assets. It applies profiling, validation rules, monitoring, and remediation processes so that downstream consumers can trust the data they build on.
itData engineering and analytics | OpenSkills.info
Intro
Data Quality
Data quality tells you whether data is fit for a specific purpose. The same dataset can support one decision and fail another. Quality starts with the user, the intended use, and the harm caused by a wrong result.
That makes data quality more than cleaning. Cleaning changes data. Data quality management defines expectations, measures conformance, communicates limitations, and improves the processes that create defects.
Why data quality matters
Data moves through collection, storage, transformation, analysis, sharing, and retirement. A defect introduced early can travel through every later stage. A polished dashboard cannot repair an incorrect source value.
Poor quality can weaken decisions, interrupt services, increase manual work, and reduce trust. Unknown quality creates a second problem: users cannot tell which decisions the data can safely support.
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://www.gov.uk/government/publications/the-government-data-quality-framework/the-government-data-quality-framework
Supports
- Data quality is fitness for purpose and depends on user needs
- Data quality is broader than data cleaning
- Quality should be assessed and communicated throughout the data lifecycle
- The six core dimensions are completeness, uniqueness, consistency, timeliness, validity, and accuracy
- Completeness and accuracy are distinct, and validity does not establish accuracy
- Quality dimensions can trade off according to intended use
- Continuous improvement includes regular assessment, communication, and early correction
- https://www.gov.uk/government/publications/the-government-data-quality-framework/the-government-data-quality-framework-guidance
Supports
- Action plans identify critical data, rules, baselines, goals, and ongoing monitoring
- Root-cause analysis should trace the data journey and move fixes toward the source
- Quality results and caveats should be communicated to users
- Metadata helps users interpret data and its limitations
- https://www.gov.uk/government/publications/implement-a-data-quality-action-plan/data-quality-action-plan-implementation-guide
Supports
- A data quality action plan follows seven steps from critical-data identification through repeated measurement
- Rules align with user needs, business goals, dimensions, methods, and percentage targets
- Targets should describe realistic fitness for purpose instead of absolute perfection
- Data owners or process owners oversee plans with stewards, custodians, analysts, and subject experts
- Stewards and subject experts define business rules while technical experts implement checks
- Repeated assessment, user feedback, issue tracking, and change management sustain quality
- https://www.w3.org/TR/vocab-dqv/
Supports
- Dataset quality information helps users judge fitness for purpose
- A dimension is a criterion for assessing quality
- A metric defines how a quality dimension is measured
- A quality measurement evaluates specific data against a metric and records a value
- Quality metadata can group policies, measurements, certificates, and annotations
- Provenance can connect derived metrics, measurements, and annotations
- https://www.iso.org/standard/81745.html
Supports
- ISO 8000-1 provides the current overview of the ISO 8000 series
- The overview covers data-quality principles, a path to data quality, the series structure, and relationships to other standards