Mostly harmless, conspicuously useful

The Hitchhiker's Guide to Becoming a Data Analyst

Welcome to data analysis, the profession in which an innocent request for “the number” is escorted through several databases and returns with three definitions, a caveat, and a meeting. Data analysts turn imperfect records into evidence people can actually use, beginning with a precise question and proceeding through rows, definitions, calculations, comparisons, and the tactful discovery that the obvious answer counted every customer twice. The finished artifact may be a spreadsheet, SQL query, chart, dashboard, or recommendation, each of which can look impressively certain while resting on a denominator last discussed during a reorganisation. Progress means accepting larger territories of ambiguity without surrendering accuracy: first making one bounded answer trustworthy, then governing shared metrics and dashboards, and eventually advising leaders when evidence, incentives, and organisational enthusiasm have selected different destinations.

Level 1 · Novice

Establish what a row is before allowing it an opinion.

Novice analysts answer tightly bounded questions by making small datasets consistent, checking obvious defects, and showing their calculation clearly enough that the spreadsheet cannot revise history when nobody is looking.

In novice work, you turn a limited question into a dependable table for one nearby stakeholder, perhaps an operations coordinator asking how many orders arrived late last week. First inspect what each row represents, because a row that looks like an order may privately be a status update with several siblings. Standardise dates and labels, preserve the raw input, record every cleaning choice, and check the total against a known source. A modest answer with visible assumptions beats a magnificent figure assembled from duplicates, blanks, and a column whose heading says “Final2.”

Suppose a delivery export stores one row per status update rather than one per order, while several dates have been converted into spreadsheet serial numbers with the serene confidence of a clerk stamping the wrong passport. Keep an untouched copy, identify orders by unique ID, convert dates explicitly, remove duplicate events under a written rule, and reconcile the result with the source system. The checked evidence shows 43 late orders, not the pivot table’s emphatic 71, so the coordinator avoids scheduling needless overtime and the pivot table is thanked for its initiative rather than its arithmetic.

Words from the spaceship manual, translated

Row grain: The real-world thing one row represents, such as an order, customer, or status event. Grain is the dataset’s unit of reality; mix two grains and totals begin acquiring extra limbs.
Dirty data: Records containing missing values, duplicates, inconsistent labels, impossible dates, or stale entries. “Dirty” describes analytical risk, not moral character, despite what the validation report implies.
Spreadsheet coercion: An automatic conversion that changes an entered value, such as turning a product code into a date or removing leading zeroes. The spreadsheet calls this assistance and does not wait for consent.