Data quality checks to run before trusting a dataset

Most wrong analysis is not wrong maths — it is correct maths over data that did not mean what the analyst assumed. These checks take a few minutes and catch the errors that otherwise surface in a meeting.

1. Confirm the row count

Compare the row count against what the source system claims to have exported. A mismatch means truncation, a filter you did not know about, or a header row counted as data.

SELECT COUNT(*) AS rows FROM orders;

2. Check for duplicate keys

A column that is supposed to be unique frequently is not, and every join built on it will silently multiply rows. Check before joining, not after the totals look strange.

SELECT order_id, COUNT(*) AS occurrences
FROM orders
GROUP BY order_id
HAVING COUNT(*) > 1
ORDER BY occurrences DESC;

3. Look at nulls per column

Null rates tell you which columns are safe to aggregate. A column that is 60% null will produce an average that describes a minority of records while looking authoritative.

SELECT
  COUNT(*)                                   AS rows,
  COUNT(*) - COUNT(revenue)                  AS revenue_nulls,
  COUNT(*) - COUNT(order_date)               AS date_nulls,
  COUNT(*) - COUNT(customer_id)              AS customer_nulls
FROM orders;

4. Catch numbers that arrived as text

This is the most common and most damaging parsing error. If a numeric column was inferred as text, sorting and aggregation are both wrong, and neither fails loudly.

A cast that fails on any row tells you the column is not clean.

SELECT revenue
FROM orders
WHERE TRY_CAST(revenue AS DOUBLE) IS NULL
  AND revenue IS NOT NULL;

5. Range-check anything physical

Negative quantities, dates in 1899 or 2087, and percentages above 100 all indicate an upstream problem. Sentinel values like -1 and 9999 used to mean "unknown" are especially dangerous because they aggregate silently.

SELECT *
FROM orders
WHERE quantity < 0
   OR revenue < 0
   OR order_date < DATE '2000-01-01'
   OR order_date > CURRENT_DATE;

6. Review categorical values

Listing the distinct values of every column you plan to group by reveals the variants that split one real category into several: "UK", "U.K.", "uk", and " UK" are four groups in any GROUP BY.

SELECT region, COUNT(*) AS rows
FROM orders
GROUP BY region
ORDER BY rows DESC;

7. Check the date range is what you expect

Confirming the minimum and maximum date catches partial exports and timezone shifts. A report meant to cover a full year that stops in October is a truncated extract, not a business trend.

SELECT
  MIN(order_date) AS first_order,
  MAX(order_date) AS last_order,
  COUNT(DISTINCT order_date) AS distinct_days
FROM orders;

Running all of this at once

NoEgress profiles every column automatically when a file is opened — inferred type, null count, distinct values, and min/max — which covers checks 1 through 7 at a glance and tells you which columns deserve a closer query.