pytest data quality tests failed in CI
A pytest suite that asserts data quality (row counts, null rates, referential integrity) failed because a DataFrame or query result did not meet the assertion. The test is catching a real defect, or the data was not produced before the test ran.
What this error means
A pytest step fails an assertion on a DataFrame or query result, for example a non-zero null count or an unexpected row count, with the AssertionError showing the actual value.
def test_no_null_customer_id():
> assert df["customer_id"].isnull().sum() == 0
E assert 412 == 0
E + where 412 = ...isnull().sum()Common causes
The data genuinely fails the assertion
Nulls, duplicates, or wrong counts exist because of an upstream data or transform defect the test is designed to catch.
Data was not built before the test
The test queries a table that transforms had not populated yet in CI, so the assertion sees empty or partial data.
How to fix it
Produce data before testing
- Run the transforms or fixtures that populate the data.
- Then run the pytest suite that asserts on it.
- Fix the upstream defect the assertion surfaced if it is real.
- run: dbt build # populate tables
- run: pytest tests/data # then assert on themMake assertions report offending rows
Include the failing sample in the assertion message so CI logs show what broke, not just the count.
bad = df[df["customer_id"].isnull()]
assert bad.empty, f"{len(bad)} rows with null customer_id: {bad.head()}"How to prevent it
- Sequence data production before quality tests.
- Report offending rows in assertion messages.
- Keep data fixtures deterministic so tests are stable.