Data Audit
Transparency report on data quality and integrity
AustraliaMetrics tracks 63 national performance metrics across 8 categories, drawing from 12+ authoritative sources including the Australian Bureau of Statistics, Reserve Bank of Australia, World Bank, and Productivity Commission. Our database holds over 9,900 native observations spanning 26 years, expanded to 45,900+ monthly records through carry-forward derivation.
We run comprehensive audits across this data to confirm its accuracy, currency, completeness, and internal consistency. These audits are not just internal checks — we publish the results here so that citizens, journalists, researchers, and policymakers can have confidence in the numbers they see on this dashboard.
What We Audit
Every audit evaluates data quality across four dimensions. Each dimension is scored from 1 to 5. The scores shown below are the actual results from our most recent audit on 31 March 2026, in which automated scripts queried every table in the production database across all 63 metrics.
Is the data up to date? We check every table for staleness against its expected refresh interval — weekly for ABS and RBA, quarterly for government services, annually for World Bank.
Audit result: Core monthly data (ABS, RBA) is fresh within expected windows. Several quarterly metrics remain naturally stale due to source agency publication delays (population growth, labour productivity, dwelling completions). Freshness endpoint now uses UTC consistently, preventing timezone-dependent staleness miscalculation on edge-region deployments.
Is the data correct? We sample-verify stored values against live source APIs, scan for statistical outliers (3-sigma), and validate that negative and zero values are explainable.
Audit result: All sampled values matched their source APIs exactly. No 3-sigma outliers detected. All negative values (e.g. budget deficits, deflation periods) are legitimate. Scoring engine percentage-of-baseline mode fixed — division-by-zero guard added, and unfavorable-trend RED escalation rule removed to prevent metrics near the green threshold from incorrectly skipping AMBER.
Is all expected data present? We check NULL rates across every column, verify all 8 states are covered for state-level metrics, and confirm every metric has a scoring rule.
Audit result: All 63 metrics have scoring rules (confirmed present). Some state-level coverage gaps remain where source data is not published by state. A few ROGS metrics have very sparse histories (1–3 data points).
Is the data internally consistent? We check for orphaned records, duplicates, out-of-range scores, referential integrity, and verify that category weights sum to 100.
Audit result: Zero orphaned records, zero duplicates, all scores within 0–100 range, all foreign keys valid, category weights sum to exactly 100.
How Audits Work
Our audit process is automated and reproducible. Purpose-built scripts query the production database directly via Prisma ORM, executing checks across every table and every metric.
- Freshness checks — compare the latest record timestamp in each table against the expected refresh interval (weekly, monthly, quarterly, or annual)
- Sample verification — fetch the same data points directly from source APIs (ABS SDMX, RBA CSV, World Bank REST) and compare against stored values
- Anomaly detection — scan all numeric columns for 3-sigma outliers, unexpected zeros, implausible negatives, and values outside known bounds
- NULL analysis — check every column in every table for unexpected NULL rates that might indicate broken ingestion
- Referential integrity — verify no orphaned records exist, no duplicates violate unique constraints, and all foreign keys resolve
- Score validation — confirm all scores fall within the 0–100 range, all traffic lights are valid (Red/Amber/Green), and category weights sum to 100
- Source health checks — test connectivity to all external APIs to confirm they are responding and have not changed their data format
Latest Audit Results
Last audit: 31 March 2026 · Overall score: 4.38 / 5 — Good · Tables audited: 19 · Metrics checked: 63
Key Findings
- ✓Zero orphaned records, zero duplicates, and zero out-of-range scores across all tables — referential integrity is perfect.
- ✓No 3-sigma statistical outliers detected in recent national data. All negative values (e.g. budget deficits, deflation periods) are explainable.
- ✓All 6 external data sources (ABS, RBA, World Bank, data.gov.au, NDIS, Medicare) responded successfully during health checks.
- !Scoring engine percentage-of-baseline mode had a division-by-zero risk when green/red thresholds were set to 0, and an overly aggressive RED escalation rule that penalised metrics with unfavorable trends even when their value was near the green zone.
- !Freshness health endpoint mixed local and UTC date methods, which could produce incorrect staleness results when deployed to non-UTC edge regions.
- !Several quarterly metrics remain naturally stale (8–12 months) due to delayed publication by source agencies (ABS, Productivity Commission).
- !A few ROGS metrics have very sparse observation histories (1–3 data points), limiting trend analysis accuracy.
Controls & Fixes Implemented
The following changes were made immediately following the audit to address its findings:
Scoring engine division-by-zero guard
Added safe divisor guards in the percentage-of-baseline scoring mode. When green or red thresholds are configured as 0, the score formula now uses a floor of 1 to prevent division by zero, ensuring valid score output for all threshold configurations.
Removed unfavorable-trend RED escalation rule
The scoring engine previously escalated metrics from AMBER to RED if their trend was unfavorable, even when the value was only slightly above the green threshold (within a hardcoded 5-point buffer). This caused metrics near the green zone to display as RED incorrectly. Traffic lights now reflect position within the threshold range only; trend direction is reported separately via the trend badge.
Freshness endpoint UTC consistency
Fixed the data freshness health check to use getUTCFullYear()/getUTCMonth() consistently instead of mixing local-time getFullYear()/getMonth() with Date.UTC(). This prevents incorrect staleness calculations when the endpoint runs on non-UTC edge regions.
Prior fixes confirmed working
Amber-zone gradient (prior audit), step-level pipeline logging, government-services scoring rules, and composite index timeout fix all confirmed present and functioning correctly.
The full technical audit report is maintained internally and available on request. For questions about data quality or to report a data issue, please contact us.
View all data sources on our Data Sources page, or check real-time ingestion status on the Data Status page.