Sources, and what is wrong with each of them. More results are invalidated here than by any failure of statistics, and almost none of it is a statistics problem.
A series that looks clean is usually a series whose faults have not been looked for. Six recur, and all six are silent.
The six silent faults
Fault
What it does
Control
Look-ahead
Uses a figure before it was released, or a revision that did not exist yet.
Release timestamps, reads refused before them
Survivorship
Measures only the cases that lasted, so the failures vanish from the record.
Exits recorded with an outcome rather than dropped from the sample
Identifier drift
Labels get reused and reassigned, so a long join silently changes subject.
Permanent identifiers with date-ranged validity, never a reusable label
Universe drift
Studies the population as it stands today rather than as it stood then.
Membership lists as at the date, point in time
Reporting lag
Treats the period a figure describes as the moment it became available.
Publication date plus a buffer
Definitional break
The series changes what it measures without changing its name.
Break calendar, flagged automatically when a test spans one
The break calendar
Any test spanning one of these dates is testing two different series and calling them one. The engine raises a flag rather than allowing it silently.
Date
What changed
1948, 1968, 1979, 1994, 2013, 2022
Successive revisions of the International Classification of Diseases, each redefining cause of death
1983
Owners' equivalent rent enters the consumer price index
A national currency is redenominated, and a long-standing correlation between two asset classes reverses
1999
Geometric mean introduced in consumer price index construction
2003
Industry classification moves to NAICS
2005
A second redenomination, six zeroes removed, rescaling every historical value
2013
Research and development capitalised in national accounts
January 2015
Swiss franc peg abandoned
August 2015
Chinese yuan regime shift
2016 and 2018
Sector classification revisions
2019
Lease accounting under ASC 842
1990s onward
Weather station relocation and automation, breaking long temperature records
2020
Case definitions for a novel pathogen revised repeatedly inside a single year
2022
The long-standing correlation between two major asset classes reverses
The same calendar, as a shape
Seventeen breaks in seventy-eight years, and they do not thin out with time. Any test spanning one of these marks is testing two different series and calling them one.
This is the argument for the break calendar in one picture. Not that breaks happen, which everyone concedes, but that they happen constantly, so a long sample is more exposed than a short one rather than less.
Time
Everything is stored in coordinated universal time to microsecond precision and converted only at the point of display. Where two venues disagree on a closing price, one is designated authoritative and the choice is recorded rather than assumed.
Daylight saving is handled by timezone-aware conversion rather than a fixed offset. The transitions in the United States and the United Kingdom do not coincide, which produces two or three weeks in March and about a week in October where a fixed offset is simply wrong.
Known open defect
One contamination issue is currently unresolved in a derived label set, and it is a hard block on the validation work rather than a caveat. Nothing dependent on it proceeds until it clears. It is listed under Data integrity in the theories index as open.
Raw deliveries are immutable. Every derived layer can be rebuilt from them. If that is not true, no result is reproducible and the rest of this is decoration.