The Currency Times

Data

Sources, and what is wrong with each of them. More results are invalidated here than by any failure of statistics, and almost none of it is a statistics problem.

A series that looks clean is usually a series whose faults have not been looked for. Six recur, and all six are silent.

The six silent faults

Fault
What it does
Control
Look-ahead
Uses a figure before it was released, or a revision that did not exist yet.
Release timestamps, reads refused before them
Survivorship
Measures only the cases that lasted, so the failures vanish from the record.
Exits recorded with an outcome rather than dropped from the sample
Identifier drift
Labels get reused and reassigned, so a long join silently changes subject.
Permanent identifiers with date-ranged validity, never a reusable label
Universe drift
Studies the population as it stands today rather than as it stood then.
Membership lists as at the date, point in time
Reporting lag
Treats the period a figure describes as the moment it became available.
Publication date plus a buffer
Definitional break
The series changes what it measures without changing its name.
Break calendar, flagged automatically when a test spans one

The break calendar

Any test spanning one of these dates is testing two different series and calling them one. The engine raises a flag rather than allowing it silently.

Date
What changed
1948, 1968, 1979, 1994, 2013, 2022
Successive revisions of the International Classification of Diseases, each redefining cause of death
1983
Owners' equivalent rent enters the consumer price index
1996
Chain-weighted gross domestic product replaces fixed weight
1998
A national currency is redenominated, and a long-standing correlation between two asset classes reverses
1999
Geometric mean introduced in consumer price index construction
2003
Industry classification moves to NAICS
2005
A second redenomination, six zeroes removed, rescaling every historical value
2013
Research and development capitalised in national accounts
January 2015
Swiss franc peg abandoned
August 2015
Chinese yuan regime shift
2016 and 2018
Sector classification revisions
2019
Lease accounting under ASC 842
1990s onward
Weather station relocation and automation, breaking long temperature records
2020
Case definitions for a novel pathogen revised repeatedly inside a single year
2022
The long-standing correlation between two major asset classes reverses

The same calendar, as a shape

Seventeen breaks in seventy-eight years, and they do not thin out with time. Any test spanning one of these marks is testing two different series and calling them one.

ICD1948OER into CPI1983Chain-weighted GDP1996Redenomination · correlation turns1998Geometric mean CPI1999NAICS2003Redenomination2005R&D capitalised2013Two currency regimes break2015Classification revisions2018ASC 8422019Case definitions revised2020Correlation reverses202219501960197019801990200020102020

This is the argument for the break calendar in one picture. Not that breaks happen, which everyone concedes, but that they happen constantly, so a long sample is more exposed than a short one rather than less.

Time

Everything is stored in coordinated universal time to microsecond precision and converted only at the point of display. Where two venues disagree on a closing price, one is designated authoritative and the choice is recorded rather than assumed.

Daylight saving is handled by timezone-aware conversion rather than a fixed offset. The transitions in the United States and the United Kingdom do not coincide, which produces two or three weeks in March and about a week in October where a fixed offset is simply wrong.

Known open defect

One contamination issue is currently unresolved in a derived label set, and it is a hard block on the validation work rather than a caveat. Nothing dependent on it proceeds until it clears. It is listed under Data integrity in the theories index as open.

Raw deliveries are immutable. Every derived layer can be rebuilt from them. If that is not true, no result is reproducible and the rest of this is decoration.

NextLedgerthe same standard, turned on ourselves