Back-test: would the framework have flagged Regime A in May 2002?

This file is generated. Edit dataroom/backtest/2002/narrative.md, then run make backtest-2002 in dataroom/. Every number in the text and tables is computed by dataroom/backtest/backtest_2002.py from the data room, and every table is also saved as CSV (Appendix C). The framework is 02-framework/rates-regime-framework-2026.md; this tests item 4 of section 8.

1.Answer

No. Run exactly as written, with the 2026 bands and the "3 of 5 headline metrics for 3 months" rule, the framework would not have flagged Regime A in May 2002. The tally that month was A1 B2 C2, so no call. Nor would it have flagged A in any of the 62 months from Nov 2001 to Dec 2006. In that period the most A votes in any month was 2, although the 10-year fell from 5.08% to 3.37% within a year. The live data room, which runs the same rules from 2003, first calls A in Oct 2015.

Three qualifications matter:

• The framework did call A, but earlier and only briefly: A Feb 2001 (1 mo); A Sep 2001 to Oct 2001 (2 mo). These calls came during the Fed's 2001 cutting cycle. They depend on how the pre-2003 5y5y is reconstructed: with the raw Fed Board series instead of the level-adjusted one, there are no A calls at all, and the only calls are B (B Oct 2002 (1 mo); B Sep 2005 to Jan 2006 (5 mo); B Oct 2006 to Dec 2006 (3 mo)). All calls in the main version: A Feb 2001 (1 mo); A Sep 2001 to Oct 2001 (2 mo); B Oct 2002 (1 mo); B Sep 2005 to Jan 2006 (5 mo); B Oct 2006 to Dec 2006 (3 mo).
• What blocked an A call in 2002 was the 2026 calibration of the bands. The term premium (ACM) was 2.40%, which the 2026 bands read as Regime C. It stayed in the C band for 43 of the 43 months from Jun 2001 to Dec 2004. The 10-year TIPS yield was 3.16%, above every band. The framework's own Regime A levels arrived years later: TIPS below 1.25% first in Feb 2008, term premium below zero first in Jan 2015, and the 10-year below 4% for a full year first from Nov 2008.
• The parts of the framework that did point to A were outside the headline vote. In May 2002 the structural drivers (Layer 2) scored −5 out of a possible −7. Core PCE inflation, as published at the time, was 1.35%. Japan's 10-year was 1.37%. A direction-aware version of the vote, which asks which way each input is moving rather than where it sits, came close in 2002–03 (3 A votes in Nov 2002, Jan 2003, May 2003, Jun 2003) but never reached three months in a row.

2.What was tested and how

The framework's question: "Rebuild the dashboard with data from May 2002 … Would the framework have flagged Regime A?" I rebuilt every month-end from Jan 1999 to Dec 2006 and ran four versions of the vote:

Alongside the votes: every dashboard metric that has bands, as it read in May 2002, the Layer 2 driver scores, the journal's 12-month range method applied in May 2002, and a comparison of real-time and revised data.

Data that did not exist in 2002, and what replaced it.

• 5y5y and 10-year TIPS (FRED, from 2003): taken from the Federal Reserve Board's TIPS yield curve (Gürkaynak-Sack-Wright), which starts in Jan 1999.
• TIPS check: on the 2003–2020 overlap the Fed Board TIPS yield sits +0.07pt from FRED's, with correlation 0.997 (216 months), so it is used as is.
• 5y5y adjustment: the Fed Board 5y5y runs 0.37pt above FRED's in 2003-01 to 2005-12, so it is lowered by that gap (Appendix A).
• Fed target rate: taken from FRED DFEDTAR.
• Credit spread: FRED BAA10Y replaces HY OAS, which FRED keeps only from 2023.

None of the five headline inputs is lost.

Real-time data. Market prices are not revised, so four of the five headline inputs are exactly what was on screens in 2002. Core PCE, unemployment and GDP were taken as first published, from ALFRED (the St. Louis Fed's archive of past data releases). Federal interest payments have no ALFRED history before 2008, so interest/receipts uses today's figures, lagged two months to when the quarter was first published.

3.The dashboard in May 2002

Headline vote, 31 May 2002:

Tally A1 B2 C2: no regime has 3 votes, so no call (and a call also needs three months running). Two of the five headline metrics voted C, the fiscal or inflation regime, the opposite of what followed.

Every dashboard metric that has bands, as it could have been read on 31 May 2002. Each value is the latest one published by that date: quarterly data two months after the quarter ends, annual data four months after. Core PCE is shown as first published, with today's revised value beside it.

Count across these rows: A 3, B 4, C 1. The macro rows (core inflation, the budget balance, unit labour costs) leaned A. The market-price rows did not. Interest/receipts shows B here because this table applies its level band (15–20%). In the headline vote it counts as A because it was falling.

4.Working, month by month (Jun 2001 to Dec 2003)

The headline rules, applied each month-end:

• 5y5y and term premium: fixed bands.
• TIPS: votes C if it fell more than 10bp over 3 months while the 10-year rose more than 10bp; otherwise its band.
• Policy: compares the 3-month change in fed funds with the 30-year: A if both fell more than 10bp, C if fed funds fell and the 30-year rose, B if fed funds moved less than 10bp.
• Interest/receipts: A if lower than 12 months earlier, C if above 20% and rising, B if 15–20%.
• Call: 3 or more votes for one regime in three consecutive months.

"-" means the reading sits between bands, or the rule gives no vote.

Market inputs (rows 2, 3, 4):

Policy, fiscal and the count (rows 6, 9):

5.Why the votes did not say A

• Term premium (row 4). The ACM premium was 2.40% in May 2002 and peaked at 2.80% in Jul 2003, the very year yields bottomed. The 2026 C band (above 1.25%) is calibrated to the post-2010 world of QE and compressed premia. In 2002 a premium of 2% or more was normal, so this metric voted C every month until it fell below 1.25% in Jan 2005. Caveat: the ACM model is estimated on the full sample to today, so its 2002 values carry hindsight. Kim-Wright (on the chart above) was lower, at 1.57% in May 2002, but that is still in the C band.
• TIPS (row 3). At 3.16% the real yield was above every band. Its C rule fired in Mar and May 2002 because TIPS fell while nominal yields rose: breakevens were widening on recovery hopes, which is not the fiscal-dominance signal the rule was written for. TIPS then fell into the B band (first Aug 2002) and reached 1.48% in Mar 2004, but never below 1.25% before Feb 2008.
• 5y5y (row 2). After adjustment it was in the B band (2.0–2.6%) for nearly all of 2002–03; it went below 2% in only a few months of 2003. In 2001 it was below 2% (from 1.70%), which is where the 2001 A calls came from.
• Policy (row 6). The Fed's last cuts of the cycle (Nov and Dec 2001) were followed by a rising 30-year, so the "long end rises while the Fed cuts" C signature fired in Jan and Feb 2002. That was a normal bear steepening at the end of an easing cycle. It fired again after the Nov 2002 cut and, in Jul–Sep 2003, after the Jun 2003 cut, as the summer bond sell-off lifted the 30-year. In all, policy voted C in Jun 2001, Jan 2002, Feb 2002, Jun 2002, Dec 2002, Jul 2003, Aug 2003, Sep 2003, Dec 2003. The Jun 2002 C vote comes from a −10bp drift in the effective fed funds rate, not a cut. For most of 2002 the Fed was on hold, a B vote.
• Interest/receipts (row 9) was the only metric pointing to A most months, because interest costs were falling relative to revenue. In May 2002 the fall was just -0.02pp, so even this vote was marginal.

Effect of publication lag. Using the dashboard's quarter-end dating for interest/receipts instead of the "as usable" lag changes the call in 9 months (Sep 2001, Oct 2001, Oct 2002, Feb 2003, Feb 2005, Mar 2005, Aug 2005, Dec 2005, Jan 2006). It never produces an A call in 2002–03: A Feb 2001 (1 mo); B Feb 2003 (1 mo); B Feb 2005 to Mar 2005 (2 mo); B Aug 2005 to Nov 2005 (4 mo); B Oct 2006 to Dec 2006 (3 mo).

Tie-out with the live data room. For 2003–2006 the quarter-end version matches the live register (register/regime_votes.csv) in 47 of 48 months. The exception is Feb 2003: the back-test has a TIPS reading for Dec 2002 (from the Fed Board), while the live register starts in Jan 2003.

6.Direction-aware variant

The same five inputs, but the three level-banded ones (5y5y, TIPS, term premium) are scored on direction:

• A if the 12-month fall is at least 0.25pt (5y5y), 0.50pt (TIPS), 0.50pt (term premium) and the reading is below its own median for the trailing 10 years;
• C if the 12-month rise is at least as large and the reading is above that median;
• B otherwise.

Policy and interest votes are already directional and are unchanged. Only data available at each month-end is used.

Result: A Feb 2001 (1 mo); A Sep 2001 to Oct 2001 (2 mo); B Sep 2003 (1 mo); B Oct 2005 to Jan 2006 (4 mo); B Nov 2006 to Dec 2006 (2 mo). May 2002: A1 B3 C1 (infl_5y5y=B, tips10y=B, acm_tp10=C, policy=B, interest=A). The A count reached 3 in Nov 2002, Jan 2003, May 2003, Jun 2003, but never for three consecutive months. TIPS and the term premium were falling, yet the policy and interest votes flickered, and the 5y5y rose in 2002 as the recession scare faded.

7.Structural drivers (Layer 2) in May 2002

Scored with what could be known in May 2002, using the framework's −1 / 0 / +1 scale (−1 pushes yields down):

Total −5. This layer, not the headline vote, carried the A signal in 2002. The framework says this score "is not a forecast; it is a map of pressure". Here the map pointed down while the market-price metrics were still calibrated, in effect, to say C.

8.What happened next, and the range test

The 10-year fell from 5.08% to 3.37% by May 2003, then traded around 4–5% to 2007. It did not settle below 4% until Nov 2008 and reached 0.55% in Jul 2020. On 6 May 2003 the FOMC said "the probability of an unwelcome substantial fall in inflation, though minor, exceeds that of a pickup in inflation from its already low level". Core PCE, as published, fell to 1.19% (Nov 2003).

Range test. The journal method (every past 12-month change that started within ±1pt of today's level, applied to today's level, using data to May 2002 only) gives:

The 10-year's actual May 2003 level of 3.37% was below every past 12-month change from similar starting levels. The method's history (mostly 1962–2001) contains no fall that large from about 5%. The 30-year, at 4.59%, was also outside its 10th–90th range (4.9–7.1%).

9.Real-time against revised data

May 2002 readings, as first published and as revised today:

Revisions do not change the answer; if anything they hide the 2002 signal. Core PCE as published was 1.35% against 1.57% after revision, and over 2001–03 the published figure was on average 0.25pt below today's. In real time, disinflation looked stronger than it does in today's data. The headline vote is unaffected, because its inputs are market prices plus interest/receipts, which has no 2002 vintage (see section 2).

10.Lessons for the 2026 framework

1. The headline bands are era-specific. Run on 2002 data, they read a disinflationary decade as B or C, because a 2% term premium and a 3% real yield were normal then. They would misread 2026 the same way if the "normal" level has shifted again. Consider bands relative to each input's own trailing history (as in section 6), or to r*, alongside the fixed ones.
2. The C signatures misfire at turning points. "TIPS falling while nominal yields rise" and "long end rising while the Fed cuts" both fired in early 2002 on recovery reflation, not fiscal stress. Require them alongside a fiscal or inflation reading before they count.
3. Layer 2 was right when Layer 3 was not. In May 2002 the driver score was −5 while the headline vote leaned C. The framework already says to wait for confirmation from Layer 2; the back-test says Layer 2 can lead, so keep the driver score next to the vote rather than beneath it.
4. Three months running is strict for a turning regime. The direction-aware vote reached 3 A votes in four months of 2002–03 but never three months in a row. "Two months running" would have called A in May–Jun 2003. Treat this as a hypothesis, not a fitted rule: one episode is not enough evidence, and a looser rule also produces more false calls.
5. History-based ranges anchor on the old regime. The journal's range method put May 2003 at 4.7–6.5% for the 10-year; the outcome was 3.37%. This is the 2002 error in numerical form, and it argues for the regime-tilted ranges the journal already uses.

11.Caveats

• 5y5y reconstruction. The adjustment (0.37pt) is measured on 2003–05, after the test date, and the gap has a standard deviation of 0.15pt. The 2001 A calls disappear without it, so treat them as fragile.
• ACM hindsight. The model is re-estimated on today's full sample, so its 2002 values were not available in that form at the time.
• Interest/receipts uses today's revised BEA data; no 2002 vintage exists in ALFRED.
• 30-year yield. Treasury held no 30-year auctions from Aug 2001 to Feb 2006, so FRED's DGS30 for 2002–06 does not come from a newly issued 30-year bond, and the data room treats it as extrapolated. It tracks the Fed Board 30-year curve as closely as in other years. The policy vote uses it.
• Early TIPS market. In 1999–2002 the TIPS market was small and illiquid, so its real yields and breakevens contain a liquidity premium.
• One episode. This is one historical episode, and the lessons above are hypotheses to test, not recalibrations. The framework file has not been changed.

Appendix A. 5y5y splice

The 5y5y uses FRED T5YIFR from Jan 2003. Before that it uses the Fed Board 5-year, 5-year-forward breakeven (BKEVEN5F05) minus the mean gap between the two series over 2003-01 to 2005-12: 0.37pt (standard deviation 0.15pt, 36 months, correlation 0.83). The two series are built differently: the Fed Board one is a fitted zero-coupon forward, FRED's comes from Treasury par yields. Unadjusted, the May 2002 reading would be 2.71% instead of 2.34%: a 5y5y "-" vote (between the B and C bands) instead of B.

Appendix B. Every month, 1999–2006

Votes are shown as A-B-C counts; a bold letter marks a regime call. "Detail" is the per-metric vote in the main version.

Appendix C. Data and sources

All files are in dataroom/backtest/2002/data/. Each series' raw download, checks and source are in the data room register (dataroom/register/audit_register.csv).

Sources:

• FRED (St. Louis Fed): DGS10, DGS30, DFF, DFEDTAR, T5YIFR, DFII10, BAA10Y, PCEPILFE, UNRATE, GDP, A091RC1Q027SBEA, FGRECPT.
• ALFRED: past releases of PCEPILFE, UNRATE, GDP.
• Federal Reserve Board: TIPS yield curve and inflation compensation (feds200805.csv) and the nominal curve (feds200628.csv).
• New York Fed: ACM term premia.
• US Treasury: TIC.
• Ministry of Finance Japan: JGB yields.