From 729aa5062ae2c1ca54dc45d335bd9178b3d3278b Mon Sep 17 00:00:00 2001 From: Vahid Ahmadi Date: Fri, 24 Jul 2026 15:42:20 +0100 Subject: [PATCH 1/2] Bump hosted SVAR estimation window 2023Q2 -> 2025Q1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The hosted server reported two different samples for the same model: model_summary read the committed replication artifacts (1992Q1-2025Q1, refreshed in #85) while forecast_uk and get_model_status reported 1992Q1-2023Q2. That was not just a label — _SVAR_EST_END truncates the estimation frame in _estimate(), so the live forecast discarded seven quarters of data the package already ships. The constant is deliberately pinned so hosted results only move on a deliberate refresh; this is that refresh. Held one quarter short of the 2026Q1 data edge, matching the repo, so the forecast origin stays outside the estimation sample. Measured effect (seed 0, 2000 draws, horizons 4): accepted 165 -> 135, ESS 63.6 -> 65.3 GDP medians +0.05 to +0.11pp; CPI within +/-0.02pp Acceptance falls because the longer sample tightens the posterior, so the draw-count calibration in the comment and the forecast_uk docstring is re-measured on the new sample. 2000 remains the smallest default clearing the 100-accepted threshold (1000 gives 69); ESS >= 100 still needs ~3500. Also updates the five site locations quoting the old endpoint, so the published sample matches what the server reports. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/index.html | 2 +- index.html | 2 +- .../src/policyengine_macro/capabilities.py | 4 +-- integration/src/policyengine_macro/core.py | 29 ++++++++++++------- .../src/policyengine_macro/mcp_server.py | 12 ++++---- models/index.html | 2 +- svar/index.html | 6 ++-- 7 files changed, 32 insertions(+), 25 deletions(-) diff --git a/docs/index.html b/docs/index.html index 69fd880..7df4a13 100644 --- a/docs/index.html +++ b/docs/index.html @@ -106,7 +106,7 @@

Five model classes, side by side.

Question classSelected near-term UK fiscal and macro scenariosWhat drove the UK economy at the latest model-data vintage; short-run forecasts and revision narrativesUS macro dynamics: monetary and fiscal shocks, quarter by quarterLong-run incentives: labour supply, saving, the capital stock over decadesWhat a specific family pays and receives; distributional effects MethodStructural macroeconometric emulator — Gauss–Seidel over 372 equations per quarterBayesian structural VAR — posterior sampling with zero + sign restrictions naming the shocksLarge-scale macroeconometric model — 284 equations solved under VAR expectationsDynamic general equilibrium (OLG) — steady-state root-find and transition-path iterationStatic tax-benefit microsimulation — direct rule evaluation, no behavioural response CountryUKUKUSUK (OG-UK)UK & US - Data vintageOBR Economic and Fiscal Outlook, March 2026ONS, BoE and FRED series — coefficients estimated through 2023Q2; conditioned through the 2026Q1 data edgeThe Fed's model.xml and LONGBASE database, April 2026ONS, OBR and BoE national accounts as calibration targets; OG-UK 0.3.2 (from GitHub)UK & US statute; enhanced FRS 2023–24 microdata for UK population runs + Data vintageOBR Economic and Fiscal Outlook, March 2026ONS, BoE and FRED series — coefficients estimated through 2025Q1; conditioned through the 2026Q1 data edgeThe Fed's model.xml and LONGBASE database, April 2026ONS, OBR and BoE national accounts as calibration targets; OG-UK 0.3.2 (from GitHub)UK & US statute; enhanced FRS 2023–24 microdata for UK population runs Evidence class (numbers →)Validated for selected scenarios; published anchor is not out-of-sample validationValidated replication for selected outputsValidated software replication; substantive comparisons are approximateResearch prototype; calibrated counterfactual with no ground truthDeterministic rule calculation; population estimates add survey uncertainty Reform scoringYes — shocks to exogenous model variables; static-costing bridge from PolicyEngine reformsNo — the baseline/conditioning member: it reads the economy reforms are scored againstNo — shock experiments only (funds-rate or fiscal shocks under VAR expectations); score_reform refuses model="frbus", as no PolicyEngine reform bridge existsYes — a PolicyEngine policy translated into estimated tax functionsYes — a PolicyEngine parameter + a new value, applied directly to households Cannot establishArbitrary reform incidence or borrowing through the current adapterThe causal effect of a specific statutory reformPolicyEngine reform effects, anticipated-policy paths, or MCE scenariosA short-run forecast or independently validated reform effectGDP, inflation, interest rates, or macro feedback diff --git a/index.html b/index.html index 2f9369b..6f6c1d4 100644 --- a/index.html +++ b/index.html @@ -136,7 +136,7 @@

OBR macroeconometric model

Bank of England structural VAR model

Diagnose the latest model-data vintage and forecast GDP and inflation - with uncertainty ranges. Coefficients are estimated through 2023Q2; + with uncertainty ranges. Coefficients are estimated through 2025Q1; the model does not score reforms.

read the model → diff --git a/integration/src/policyengine_macro/capabilities.py b/integration/src/policyengine_macro/capabilities.py index 0d2abde..b5e9f33 100644 --- a/integration/src/policyengine_macro/capabilities.py +++ b/integration/src/policyengine_macro/capabilities.py @@ -76,9 +76,9 @@ "runtime": "minutes per estimation and identification run", "uncertainty": "posterior 68% and 90% intervals", "status": "validated replication for selected outputs", - "estimation_sample": "1992Q1-2023Q2", + "estimation_sample": "1992Q1-2025Q1", "data_edge": "2026Q1", - "data_vintage": "2026Q1 conditioning data; estimation ends 2023Q2", + "data_vintage": "2026Q1 conditioning data; estimation ends 2025Q1", }, "frb-us": { "display_name": "Federal Reserve FRB-US implementation", diff --git a/integration/src/policyengine_macro/core.py b/integration/src/policyengine_macro/core.py index 6b42b0d..d4888dd 100644 --- a/integration/src/policyengine_macro/core.py +++ b/integration/src/policyengine_macro/core.py @@ -725,17 +725,24 @@ def _covid_dummies(index) -> np.ndarray: # Estimation window for the hosted SVAR (start of Great-Moderation sample; # end held fixed so hosted results only move on a deliberate refresh). The # provenance strings are derived from the actual df_est index, never retyped. +# +# 2026-07-24 refresh: bumped 2023Q2 -> 2025Q1 (126 -> 133 observations) to match +# the sample boe-var-model's production replication now runs on. Held one +# quarter short of the 2026Q1 data edge, matching the repo, so the forecast +# origin sits outside the estimation sample. _SVAR_EST_START = "1992Q1" -_SVAR_EST_END = "2023Q2" - -# Default posterior draws for hosted calls. Measured on the 2026Q1 vintage -# (Apple Silicon, single process): draws=500 -> 42 accepted, ESS 15.5, ~23s; -# 1000 -> 76 accepted, ESS 26.5, ~44s; 2000 -> 165 accepted, ESS 63.6, -# ~117s. Acceptance is ~8% and ESS ~3% of draws, so 2000 is the smallest -# default that clears the 100-accepted-draws reliability threshold at -# roughly two minutes of first-call runtime (results are cached in-process). -# Full ESS >= 100 needs ~3500 draws (~3.5 min); a `warnings` entry flags the -# residual ESS shortfall honestly instead of hiding it. +_SVAR_EST_END = "2025Q1" + +# Default posterior draws for hosted calls. Re-measured on the 1992Q1-2025Q1 +# estimation sample (seed 0), since the longer sample tightens the posterior +# and lowers the sign-restriction acceptance rate: draws=500 -> 35 accepted, +# ESS 13.6; 1000 -> 69, ESS 31.5; 2000 -> 135, ESS 65.3; 3500 -> 233, +# ESS 101.4. Acceptance is ~6.8% and ESS ~3.3% of draws (was ~8% and ~3% on +# the old 2023Q2 sample), so 2000 remains the smallest default that clears +# the 100-accepted-draws reliability threshold -- 1000 would not (69). +# Full ESS >= 100 still needs ~3500 draws; a `warnings` entry flags the +# residual ESS shortfall honestly instead of hiding it. First-call runtime is +# a couple of minutes end to end and results are cached in-process. _SVAR_DEFAULT_DRAWS = 2000 @@ -2705,7 +2712,7 @@ def svar_summary() -> dict: "provenance": _provenance( model_id="boe-svar", distribution="boe-var-model", - data_vintage="estimation through 2023Q2; conditioning data through 2026Q1", + data_vintage="estimation through 2025Q1; conditioning data through 2026Q1", baseline="committed replication and forecast artefacts", ), } diff --git a/integration/src/policyengine_macro/mcp_server.py b/integration/src/policyengine_macro/mcp_server.py index b5001ac..553e005 100644 --- a/integration/src/policyengine_macro/mcp_server.py +++ b/integration/src/policyengine_macro/mcp_server.py @@ -290,7 +290,7 @@ def frbus_summary() -> dict: def forecast_uk(horizons: int = 12, draws: int = 2000) -> dict: """Forecast UK YoY GDP growth and CPI inflation with the UK SVAR model. - Estimates a Bayesian VAR (1992Q1-2023Q2 sample, sign-identified structural + Estimates a Bayesian VAR (1992Q1-2025Q1 sample, sign-identified structural shocks) and simulates the predictive distribution from the latest data quarter. Returns, per future quarter, the median and 68%/90% bands for YoY GDP growth and YoY CPI inflation, both in percent. The response includes a @@ -299,11 +299,11 @@ def forecast_uk(horizons: int = 12, draws: int = 2000) -> dict: Args: horizons: Forecast horizon in quarters (default 12 = 3 years). - draws: Posterior draws (default 2000: ~165 accepted draws, ESS ~64, - about two minutes of runtime on first call; ~3500 draws reaches - ESS >= 100 in ~3.5 minutes; 500 responds in ~25s but yields ESS - ~15 and a warning). Results are cached in-process, so repeated - calls with the same (horizons, draws) are instant. + draws: Posterior draws (default 2000: ~135 accepted draws, ESS ~65, + a couple of minutes on first call; ~3500 draws reaches ESS >= 100; + 500 is faster but yields ~35 accepted, ESS ~14, and a warning). + Results are cached in-process, so repeated calls with the same + (horizons, draws) are instant. """ return core.svar_forecast(horizons=horizons, draws=draws) diff --git a/models/index.html b/models/index.html index c08262a..ff33b42 100644 --- a/models/index.html +++ b/models/index.html @@ -52,7 +52,7 @@

Choose the model that fits the quest
all models

Inspect the model before you use it.

Each guide covers inputs, outputs, method, validation, limitations, and how to run the model.

obr-macroUKhosted

OBR macroeconometric model

Selected near-term scenarios. Trace GDP, consumption, and investment against the official forecast. Validated for selected shocks; borrowing is not yet returned.

open model → -
boe-svarUKhosted

Bank of England structural VAR

UK conditions and forecasts. Identify shocks and forecast GDP and inflation with uncertainty ranges. Estimated through 2023Q2; conditioned through 2026Q1.

open model → +
boe-svarUKhosted

Bank of England structural VAR

UK conditions and forecasts. Identify shocks and forecast GDP and inflation with uncertainty ranges. Estimated through 2025Q1; conditioned through 2026Q1.

open model →
frb-usUShosted

Federal Reserve macroeconomic model

Selected US shocks. Run raw monetary and fiscal levers under VAR expectations. No reform bridge or model-consistent expectations.

open model →
psl-ogUKlocalexperimental

PSL overlapping generations

Long-run incentives and capital. Run local steady-state reform analysis; transition paths are available from the package. Calibrated, not outcome-validated.

open model →
pe-microsimUKUShosted

PolicyEngine microsimulation

Households and distribution. Calculate taxes, benefits, net income, and population impacts. Population results depend on survey data; no macro feedback.

open model → diff --git a/svar/index.html b/svar/index.html index b8bffa4..42b2807 100644 --- a/svar/index.html +++ b/svar/index.html @@ -137,7 +137,7 @@

Eight variables, six named shocks.

A VAR in eight quarterly variables — three global, five UK — estimated - over the inflation-targeting era — 1992Q1–2023Q2 in + over the inflation-targeting era — 1992Q1–2025Q1 in both the paper replication and the hosted adapter. The hosted forecast conditions on observations through 2026Q1, but does not re-estimate its coefficients on those later observations. It is estimated in @@ -178,7 +178,7 @@

Eight variables, six named shocks.

so medians and bands are correct but noisier than the paper's. The hosted forecast conditions on the current vintage — data through 2026Q1 — with 6,000 draws (430 accepted, ESS 197.9), while retaining - the 2023Q2 coefficient-estimation endpoint. + the 2025Q1 coefficient-estimation endpoint.

@@ -368,7 +368,7 @@

Where this departs from the Bank's model.

Frozen data edge - Coefficient estimation to 2023Q2; conditioning data to 2026Q1 (refreshed July 2026). Results shift with data revisions; the replication claims are scored on the paper's original sample. + Coefficient estimation to 2025Q1; conditioning data to 2026Q1 (refreshed July 2026). Results shift with data revisions; the replication claims are scored on the paper's original sample. Proxied world aggregates From 6da9df489b1077fe3602cb913e2d089e898cfc3f Mon Sep 17 00:00:00 2001 From: Vahid Ahmadi Date: Fri, 24 Jul 2026 15:44:20 +0100 Subject: [PATCH 2/2] Derive the site-contract estimation endpoint from _SVAR_EST_END The guard hardcoded "estimated through 2023Q2", so it failed on exactly the deliberate refresh it exists to permit, while a literal also cannot catch drift to any other quarter. It now reads the constant and rejects any other quarter presented as the estimation endpoint. Verified non-vacuous: editing the site to a different quarter fails it. Co-Authored-By: Claude Opus 4.8 (1M context) --- integration/tests/test_site_contract.py | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/integration/tests/test_site_contract.py b/integration/tests/test_site_contract.py index 5072d82..aebaf3a 100644 --- a/integration/tests/test_site_contract.py +++ b/integration/tests/test_site_contract.py @@ -14,10 +14,25 @@ def _read(path: str) -> str: def test_svar_site_uses_runtime_estimation_endpoint(): + """The published estimation endpoint must match the one the server actually + estimates on. + + Derived from ``_SVAR_EST_END`` rather than hardcoded: the point of this + guard is to catch the site drifting away from the runtime, and a literal + makes it fail on the refresh it is supposed to permit while still missing + drift in the other direction.""" + from policyengine_macro.core import _SVAR_EST_END + pages = _read("svar/index.html") + _read("docs/index.html") - assert "estimated through 2023Q2" in pages - assert "estimation sample to 2025Q1" not in pages - assert "Estimation to 2025Q1" not in pages + assert f"estimated through {_SVAR_EST_END}" in pages + + # No other quarter may be presented as the estimation endpoint. + stale = { + q + for q in re.findall(r"estimat\w*\s+(?:through|to)\s+(\d{4}Q[1-4])", pages) + if q != _SVAR_EST_END + } + assert not stale, f"site quotes stale estimation endpoint(s): {sorted(stale)}" def test_og_is_not_labelled_as_us_model():