Can LLMs Take the Pulse of the Economy? A Real-Time Evaluation of LLM Nowcasts on Macroeconomic Indicators
Nowcasting means estimating an indicator before it is officially released. Central banks staff whole teams to do it, and an LLM agent can be asked far more often than any of them.
The issue is LLM memorization leading to data contamination. GDP and CPI are among the most reported numbers on earth, so their past values are almost certainly memorized during pretraining, and testing on old releases measures recall rather than nowcasting.
LiveMacroEval scores only nowcasts made before the release, which keeps it clean by construction.
Institutional nowcasts, where they exist.
Bloomberg ECOS is the professional economist consensus, covering all sixteen indicators, and it is famously hard to beat.
ARIMA is a univariate time-series baseline, refit at the start of every window.
Plain error on the released level ignores that indicators differ in economic weight. We use two market-grounded scores instead. Full definitions and estimation are in the paper.
Macro surprises move equity prices. We weight each indicator by its measured effect on S&P 500 futures at the moment of release, so sixteen accuracies collapse into one comparable number. +1 is perfect, 0 ties the consensus, −1 is arbitrarily worse. Each release is scored on the nowcast in force when it comes out, and only if that nowcast is less than a day old.
At fixed times each day before a release, one hour after the agents' scheduled runs, we bet $1 on the Polymarket bucket holding the agent's latest nowcast, provided that nowcast is less than a day old. Prices are the crowd's own odds, so liquidity is priced in: beating a confident crowd pays a lot, agreeing with it pays little. The score is unit-free, so it aggregates across very different indicators.
| # | Model | LiveMacro | 90% CI | vs. consensus |
|---|---|---|---|---|
| Loading… | ||||
—
One base model, three ways of running it: a plain prompt, the same prompt with a financial-data plug-in, and a multi-agent team.
| # | Configuration | LiveMacro | vs. consensus |
|---|---|---|---|
| Loading… | |||
—
Nine target months live. The overall score hides sharp differences across the economy: the same arm can beat the consensus on one block and trail it on another.
—
GPT-5's March 2026 CPI nowcast sits near 2.7% through early April, then jumps to about 3% on April 8 and tracks the realized print from there. Two scheduled events land inside that window: a large gasoline-inventory drawdown, and FOMC minutes flagging upside inflation risk. The size of the revision matches a back-of-the-envelope gasoline contribution under BLS expenditure weights.
Curves are a Gaussian smooth of the hourly nowcasts.
@inproceedings{zhao2026livemacroeval,
title = {Can LLMs Take the Pulse of the Economy? A Real-Time Evaluation
of LLM Nowcasts on Macroeconomic Indicators},
author = {Zhao, Xinyue and Zhang, Ruiyi and Ye, Liqin and
Cao, Rui and Xie, Pengtao and Chava, Sudheer},
booktitle = {Proceedings of EMNLP},
year = {2026},
eprint = {2608.30110},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
url = {https://arxiv.org/abs/2608.30110}
}