Compute, data and the shape of the next generation

Revision note, October 16, 2026. This essay was substantially revised to add primary references, narrow unsupported claims and distinguish proposals from measured results. The original address and publication date are retained.

Training and inference are separate, interacting uses of compute. Deciding where to spend a budget requires a workload, an evaluation target and an explicit account of data and serving costs.

What a scaling result establishes

Chinchilla studied model size and training-token allocation under a given training-compute budget. Its results show why the relationship between those inputs matters when selecting a training run. They do not, by themselves, prove that all useful data has been exhausted or that a later model has reached a universal data wall. Training compute-optimal large language models.

A claim about data scarcity needs its own evidence. Define the inventory, permitted uses, language and domain coverage, deduplication rules and quality criteria. A large token count does not settle whether the data supports the intended capability, and a filtered inventory does not establish a global limit on future training.

Keep forecasts separate from measurements. Predicting a future shortage is different from identifying which constraint limited a particular completed training run.

Synthetic data depends on the process

Recursive training does not have one inevitable outcome across all data policies. Gerstgrasser and colleagues distinguish replacement of prior data from accumulation that retains the original real data. Their experiments found that retaining original real data alongside successive synthetic generations avoided collapse in the settings studied. That is neither a universal safety guarantee nor support for claiming that every synthetic-data pipeline must collapse. Is model collapse inevitable?.

An applied study should specify the generator, selection rule, retained real data, mixture, number of generations and evaluation distribution. Track failures that an aggregate score might miss, especially when a pipeline repeatedly selects outputs from its own earlier models.

Additional inference can help under stated conditions

Snell and colleagues study allocation of test-time computation and report improvements that depend on the problem and allocation strategy. Their results motivate measuring inference budgets carefully. They do not establish that extra inference is always a cheaper replacement for further training. Scaling LLM test-time compute optimally.

More generated candidates, verification steps or tool calls can also increase latency and resource use. Evaluate the final accepted outcome and the full cost of producing it. Counting only the successful candidate conceals the rejected work that made selection possible.

Use comparable denominators

Training FLOPs for one run, total service FLOPs over a period and inference FLOPs for one query answer different questions. A claim that one stage consumes more than another needs comparable disclosures and a defined time horizon.

For a planning estimate, separate fixed development costs from recurring query and review costs. State the expected number and distribution of queries before amortizing a training budget. Include unsuccessful attempts and human correction in the workload definition. Label assumptions so the estimate can be updated rather than mistaken for an observed industry ratio.

Compare alternatives on the same work

A useful decision process would compare a fixed model with several inference budgets, then compare those results with a separately trained or adapted model on the same held-out tasks. Hold the acceptance criteria constant and report quality, latency and cost together.

This essay reports no such experiment and makes no dated prediction about which approach will dominate. It identifies the measurements needed to decide when more data, more training or more inference is the better use of a particular budget.