How training data shapes alignment

Revision note, October 16, 2026. This essay was substantially revised to add primary references, narrow unsupported claims and distinguish proposals from measured results. The original address and publication date are retained.

Training data shapes the model that later training and deployment controls act on. Understanding its role requires examining those stages together and measuring the limits of each intervention.

Start with the whole training process

Pre-training exposes a model to patterns selected through collection, filtering, weighting and sampling. Those choices deserve scrutiny because they determine what material the model encounters and how often. A useful data review describes the corpus and the intended intervention, then tests its consequences against a stated baseline.

It is too strong to say that post-training only changes presentation or cannot improve capabilities. DeepSeek-R1 reports substantial improvements in measured reasoning performance after reinforcement learning. Those results do not settle whether every improvement creates a new capability or makes an existing one easier to elicit. They do show why measured performance and a theory about internal capability must be kept separate. DeepSeek-R1.

For an applied evaluation, the question is concrete: what changed after the intervention, on which tasks and under which access conditions? A gain in benchmark accuracy, a reduction in harmful completions and robustness against an adversary are different outcomes. None should silently stand in for the others.

Be precise about removal

A model declining to provide an answer does not establish that the relevant knowledge is absent. Conversely, the possibility of recovering unwanted behavior in some settings does not prove that every method of reducing that behavior is ineffective.

The WMDP work evaluates hazardous knowledge and reports reductions using an unlearning method, while studying preservation of general capabilities. Its findings support a measured reduction under its evaluation conditions. They do not establish perfect deletion or robustness against every future attack. The WMDP benchmark.

A meaningful claim about removal therefore needs a threat model. State who has access, what they can change, which attacks were tried and which outcomes were measured. Report successful recoveries as well as successful restrictions. Do not infer universal transfer from a jailbreak that worked on one model or task.

Document the choices that can be inspected

Data documentation can record provenance, permitted uses, filtering criteria, deduplication, mixture weights and known gaps. Access restrictions may prevent publishing raw records, but a review should still make clear what an auditor could inspect and what remains unknown.

This is already an area of institutional work. In July 2025, the European Commission published a template for public summaries of general-purpose AI training content. A public summary is different from confidential access to a corpus or a proposed approval process before training. Those differences should be stated when arguing for additional oversight. Commission announcement.

A testable research question

A focused study could compare two specified data mixtures while holding the model, compute budget and later training as constant as practical. It would evaluate both useful performance and unwanted behavior, including attempts to recover that behavior after post-training. The necessary controls and limits would depend on the question being asked.

That is a proposed study, not a result reported here. Its value would come from making the contribution of a data decision measurable. Data governance, post-training and deployment safeguards each address part of the problem; their relative effectiveness needs evidence under comparable conditions.