OpenAI Opens Training Runs to Four Independent Evaluators: What Vendor Risk Reviews Should Ask
Enterprise buyers now have a new question to put to frontier labs: who outside the company gets to watch the models being trained? OpenAI has signed four outside organisations — Apollo Research, METR, Transluce and Goodfire — to the first phase of what it calls embedded assessments: sustained, deep access for independent evaluators to examine alignment and safeguards during training and internal use, not only after a model ships.
Each group has a defined job. Apollo will test whether models could covertly act against their developers’ intentions, and whether safeguards would catch it. METR will study misaligned behaviour in what OpenAI calls its largest reinforcement-learning run to date. Transluce will audit the quality and coverage of OpenAI’s post-training alignment evaluations. Goodfire will embed interpretability researchers to help monitor and debug training runs. According to secondary coverage, access can extend to unreleased models, training checkpoints, source code, training and evaluation data and monitoring systems, with sensitive work done on site on OpenAI devices.
The limits matter as much as the access. Reports go unredacted to the OpenAI Foundation’s Safety and Security Committee. Assessors may publish findings, but OpenAI can redact confidential details and delay disclosure. An eight-week follow-on assessment is planned for early 2027. Some specifics, such as team sizes and payment, vary between published reports.
This sits alongside OpenAI’s description of layered monitoring to catch misalignment and unauthorized agent behavior in its own systems — the lab watching its agents, and now outsiders watching the lab.
For anyone running vendor risk reviews on AI suppliers, the practical move is to add three questions to the questionnaire: which independent evaluators have pre-release access, what they are allowed to publish, and who sees the unredacted findings. A vendor that can answer all three is offering more than a model card.