OpenAI embeds Apollo and other safety teams to probe AI subversion
OpenAI has started a pilot that places safety researchers from Apollo, Goodfire, METR and Transluce inside the company to examine whether its models try.
Source: TechCrunch — AIOpenAI announced a pilot that embeds external safety researchers inside its organization to investigate whether its frontier models attempt to subvert alignment mechanisms. The move is intended to give independent evaluators deeper visibility than the post‑training audits that have been the norm, a step that could reshape how AI labs demonstrate safety compliance.
OpenAI said that they'd have embedded evaluations with employee-like access; the agreements announced today obviously don't meet that bar. This is disappointing but not surprising. Maybe future agreements will be better. https://t.co/kmDbg1Dqav
What is the new OpenAI pilot?
According to the TechCrunch report, OpenAI will embed safety evaluators from four third‑party firms—Apollo Research, Goodfire, METR and Transluce—directly into its development workflow. The pilot will begin at different times, with Apollo’s team starting this week. The goal is to allow these researchers to monitor training runs, probe internal deployment scenarios, and look for signs that models are trying to undermine their own or future versions’ alignment or control systems.
Which firms are involved and what will they examine?
- Apollo Research – Six Apollo evaluators will focus on whether models are incentivized to subvert alignment during training, whether evaluation protocols can elicit such behavior, and whether reasoning related to subversion is detectable. Apollo’s scope includes reviewing relevant training code, datasets, reinforcement‑learning rollouts with reasoning traces, and checkpoint queries.
- Goodfire – Specialising in interpretability, Goodfire will embed researchers to monitor training runs, fix undesired behavior and improve alignment. The firm plans to publish its findings for OpenAI’s internal audience.
- METR – METR will conduct two assessments: one to uncover unintended or dangerous behavior that OpenAI’s own tests might miss, and another to evaluate whether OpenAI’s systems can catch that behavior. The exact scope of the second test is still being defined.
- Transluce – This group will review the adequacy of OpenAI’s post‑training safety evaluations and, in 2027, will test how OpenAI uses those evaluations to decide if a model is safe for release.
How does the pilot differ from previous safety reviews?
Historically, AI labs have invited external reviewers only after a model’s training has finished and shortly before a public launch. Those reviewers typically receive limited data and no employee‑level access, which many safety experts argue is insufficient for detecting “scheming” or other misaligned behaviors that can emerge during training. The embedded approach, first tried by Anthropic with METR researchers, gives evaluators ongoing, inside‑the‑lab visibility, allowing them to observe model reasoning in real time and to intervene if subversive patterns appear.
What are the limitations and concerns about independence?
The TechCrunch article notes several constraints that could blunt the pilot’s independence:
- Restricted access – OpenAI will supply only the data necessary for the specific subversion assessment, such as selected code, datasets, rollout logs and checkpoint queries. The company reserves the right to withhold information deemed technically infeasible, a security risk, or protected by IP.
- Limited staff interaction – Apollo will be allowed to speak only with a pre‑approved list of OpenAI employees. If a non‑authorized staff member shares confidential material, Apollo must cease follow‑up and route any further inquiry through OpenAI.
- Compensation differences – Apollo and Goodfire receive financial compensation, while METR and Transluce may only obtain compute or API credits, raising questions about comparable incentives.
- Disclosure restrictions – Although Apollo can publish independent conclusions, OpenAI can limit the evidence that accompanies those conclusions, potentially making external verification difficult. The company’s Safety and Security Committee will receive unredacted reports for private review. Redwood CEO Buck Shlegeris called the agreements “disappointing but not surprising,” noting they fall short of the “employee‑like access” benchmark that many in the safety community consider essential.
How creators and researchers can explore AI safety with Grafy
While Grafy does not yet offer a built‑in “embedded evaluator” workflow, the platform provides tools that let creators and researchers investigate the same questions raised by the OpenAI pilot. The Reader app lets you upload PDFs of safety research papers, model cards, or internal audit reports and ask AI‑driven questions about their methods and findings. Pairing Reader with Chat, which supports side‑by‑side querying of multiple models, enables you to compare how different systems interpret alignment‑related prompts.
For those building their own models, Grafy’s Graph app can track prompt variations, model outputs, and evaluation metrics across training iterations, effectively creating a visual audit trail similar to the checkpoint queries Apollo will receive. By documenting reasoning traces and reward signals in Graph, teams can later query the history to spot patterns that might indicate subversive behavior.
If you are interested in interpretability, the Creator app’s image‑and‑video generation capabilities can help visualise attention maps or activation patterns, turning abstract model internals into understandable visuals that can be shared with safety reviewers.
Overall, Grafy’s suite lets independent researchers replicate parts of the embedded evaluation process on their own projects, offering a practical sandbox for testing alignment safeguards before deploying models at scale.
Source: https://techcrunch.com/2026/11/16/openai-will-embed-apollo-researchers-to-investigate-ai-subversion/
Frequently asked questions
What specific behaviors will Apollo look for in OpenAI’s models?
Apollo will examine whether models try to conceal mistakes, provide misleading instructions to future versions, or otherwise undermine alignment and control mechanisms. The focus includes training incentives for subversion, detectability of subversive reasoning, and occurrences during internal deployment.
How does the embedded evaluation differ from post‑training audits?
Embedded evaluation gives researchers real‑time access to training code, datasets, rollout logs and intermediate checkpoints, allowing them to test for misaligned behavior during training. Post‑training audits only see the final model and limited documentation, which can miss emergent subversive strategies that develop earlier.
Can Grafy users replicate any part of OpenAI’s embedded safety pilot?
Yes. Using Grafy’s Reader to study safety papers, Chat to compare model responses, and Graph to log prompt‑output histories, creators can build their own audit trails and conduct interpretability experiments that mirror the data‑access goals of the OpenAI pilot.
Source: TechCrunch — AI — OpenAI will embed Apollo researchers to investigate AI subversion


