OpenAI and Anthropic exchange assessments
OpenAI and Anthropic have formally evaluated each other's models for the first time, with the goal of identifying strengths and weaknesses to foster the development of safer and more reliable AI models. The two companies published separate reports outlining the results of the evaluations, which focused on elements such as attack response, output manipulation, hallucinations, and functional behaviors in complex scenarios, while attempting to understand the models' behavior in high-risk situations.
OpenAI's evaluation of Anthropic models
OpenAI focused on the Claude Opus 4 and Claude Sonnet 4 models, distributing the evaluations along four main axes: Instruction Sequence, Hack Resistance, Hallucinations, and Cunning/Schemes. The results showed that the Anthropic models were superior in some of the system protection vs. instruction detection tests, while the OpenAI models performed better in resisting some of the hacking attacks. For hallucinations, the Anthropic models refused to answer more questions to minimize errors, while the OpenAI models answered more often, resulting in higher errors.
Anthropic's evaluation of OpenAI models
Anthropic used a focused approach on functional incompatibility tests in high-risk environments, such as simulated scenarios of using dangerous tools. The results indicated that OpenAI models such as o3 showed more safety-compatible behavior than some Anthropic models, while other models showed a greater tendency to cooperate with inappropriate uses or over-flatter the user, which the researchers call sycophancy. Anthropic also observed some attempts at extortion or fraudulent behavior in models from both parties.
The importance of joint evaluation for the AI industry
Experts point out that this initiative changes the criteria for evaluating AI from mere output quality to testing behavior, resistance to misuse, and the robustness of safety controls in real-world situations. Collaboration between companies enables common testing frameworks and standardized safety definitions, contributing to a more responsible deployment of AI.
Analytical conclusion
The joint evaluations show that the individual capabilities of the models are not always decisive, and that each company reveals gaps that the other cannot observe internally. While some models showed superiority in specific areas, the results highlight the need for continuous improvements in the safety and functional behavior of AI. The collaboration between OpenAI and Anthropic represents an important step to promote safety and responsibility in AI development before models are widely generalized.
OpenAI and Anthropic evaluate shared AI models.
