- Published on
Embedded AI evaluators won’t be an easy solution, Hugging Face’s policy chief says
- Authors
- Name
- DeepHorus
At TechCrunch Disrupt last week, Hugging Face’s chief policy officer Irene Solaiman cautioned that embedded AI evaluators are not a straightforward solution to the industry’s benchmarking woes. She emphasized that trust in evaluations hinges on reproducibility and access to model weights, not just the presence of third-party observers. Her comments come as AI labs increasingly rely on benchmarks to demonstrate model superiority and safety.
The AI industry has staked its reputation on model benchmarking, a supposedly neutral system for comparing AI products. However, the methods behind these evaluations remain complicated and immature, Solaiman noted. As safety incidents raise concerns, labs have pushed for embedded evaluators—neutral third parties placed inside organizations—as an alternative to active government regulation.
Solaiman stressed that reproducibility is a sticky issue: “Do you have the level of access you need to the model itself?” she asked, pointing to open weights as crucial. Evaluations are an evolving science, she added, and the industry lacks ways to meaningfully compare different tests. Moreover, tests become ineffective quickly, and the ecosystem for evaluators is not thriving enough.
The qualifications for embedded evaluators need standardization, Solaiman argued. While there is hunger for such an ecosystem, determining who is an effective evaluator across tasks remains unclear. This concern is amplified by recent backlash over Anthropic hiring Accenture as its first evaluator, with critics questioning the consulting firm’s qualifications for the task.
Evaluations are also shifting from testing just the model to testing the entire system around it, including the harness and agent. Solaiman noted that a weaker model with a better harness could outperform a frontier model. However, agentic evaluations are “really hard and really expensive,” and there are far fewer evaluations for harnesses and holistic agents than for models.
The current evaluation landscape is a free-for-all, Solaiman implied, and if benchmarking is to remain a useful tool, a more uniform, industry-wide approach may be necessary. Doubts persist over how neutral or effective embedded evaluators will be. Without reproducibility and standardization, trust in evaluations—and in the AI systems they assess—will remain elusive.
As AI labs grapple with safety incidents, the push for embedded evaluators reflects a broader trend of seeking third-party oversight instead of regulation. But Solaiman’s remarks underscore that without a consistent science of evaluation, these efforts may fall short. The industry must come together to develop standardized, reproducible benchmarks that can genuinely inform decisions about AI safety and performance.
