- Published on
The AI cost of JavaScript frameworks
- Authors
- Name
- DeepHorus
A developer has created a benchmark to measure the “AI cost” of JavaScript frameworks by having AI agents build a Pokémon app. The experiment used Opus 5.5 to test frameworks including Next.js, Nuxt, and SvelteKit, tracking development time, tokens, lines of code, and Lighthouse scores. It responds to the web community’s habit of choosing frameworks based on unsubstantiated “vibes.”
Critics have long noted this tendency: Alex Russell calls it “frontend’s lost decade,” and Baldur Bjarnason describes programming as pop culture. Even in the agentic era, claims that certain frameworks are “better for agents” often lack evidence. The benchmark aims to provide empirical data to fill that gap.
To ensure fairness, each framework used its official MCP or skills, and agents ran in a sandbox without knowledge of Lighthouse testing. A spec conformance suite based on the accessibility tree prevented cheating, and CSS frameworks like Tailwind were prohibited. The app featured Pokémon lists with types, moves, and game data.
The author acknowledges caveats: the app is content-based with little interactivity, and results are stochastic because LLMs are random-word generators. The benchmark is not perfect but offers a rare empirical comparison. It highlights the need for more evidence-based decision-making as AI agents become developers.
The results, though variable, underscore that framework choices should be grounded in data rather than gut feelings. Without benchmarks, claims about agent-friendliness remain speculative. The experiment provides a template for evaluating how different tools perform under AI-driven development, potentially guiding future framework selection.
The developer hopes the benchmark will inspire more rigorous testing and counter vibes-based decision-making. As AI agents take on larger coding roles, objective metrics will become increasingly important for teams. More comprehensive benchmarks are needed to cover diverse application types and frameworks to ensure reliable guidance.
