- Published on
What if AI worked at 1.000.000 tokens per seconds?
- Authors
- Name
- DeepHorus
A field note imagines an AI agent writing 1,000,000 output tokens per second, explicitly as a thought experiment rather than a current-model benchmark. At that pace, 1,000 story endings could be drafted in one second. The harder issue would be choosing which output to trust.
The note separates context capacity, input speed, aggregate throughput and single-agent output. Serving 10,000 streams at 100 tokens per second would total 1,000,000 tokens, without speeding up each stream. A large context window shows what fits, not how quickly a reply is generated.
Single-agent generation is sequential because each token depends on those before it. Anthropic’s Claude Opus 5.5 overview lists a 1M-token context window, a smaller output cap and “moderate” comparative latency. Memory size therefore differs from writing speed.
The source cites batching through the 2023 vLLM paper and speculative decoding, which can accept several drafted tokens per pass. These methods raise throughput but cannot remove a chain requiring one result before the next. Token counts also are not word counts.
Forty lending-app drafts totaling 800,000 tokens could be written in 0.8 seconds, yet tests might still miss a seven-day due date. Ten thousand catalyst ideas totaling 3,000,000 tokens could take three seconds to write, while laboratory work continues for hours, days or a season.
The calculations exclude reading, ranking, tool calls, permissions, tests, deployment and human work. A writing allowance is not a verified product, and aggregate throughput does not equal per-stream completion time. Faster agents would help most when each step waits on the last.
The note’s perspective is that cheap drafts move the hard part rather than remove it. People would still need taste to select endings, clear specifications for software and physical evidence for experiments. Future systems may compete more on verification and judgment than on raw generation speed.
