- Published on
Building Token‑Metered AI Services on Telco AI Factories
- Authors
- Name
- DeepHorus
Telcos worldwide are building sovereign AI factories based on the NVIDIA Cloud Partner (NCP) reference architecture, but are now realizing that infrastructure alone does not deliver high-margin, production-ready enterprise AI services. The key shift involves moving from selling GPU hours to delivering AI services metered by tokens, which aligns economic incentives with actual AI consumption. This transition positions telcos to become "token factories" that offer transparent, token-based pricing to enterprises seeking predictable performance without managing underlying clusters.
These AI factories are built on a five-layer model comprising energy, chips, infrastructure, models, and applications. Telcos anchor the infrastructure layer with NVIDIA-certified compute, networking, and storage that secure host models and applications. The foundational layer sets the cost of compute-as-a-service and enforces data residency and tenant controls, establishing the baseline cost per token as the stack evolves.
Compute-as-a-Service (CaaS) currently allows telcos to monetize the energy, chips, and infrastructure layers by renting GPU capacity by the hour. A Kubernetes-based platform layer adds managed multi-tenant clusters and GPU scheduling, but billing remains tied to GPU-hours. While CaaS offers flexibility and sovereignty, it keeps the business model anchored in infrastructure time rather than AI output.
Token-as-a-Service (TaaS) represents the economic shift to the model and application layers, where value is measured in tokens, API calls, and workflows. Telcos package GPU capacity into products billed per token, with revenue no longer limited by rental hours but by token throughput at given price points and SLAs. This approach enables enterprises to adopt AI services without operating infrastructure themselves.
Enterprise demand is driving this shift, as companies want production-ready applications with predictable performance metered by token consumption. They seek SLAs tied to AI-native metrics like tokens per second, time-to-first-token (TTFT), and end-to-end query latency. This avoids the complexity of managing clusters, runtimes, or model weights, making TaaS an attractive option for enterprises focusing on core business outcomes.
Telcos typically begin TaaS with a portfolio powered by open-source models such as NVIDIA Nemotron and NVIDIA NIM, which are optimized for NVIDIA-accelerated infrastructure. These services provide transparent token-based economics that enterprises can easily adopt. By integrating with existing cloud billing systems, telcos can offer unified usage-based pricing across their AI service portfolios.
The transition from CaaS to TaaS is not without challenges. Telcos must invest in software platforms for token metering, billing, and SLA monitoring. They also need to ensure model latency and throughput meet enterprise expectations, which requires fine-tuning inference runtimes and hardware utilization. However, the long-term economic upside of TaaS—where margins are higher and stickiness greater—makes this evolution strategic.
Looking ahead, as model sizes and reasoning workloads continue to grow, token demand will rise, making token-metered services even more valuable. NVIDIA's reference architecture and ecosystem provide telcos with the building blocks to innovate in this space. By evolving from infrastructure landlords to token factories, telcos can capture more value in the AI value chain and meet enterprise demands for sovereign, performant, and consumption-based AI services.
