- Published on
A Stateless Proxy Tier That Unifies ZippyDB Traffic and Handles Over 1 Billion Operations Per Second
- Authors
- Name
- DeepHorus
Meta has introduced ZGateway, a stateless proxy tier that now sits between client applications and ZippyDB, the company's most widely used key-value store. The new layer handles more than 1 billion operations per second and currently carries about 40% of ZippyDB traffic, with projections to pass 60%. ZGateway started as a fix for connection sprawl across more than a million client hosts and grew into the home for batching, admission control, caching, and failover.
ZippyDB backs product metadata, counters, and configuration at billions of operations per second. Under direct access, every ZippyDB client connected to every database host it needed, so a single client could touch tens of thousands of shards across hundreds of thousands of hosts. Each idle connection consumed memory, CPU, and a file descriptor on both ends, and reconnection storms caused crashes from file descriptor exhaustion and OOMs. Client-side fixes were impractical because hundreds of teams own the client fleet.
ZGateway runs as regional tiers discovered through ServiceRouter, Meta's service mesh, in two flavors: a pure proxy and a read-through cache. The engine is Meta's thick C++ ZippyDB client, making ZGateway effectively a ZippyDB client run as a managed service. Meta reports about 6% computational overhead for an average use case.
A client sends requests over a sticky connection to a regional ZGateway host, which terminates TLS, authorizes against the use case's ACLs, and applies per-tenant admission control and shaping. The proxy resolves the shard, checks the local cache on caching tiers, batches the request with other in-flight work for that shard, and forwards it to the correct replicas. Responses are demultiplexed back with per-use-case metrics, traces, and quota usage recorded.
Meta models the fleet as balls thrown into bins, and the math shows dramatic improvements. With mock figures of 20 regions, 500,000 database hosts, 30,000 proxy hosts, 1,000,000 clients, and 50,000 shards per client, per-host connection counts collapse by roughly 97 to 98%, and total persistent connections drop about 19x. The deeper win is scaling: direct-access fan-in grows linearly with clients, while ZGateway fan-in reduces to roughly regions times shard density per host, independent of both fleets.
Meta engineers emphasized safe migration, with configuration flags scoped per service and shard prefix providing a percentage ramp, a region filter, and a global kill switch. The team described ZGateway as a fix that grew into a central control point for traffic management. Industry observers see it as a practical model for taming connection sprawl in hyperscale storage systems.
ZGateway is projected to carry more than 60% of ZippyDB traffic, and further capabilities are likely as adoption expands. The proxy tier approach could influence other large-scale infrastructure operators facing similar fan-in challenges. As systems grow, stateless proxy layers are becoming essential for managing connections and adding control points without requiring changes across hundreds of client teams.
