How Cloudflare Freed 100TB of RAM with Five Rust Optimizations

The Challenge of Scale#

When you're running one of the world's largest DNS resolvers, every byte counts. Cloudflare's Big Pineapple platform—the infrastructure behind 1.1.1.1, Gateway DNS, DNS Firewall, and AS112—stores over 250 billion DNS cache entries at any given time. At that scale, a single wasted byte per entry translates to more than 250 gigabytes of memory across the entire fleet.

That realization kicked off a systematic optimization effort that ultimately freed approximately 100 terabytes of memory—equivalent to the RAM in 130 of Cloudflare's Gen 13 servers. But perhaps more impressively, the cache didn't just get smaller. It got faster.

Five Rust-Level Optimizations#

The Cloudflare team applied five successive changes to how DNS cache entries are stored in memory, each one finding waste the previous optimization left behind.

1. Drop the Capacity Field#

Rust's Vec<T> and String types carry three fields: a pointer, a length, and a capacity. The capacity field exists to support growth—when you push items, Vec checks if there's room and reallocates if needed. But once a DNS response is cached, it never changes. That capacity field becomes dead weight.

Switching to Box<[T]> and Box<str> eliminated the capacity field and the over-allocated heap space that Vec reserves. Each cache entry stores 8 Vec and String fields, so this change saved 64 bytes per entry, adding up to over 15 terabytes across the fleet.

2. Merge Record Lists#

DNS responses contain three sections: answer, authority, and additional. Originally, these were stored as three separate lists, each with its own 8-byte pointer and 8-byte length. By storing a single list with two 2-byte offsets marking where each section begins, the team removed two full list headers—saving 28 bytes per entry.

3. Drop Redundant Owner Names#

Each DNS record has an owner—the domain the record belongs to. In most cases, this owner is identical to the queried domain. For example, a query for example.com A returns records where the owner is also example.com. The cache key already contains that information.

By making the owner field optional (Option<Box<Name>>) and setting it to None when the owner matches the query, the majority of records avoid a heap allocation entirely. Only records like CNAMEs, where the owner differs from the queried domain, store the full name.

4. Box Large Enum Variants#

Rust enums are sized to fit their largest variant. The RecordData enum originally stored every DNS record type inline—A, AAAA, TXT, NAPTR, SVCB, and more. The problem? NAPTR records are 136 bytes, while A records are just 4 bytes and AAAA records are 16 bytes.

Since A and AAAA make up over 80% of traffic, most records were wasting over 120 bytes on padding. Boxing the larger variants moved them to separate heap allocations, shrinking the enum to 24 bytes for common cases.

5. Store Records in Wire Format#

The final optimization replaced parsed enum variants with raw DNS wire format. Instead of storing structured record data that must be serialized field-by-field when building responses, records are now stored as a Box<[u8]> containing length-prefixed byte sequences.

For A, AAAA, TXT, and DNSSEC records—the vast majority of traffic—this means the cached data can be copied directly into outgoing responses. Only records containing domain names (CNAME, NS, MX, SOA) still require parsing for DNS name compression.

This change eliminated per-record boxing overhead, improved CPU cache locality by packing data contiguously, and reduced work on the lookup path.

The Results#

In production, these optimizations reduced per-entry memory from 953 bytes to 420 bytes—a 56% reduction. Per-entry allocations dropped 58%. But memory savings weren't the only gain.

Performance improved across the board:

• Cache insert throughput increased 43%, from 625,000 entries/second to 893,000 entries/second
• Lookup latency dropped 19%, from 828 nanoseconds to 670 nanoseconds
• P99 per-instance memory fell from 9.3 GB to 5.3 GB (43% reduction)
• P90 per-instance memory dropped from 6.5 GB to 3.8 GB (42% reduction)

The rollout began on May 18, 2026, and completed on July 6, 2026. Aggregate working-set memory across the fleet ended up roughly 100 terabytes lower once the changes settled.

Why This Matters#

These aren't exotic tricks. Boxed slices, deduplication, enum boxing, and wire format storage are all standard patterns in Rust performance work. What's remarkable is that a production system at this scale had all five inefficiencies in the same hot path, and fixing them sequentially produced compounding gains.

The freed memory is now being reinvested into increased cache capacity without raising memory usage, which improves cache hit rates and reduces upstream query load. Further optimizations to the cache layer are already in progress.

If you work with large in-memory datasets—whether DNS, key-value stores, or any system with millions of cached entries—the audit process here translates directly. At 250 billion entries, one wasted byte costs a quarter terabyte. At any scale, the method is the same: measure, optimize, verify, repeat.

---

Source: Cloudflare Blog (August 27, 2026) by Sebastiaan Neuteboom