Skip to content

Google adds a confidential-inference TPU and a double-memory TPU for agent workloads

· by

The short version

Google's TPU 8tx encrypts data in active use for regulated customers, and TPU 8i Max doubles compute, SRAM, and HBM bandwidth over April's TPU 8i, the fourth and final eighth-generation TPU system of 2026.

Google adds a confidential-inference TPU and a double-memory TPU for agent workloads

Google swapped out its own Axion CPU for an AMD chip on one new TPU just to get hardware encryption agents can run on without ever decrypting data in memory, and paired it on November 9 with a second chip built to stop many agents at once from stalling on a shared memory bottleneck, per Google's own announcement.

What Google shipped

TPU 8tx and TPU 8i Max are the third and fourth configurations of Google's eighth-generation TPU line this year, after TPU 8t and TPU 8i launched at Google Cloud Next in April, per Google's blog post. TPU 8tx reuses TPU 8t's silicon but swaps Google's Axion Arm CPU for a 5th-generation AMD EPYC host so sensitive data and model weights stay encrypted even while a chip is actively processing them, a capability Google is pitching at governments and regulated enterprises running proprietary models alongside confidential data. NextPlatform reported that the switch is specifically to get AMD's on-board root-of-trust and encryption accelerators, something Google's own Axion chip does not yet have. TPU 8tx also moves to Google's lower-latency Boardfly interconnect, shrinking pod size from TPU 8t's 9,600 accelerators down to 1,152 and cutting typical latency roughly in half, per NextPlatform's analysis of the fabric change. TPU 8i Max, meanwhile, fuses two TPU 8i chips into one package, doubling compute, on-chip SRAM, and HBM bandwidth versus the standalone TPU 8i that shipped in April, per Google. This marks the first time Google has introduced four TPU systems in a single calendar year, and its TPU business now counts nearly 100 customers, including frontier labs, trading firms, and universities, per the same post.

Agent fleets are now the TPU design target, not single chat sessions

Both chips target a problem that only shows up once agents, not single prompts, are the workload: dozens of agents reasoning over a shared codebase or hours of video at once will exhaust memory bandwidth on a standard TPU 8i, and a regulated customer running an autonomous agent next to proprietary data cannot risk that data sitting unencrypted in memory mid-inference. Teams already running fine-tuning or reinforcement-learning loops on TPU 8t get a confidential option without switching hardware generations, since TPU 8tx shares TPU 8t's silicon. Teams hitting a memory wall on million-token contexts get roughly double the headroom per rack from TPU 8i Max without a new chip architecture to learn. Neither chip is a drop-in replacement for the April parts; they are additive configurations aimed at the two specific failure modes Google says enterprise and government customers raised after TPU 8t and 8i shipped.

The chip race around Nvidia keeps widening

TechCrunch noted that Microsoft, Amazon, OpenAI, and Anthropic are all now building or have built their own AI-specific chips, each citing the same motive Google has stated for TPUs since 2026 began: reducing reliance on Nvidia. Last month, Google open-sourced TorchTPU, letting PyTorch code and Hugging Face models run natively on TPUs without rewrites, a move the company says is meant to widen the TPU customer base beyond the hand-tuned-code era of its first chip generation a decade ago.

What to watch next

Watch for Google Cloud pricing and general-availability dates for TPU 8tx and TPU 8i Max, since the April TPU 8t and 8i launch took months to reach broad customer access. Also watch whether any of the regulated-industry customers Google is targeting for TPU 8tx go public as named references, which would be the first real signal of enterprise demand for in-memory confidential inference at this scale.

Sources