Google Ironwood: The TPU That Marks the Shift to the Era of Inference

Google Ironwood: The TPU Signaling The Shift To The Era Of Inference

Google is not just keeping up with the AI race — it is redefining the economic rules. With the general availability of Ironwood TPU, unveiled at Google Cloud Next, the company introduces a chip designed primarily for inference.

This marks a significant strategic shift, as the true cost of AI now hinges on execution at scale rather than just training.

Ironwood: A Showcase of Technical and Industrial Strength

On paper, Ironwood impresses. Each chip reaches 4.6 petaFLOPS in FP8, with 192 GB of HBM3e memory and a bandwidth of 7.37 TB/s. However, the real impact comes from the system level, with a superpod of 9,216 TPUs delivering up to 42.5 exaFLOPS, surpassing the performance of the El Capitan supercomputer.

Our Eighth Generation TPUs Inline

In comparison, the Nvidia Blackwell B200 offers similar performance in raw computation, with advantages in interconnectivity (NVLink) and support for FP4. However, Google plays a different card: optimization at the cluster scale and significantly improved energy efficiency.

Why Google Is Focusing on Inference

The choice to prioritize inference is deliberate. Training a model is a one-time cost, while inference represents a continuous, exponential, and structural expense. Google now needs to deliver real-time services for products like Gemini, Google Search, YouTube, and Gmail.

At this pace, inference capacity must double every six months. In this context, controlling the cost per request is key to profitability. This is where Google aims to establish dominance—not by outpacing Nvidia in all arenas, but by crafting the most cost-effective hardware for its own applications.

Ironwood is thus optimized for dominant workloads, including language models, Mixture-of-Experts architectures, diffusion models, and reinforcement learning.

Its high memory capacity helps to reduce model fragmentation, minimizing inter-chip communication—a critical factor for latency and cost.

TPU 8: Google Moves Away from a One-Size-Fits-All Model

TPU 8 Cloud Inline

Perhaps the most significant announcement is Google’s preparation for the next phase, with an eighth generation of TPU, divided for the first time into two distinct architectures:

  • TPU 8t (Sunfish): dedicated to training, co-developed with Broadcom
  • TPU 8i (Zebrafish): optimized for inference, designed with MediaTek

Manufactured in 2 nm by TSMC, these chips are expected to be released by 2027.

This marks a major turning point. Previously, each TPU had to balance training and inference—an ongoing technical compromise. Google now recognizes a market reality: the two functions are fundamentally incompatible in terms of optimization.

A Far-Reaching Industrial Strategy

Ironwood is just the visible tip of a much larger plan. Google is building a multi-partner supply chain: collaborating with Broadcom for high-performance chips, MediaTek for cost-optimized variants, discussions with Marvell Technology, and partnerships with Intel. This is all backed by a monumental investment of up to $185 billion by 2026.

Additionally, a key client is Anthropic, which could utilize up to one million TPUs and several gigawatts of capacity.

The AI War Becomes a Margin War

What Ironwood reveals extends far beyond its technical specifications. The market is shifting: while GPUs dominated training, inference is becoming the real cost center, and hyperscalers are designing their own chips. This results in a massive value transfer. Analysts predict that ASICs could account for 45% of the AI market by 2028, potentially eroding Nvidia’s dominance in inference.

Nevertheless, Nvidia retains a crucial advantage: its software ecosystem, particularly CUDA. As Jensen Huang put it, “Building something better isn’t so simple.”

Google isn’t necessarily aiming to create something “better” overall. Instead, it seeks to reduce costs at scale for its own needs.

Toward an Industrial AI Optimized Like an Electric Grid

Ironwood embodies a silent yet fundamental transition: AI is becoming an infrastructure, akin to energy or telecommunications. In this model, raw performance is less critical than cost per query, energy efficiency, and scalability.

Google is now betting on a strategic certainty: value will no longer be captured by those who provide the chips, but by those who design them for their specific applications.

With two TPU architectures in the pipeline, the company is firmly marking the end of a one-size-fits-all model. The era of general-purpose AI is giving way to specialized, segmented, and industrialized engineering.


Scroll to Top