GPU vs TPU is the hardware question buyers keep asking this fall, and the answer shifted in the past three weeks. Nvidia says Vera Rubin is in full production. Oracle made Vera Rubin NVL72 generally available on October 27. Google’s Ironwood TPU runs production work on Google Cloud while the company talks up its eighth-generation parts. AMD reported a record data center quarter on November 3 and still ships the Instinct MI355X while its MI450 racks wait for 2027.
Google is also changing how it sells silicon. CNBC reported on October 28 that Google is turning to partners to move more of its AI chips, and the Oracle deal announced a day earlier points the same way. For years a TPU meant renting from one cloud. That rule now has an expiry date attached, even if the date sits in 2027.
This guide compares the three accelerators most teams shortlist today: Nvidia Vera Rubin, Google TPU Ironwood and AMD Instinct MI355X. The Vera Rubin platform deep dive covers the silicon itself. This piece focuses on the decision: what each chip is built for, what the published numbers show, what nobody has published yet, and how to move a workload if you switch.
Don't miss new tech stories on Google
Add Tech Insider once in the Google app and our stories appear in your news suggestions.
GPU vs TPU in Brief: The Short Answer
Pick Nvidia Vera Rubin when you need the widest software coverage, rack-scale training, and the option to rent from several clouds. Pick Google TPU Ironwood when your stack already lives on Google Cloud and inference cost per token is the number your finance team watches. Pick AMD Instinct MI355X when memory per GPU is your bottleneck and you can accept a smaller ecosystem in return for more capacity per card.
That summary hides real caveats. No vendor has published a clean, independent head-to-head of all three chips on the same model, the same precision and the same batch settings. The benchmark numbers that exist come from the vendors themselves, from MLPerf submissions the vendors chose to publicize, and from a handful of analyst write-ups. Treat every ratio in this article as a claim with a source attached, and read the caveat column before you quote a number to your boss.
The core split is simple. A GPU is a general parallel processor that grew into an AI engine, backed by a mature software layer that nearly every framework targets first. A TPU is a purpose-built tensor processor that Google designs for its own workloads and then rents out. Both aim at the same matrix math. They differ in how much flexibility they keep, how they connect at scale, and who controls the roadmap.
How GPUs and TPUs Differ at the Architecture Level
Both chip families multiply large matrices as fast as physics allows. The differences show up in three places: how the silicon is organized, how chips talk to each other, and how software maps a model onto the hardware. Those three choices decide whether a workload runs well or merely runs.
What a GPU Optimizes For
A GPU keeps thousands of small cores busy and surrounds them with high-bandwidth memory. Nvidia and AMD both sell this design, and both have spent years tuning compilers, libraries and drivers around it. The payoff is flexibility. A researcher can try an odd attention variant on Monday and ship it to production on Friday without waiting for a compiler team to catch up.
The cost of that flexibility is silicon and power spent on features a given model may never touch. Nvidia answers by selling the whole stack as one platform. According to NVIDIA’s own materials, the Vera Rubin platform pairs Rubin GPUs with Vera CPUs, ConnectX SuperNICs and BlueField-4 DPUs, and the company positions it for pretraining, post-training, inference and agentic workloads. AMD takes a different route and leans on large memory per GPU and open software.
What a TPU Optimizes For
A TPU trades breadth for focus. Google designs it around the tensor operations its own models use, and it links chips into large pods with a dedicated interconnect. Ironwood is the seventh generation, and Google Cloud documentation calls it TPU v7x. Current coverage also uses the names TPUv7 and TPU7x, so expect all three labels when you search.
The strength of this design is scale and cost. Google reported that an Ironwood superpod contains 9,216 chips, and that a training path showed near-linear scaling across up to 8,192 Ironwood TPUs. Those are Google figures from 2026 and they describe Google’s own tests, so read them as an upper bound on what a tuned team can reach, not a promise for a first deployment.
Meet the Three Contenders: Rubin, Ironwood and MI355X
Each of the three has a different status today, and mixing them up is the most common mistake in GPU vs TPU coverage. One is brand new and shipping. One is a proven production part with a successor already named. One is an actively deployed accelerator whose own successor is scheduled for 2027.
Nvidia Vera Rubin: New, Shipping, Rack-Scale
NVIDIA described Vera Rubin as in full production on October 21 at GTC Berlin. On the same day Google Cloud and NVIDIA announced general availability of Vera Rubin infrastructure, with broader instance availability planned for the following month and a European expansion on the schedule. Oracle followed on October 27 when it made Vera Rubin NVL72 systems and Vera CPUs generally available on Oracle Cloud Infrastructure.
The flagship configuration is the Vera Rubin NVL72, a rack-scale system that NVIDIA describes as 72 Rubin GPUs and 36 Vera CPUs linked by NVLink. Oracle also plans to add HGX Rubin NVL8, an eight-GPU x86-based configuration. So you can buy Rubin as a whole rack or, soon, as a more familiar eight-GPU server. NVIDIA has published no purchase price for either.
Google TPU Ironwood: Proven, With Successors Named
Ironwood is generally available on Google Cloud, though sources disagree on the exact start date. One report dates the launch to October 17, while Google Cloud Next coverage points to April 22. Both agree on the substance: you can rent it today. A November 5 SemiAnalysis piece, titled “The Iron(wood) Throne,” argues that Google is staking its claim on inference.
The naming picture is messier. Google has named TPU 8t for training and TPU 8i for inference as eighth-generation parts, and Google claims up to 2.7 times better training price-performance for TPU 8t and up to 80% better inference price-performance for TPU 8i, both relative to Ironwood. A November 4 report still listed both as coming soon, so treat their availability as unconfirmed. The Rubin vs TPU 8i and 8t matchup tracks that race in more detail.
AMD Instinct MI355X: Deployed Today, Successor Due in 2027
The MI355X belongs to the AMD Instinct MI350 Series and uses the 4th Gen CDNA architecture. AMD materials dated November 4 and 5 still center on it, including a day-one support article for Kimi K3.1 Pro. The newer platform is the MI450 Series inside Helios rack-scale systems, which AMD said it is scaling alongside sixth-generation EPYC CPUs. AMD and Anthropic announced on July 22 that up to 2 gigawatts of MI450-series GPUs would deploy, with the first gigawatt starting in the first half of 2027.
So the MI355X is a current, supported part, but it is not AMD’s last word. If you buy it, you buy the generation that carries today’s workloads, not the one that carries the 2027 build-out.
GPU vs TPU Specs Table: What the Sources Confirm
Spec sheets for data center accelerators are uneven. AMD publishes memory, bandwidth and compute figures for the MI355X. Nvidia leads with rack-level claims. Google rarely lists per-chip numbers in marketing copy. The table below sticks to what the maker pages and dated announcements confirm, and it marks gaps instead of filling them with guesses. Memory is the gap that matters most, and the Intel D-Series vs MI355X memory bandwidth comparison shows how far a card can fall behind on that one line.
| Spec | Nvidia Vera Rubin | Google TPU Ironwood | AMD Instinct MI355X |
|---|---|---|---|
| Maker | NVIDIA | AMD | |
| Current product names | Vera Rubin NVL72, Vera CPU, HGX Rubin NVL8 | Ironwood (also TPUv7, TPU v7x, TPU7x) | Instinct MI355X, part of the MI350 Series |
| Architecture family | Rubin GPU with Vera CPU and BlueField-4 DPUs | Seventh-generation TPU | 4th Gen CDNA |
| Product form | Rack-scale platform plus eight-GPU server option | Pods rented through Google Cloud | GPU accelerator in OAM/UBB 2.0 configuration |
| Production status | Full production (Oct 21), GA on Oracle Cloud (Oct 27) | Generally available on Google Cloud | In production use, live at HUMAIN in Saudi Arabia (Aug 31) |
| Memory per chip | Not itemized in the sources used here | Not itemized in the sources used here | 288 GB HBM3E |
| Memory bandwidth | Not itemized in the sources used here | Not itemized in the sources used here | 8 TB/s |
| Compute units | Not itemized in the sources used here | Not applicable | 256 |
| Peak compute | Rack-level claims only | Not itemized in the sources used here | Up to 10.1 PFLOPS theoretical MXFP4/MXFP6 |
| Software path | CUDA, plus NVIDIA reference architectures | TorchTPU (open source Oct 20), llm-d 1.0 | ROCm |
| Networking in announced deployments | BlueField-4 DPUs, Spectrum-X | Google pod interconnect | Cisco Silicon One (HUMAIN deployment) |
| Open design contribution | Rack and tray designs given to the Open Compute Project via MGX (Oct 12) | None announced | Helion open rack blueprint (Oct 12) |
| Named successor or sibling | HGX Rubin NVL8 planned at Oracle | TPU 8t and TPU 8i, availability unconfirmed | MI450 Series in Helios racks, first gigawatt from H1 2027 |
| Where to rent today | Oracle Cloud, Google Cloud, plus other clouds ramping | Google Cloud only until Oracle starts in 2027 | Selected partners and sovereign builds |
| Official purchase price | Not published | Not sold as a hardware purchase in the sources used here | Not published |
Two lessons come out of the table. First, the MI355X is the only chip here with a clean per-card spec line in the sources, and 288 GB of HBM3E with 8 TB/s of bandwidth is a strong one. Second, the blank cells are a finding in themselves. When a vendor sells a system instead of a chip, per-chip numbers stop being the point, and you should compare cost per token on your own model instead.
The lineage matters too. Rubin replaces Blackwell, GB200 and GB300, which stay on the market as earlier platforms. OCI already offered GB200 and GB300 NVL72 systems before Rubin arrived. If you are weighing last generation against this one, the Blackwell vs MI350 comparison covers the predecessors in detail.
Benchmarks: What Four Sources Actually Show
Benchmarks for this class of hardware come with a built-in conflict of interest. The company that sells the chip usually picks the model, the precision, the batch size and the baseline. That does not make the numbers false. It means each one answers a narrow question. The table below lists results the sources confirm, along with the caveat that goes with each one.
| Source | Test | Result | Caveat |
|---|---|---|---|
| AMD ROCm blog (Aug 11 to Oct 6, 2026) | GLM-5.2-MXFP4 serving on MI355X | Peak per-GPU throughput rose from 10,625 to 18,327 total tokens per second (1.72x) | Software gain on the same hardware, not a cross-vendor result |
| AMD Kimi K3.1 Pro article (Nov 4) | Mono decode on MI355X, TP8, 8K input and 1K output, concurrency 1 | Mean time per output token down 33.6% versus the standard decode path | Single-request latency, not throughput under load |
| AMD Kimi K3.1 Pro article (Nov 4) | OCRBench accuracy at max reasoning effort | 0.871 against Moonshot AI’s reference of 0.872 | Shows accuracy parity, not speed |
| AMD MLPerf Inference 6.1 blog (Sep 16) | MLPerf Inference 6.1 submissions | AMD reports MI355X led in specified comparisons against NVIDIA B200, B300 and GB200 configurations | AMD-reported, limited to the comparisons AMD chose |
| NVIDIA (Sep 16) | MLPerf Inference v6.1 debut | NVIDIA reports Vera Rubin NVL72 delivered leading results | NVIDIA-reported, not an independent ranking against every Ironwood and MI355X setup |
Source 1: Vendor MLPerf Submissions
MLPerf Inference 6.1 landed in mid-September, and both camps claimed wins. AMD said MI355X led in specified comparisons with Nvidia’s B200, B300 and GB200 configurations. NVIDIA said Vera Rubin NVL72 delivered leading results in its debut. Both statements can hold at once because they cover different scenarios. The honest reading is that MI355X competes with the Blackwell generation and Rubin debuted ahead of it, which is what a newer generation should do.
Source 2: SemiAnalysis on Ironwood Economics
The strongest independent-leaning claim for TPUs comes from SemiAnalysis. A November 5 analysis reportedly found Ironwood delivering up to 50% better performance per dollar than B200 and B300 on one FP8 inference comparison using the Qwen 3.5 397B model. That is a single workload at a single precision, so it supports a narrow conclusion: on large inference jobs, Ironwood’s price can beat last-generation Nvidia parts. It says nothing yet about Rubin.
Source 3: Nvidia’s Own Efficiency Claims
NVIDIA says the NVL72 can train large mixture-of-experts models with one quarter as many GPUs as Blackwell, deliver up to 10 times higher inference throughput per watt, and cut cost per token to one tenth. Those are NVIDIA claims dating from its March 16 announcement, and no outside lab has confirmed them in the sources reviewed here. The company’s own blog on partner results frames the pitch as better performance per watt and lower token costs. If the claims hold up even partly, they will reshape the cost math below.
Source 4: Google’s Serving-Stack Tests
Google reported that prefix-aware routing and host-memory offload on an Ironwood workload cut median time-to-first-token by more than 85% and raised throughput 2.4 times versus a plain Kubernetes Service. That result measures the serving stack as much as the chip, which is useful in its own right. It shows that gains from scheduling software can rival gains from new silicon, a point that applies equally to the MI355X result above.
Pricing: What Is Published and What Is Not
Price is where GPU vs TPU articles usually overreach. As of November 6, none of the three vendors publishes a list price for the current flagship in the sources reviewed here. What exists is a mix of analyst estimates, third-party cloud listings and silence. The table separates those categories so you can see which numbers deserve trust.
| Item | Nvidia Vera Rubin | Google TPU Ironwood | AMD Instinct MI355X |
|---|---|---|---|
| Official purchase price | Not published | No purchase price in the sources used here | No official price stated |
| Analyst estimate per GPU | About $55,000 (FlexiCloud estimate, not an NVIDIA price) | Not applicable | None found |
| Analyst estimate per rack | $7.8 million to $9.1 million (FlexiCloud estimate, not an NVIDIA price) | Not applicable | None found |
| Confirmed official cloud hourly rate | None confirmed | None confirmed by Google in the sources used here | None confirmed |
| Third-party cloud listing | One marketplace reportedly treated Rubin as reservation-only on Nov 5 | Reported on-demand rate, see the text below | None in the sources used here |
| Vendor price-performance claim | Cost per token down to one tenth versus Blackwell (NVIDIA claim) | TPU 8t up to 2.7x training and TPU 8i up to 80% inference versus Ironwood (Google claim) | None stated |
The one Ironwood rate in circulation comes from a single third-party report. It lists $12.00 per chip-hour on demand in Iowa (us-central1), with discount tiers reported separately. Google has not confirmed that figure in the sources reviewed, so use it as a rough anchor and check the current console price before you budget. Oracle also said product timing and pricing for Rubin may change at its discretion, which tells you not to lock a forecast to launch-week listings.
For a wider look at rental rates across providers, the GPU cloud provider pricing roundup lists serverless and on-demand options, and the AWS vs Azure vs Google Cloud GPU price comparison shows how the big three price Nvidia hardware. Neither replaces a quote for your own workload, but both give you a baseline to negotiate against.
Why Per-Hour Prices Mislead
An hourly rate tells you what a chip costs to occupy, not what a token costs to produce. A pricier chip that serves three times the tokens per hour wins. A cheaper chip that sits half idle because your batch sizes are small loses. That is why the benchmark table and the pricing table belong together, and why the migration harness later in this guide reports cost per million tokens instead of dollars per hour.
Software Ecosystem: CUDA, ROCm and TorchTPU
Hardware gets the headlines, but software decides whether a migration takes a weekend or a quarter. CUDA remains the default target for new models and libraries. NVIDIA announced that CUDA 13.5 provides general-availability support for Vera Rubin, so the toolchain is ready on day one. Teams already on Hopper or Blackwell gain a clear upgrade path.
TPUs have historically asked more from developers. Google’s answer this month is TorchTPU. Google announced on October 20 that TorchTPU, its native PyTorch stack for TPUs, became open source. The project was co-developed with Meta’s PyTorch team and aims to run existing PyTorch workloads on TPUs with minimal code changes. Google reported that TorchTitan, integrated with TorchTPU, was validated on up to 8,000 Ironwood TPUs. The TorchTPU and GPU lock-in analysis digs into what that means for Nvidia’s moat.
Google also announced llm-d 1.0 on October 20, an open distributed-inference stack that supports Google Cloud TPUs and GPUs. That matters because it gives teams one serving layer across both chip families. If you run llm-d, you can compare an Ironwood pool and an Nvidia pool behind the same router and let measured cost per token pick the winner.
AMD’s ROCm Momentum
AMD’s software story is the most active of the three right now. In the last few weeks AMD published day-one MI355X support for Kimi K3.1 Pro, a GLM-5.2 AgentX optimization write-up, and a note on running Kimi K3 with TokenSpeed. A separate AMD example says ROCm was used to port support for GLM-5.3. The pattern is clear: AMD now ships support for frontier open models within days, not months. The remaining gap is breadth. Niche libraries and custom kernels still land on CUDA first.
Scale, Interconnect and Rack Design
At frontier scale, the unit of purchase is a rack or a pod, not a card. All three vendors now sell that way. NVIDIA sells NVL72 racks and a DSX POD reference design for complete Vera Rubin AI factories. AMD announced Helion on October 12 as an open rack-scale blueprint and is scaling Helios racks built around MI450 GPUs. Google sells pods through its cloud.
NVIDIA says DSX POD can cut deployment from about six months to as little as 14 days once site utilities are ready. The company also says its DSX AI Factory Platform can generate the same AI output with up to 30% less power and cut water use for cooling by up to 100%. Those are NVIDIA claims, but they point at the real constraint of 2026: power and cooling often limit a build-out more than chip supply does.
Cooling features in Huang’s own pitch. He told reporters at CES that Vera Rubin’s power draw is twice that of Grace Blackwell, and he added that the company cools the machine with hot water. That pairing matters for buyers. A denser rack can deliver more compute per square foot, but only if your facility can feed and cool it.
Open Designs vs Closed Pods
On October 12 NVIDIA contributed Vera Rubin rack and compute-tray designs to the Open Compute Project through NVIDIA MGX, and AMD pushed its own open Helion blueprint the same day. Google has announced no equivalent open hardware contribution for TPUs. If you value the ability to source racks from several integrators, the GPU camps currently offer more room. If you value a tightly integrated service you never have to rack yourself, the TPU model fits better.
Real-World Deployments: Who Runs What
Marketing claims fade when you look at who has signed and shipped. These six deployments, all dated 2026, show where each chip is actually landing.
- Oracle Cloud and Vera Rubin (October 27). NVIDIA and Oracle announced that Vera Rubin NVL72 systems and Vera CPUs are generally available on OCI, with plans to scale to multi-gigawatt capacity. That is the first big public cloud GA for the rack.
- Meta and Vera Rubin (October 27). Meta announced plans to expand Vera Rubin and Vera CPU deployments for next-generation AI agents, and NVIDIA said it will invest $3 billion in Meta’s equity offering. The Nvidia and Meta Muse agent pact explains the deal. AMD announced its own strategic partnership with Meta on the same day, so Meta is hedging across vendors.
- HUMAIN and MI355X (August 31). AMD, Cisco and HUMAIN said MI355X-based infrastructure was live in Saudi Arabia and serving HUMAIN customers. The HUMAIN MI355X rollout covers the build in detail.
- Anthropic and AMD (July 22). AMD said Anthropic already uses MI355X and plans MI455X in future Helios systems, as part of up to 2 gigawatts of MI450-series GPUs. A frontier lab running AMD at scale is the strongest argument that the ecosystem gap is closing.
- Google and Oracle on TPUs (October 27). The two companies announced a partnership to deploy Google TPU capacity on OCI beginning in calendar year 2027. It is not available there today, and the announcement does not say whether the capacity will be Ironwood, TPU 8t or TPU 8i.
- Project Suncatcher (November 2). Google reported that its first prototype satellite returned an AI-generated response from a data-center-class TPU in low-Earth orbit. It is an experiment, not a product, but the first AI haiku from orbit shows how far Google wants TPUs to travel.
A seventh data point sits in the financials. AMD reported on November 3 that Data Center revenue grew 97% year over year, driven by record sales of Instinct GPUs and EPYC CPUs. Alphabet’s CEO Sundar Pichai said revenue grew 25% year over year and the company is now at a half-trillion-dollar annual run rate. Demand is not the constraint for any of the three.
What Industry Voices Say
The people running these companies are loudest about Nvidia’s platform, so the sourced quotes below skew that way. Read them as positioning, not as neutral testing. When Rubin was announced, Nvidia founder and CEO Jensen Huang said: “Rubin arrives at exactly the right moment, as AI computing demand for both training and inference is going through the roof.”
At CES, Huang told reporters, as NBC News reported: “The power of Vera Rubin is twice as high as Grace Blackwell.” The same article has him describing the cooling design, which runs on hot water, as incredibly efficient. The NBC headline itself carries a caution, calling the chip a possible warning sign for the AI investing boom, which is a fair reminder that power per rack is rising as fast as demand.
On the other side, SemiAnalysis frames the TPU case in its title alone, arguing that Google’s TPU stakes its claim on inference. AMD’s case rests on its own data: its MLPerf blog says MI355X led specific comparisons, and its Q3 release credits record Instinct sales for a 97% jump in Data Center revenue. Each camp tells a coherent story. None of them has published a neutral cross-vendor test, which is why your own measurements matter more than any quote here.
Use-Case Recommendations: Which Chip for Which Job
The table maps six common jobs to a first choice and a runner-up. These are judgment calls built on the published evidence above, not lab results, so adjust them for your own constraints.
| Use case | First choice | Runner-up | Why |
|---|---|---|---|
| Frontier pretraining on a new architecture | Nvidia Vera Rubin | Google TPU Ironwood | CUDA support is ready on day one, and NVIDIA claims a quarter of the GPUs versus Blackwell for large mixture-of-experts training |
| High-volume inference on Google Cloud | Google TPU Ironwood | Nvidia Vera Rubin | SemiAnalysis reports up to 50% better performance per dollar than B200 and B300 on one FP8 test, and TorchTPU lowers the porting cost |
| Serving a very large open model on few cards | AMD Instinct MI355X | Nvidia Vera Rubin | 288 GB of HBM3E and 8 TB/s per card keep big weights on fewer GPUs |
| Multi-cloud or sovereign deployment | Nvidia Vera Rubin | AMD Instinct MI355X | GA on Oracle and Google Cloud, plus open MGX designs. AMD has live sovereign work at HUMAIN |
| Research team with custom kernels | Nvidia Vera Rubin | AMD Instinct MI355X | Niche libraries still land on CUDA first |
| Cost-sensitive startup with PyTorch code | Google TPU Ironwood or AMD Instinct MI355X | Nvidia Vera Rubin | Both avoid Rubin’s launch-week scarcity, and both now have active PyTorch paths through TorchTPU and ROCm |
Three Rules of Thumb
- If your team ships weekly changes to model code, favor the platform with the broadest software coverage. That is still Nvidia.
- If your model is stable and you serve it at huge volume, favor the cheapest cost per token, and test Ironwood and MI355X seriously.
- If your constraint is memory, count cards before you count dollars. A model that fits on four large-memory cards often beats one that needs eight smaller ones.
Migration Guide: Moving a Workload Between GPU and TPU
A switch goes best as a measured experiment, not a leap. The steps below work for GPU to TPU, Nvidia to AMD, or the reverse. They assume an inference workload, since that is where most teams see the first savings.
- Record a baseline. Capture tokens per second, time to first token and p95 latency on your real prompts at your real context lengths. Synthetic prompts flatter every chip.
- Pin precision and model version. Results at FP8 and at MXFP4 are not comparable. Hold both constant, or test each one deliberately.
- Check the framework path. For TPUs, start with the open-source TorchTPU stack. For AMD, start with ROCm and the vLLM-based recipes AMD publishes for new models.
- Put a router in front. A layer such as llm-d, which supports both TPUs and GPUs, lets you send a slice of traffic to each pool and compare them under identical load.
- Validate accuracy before speed. AMD’s Kimi K3.1 Pro write-up reports 0.871 on OCRBench against a reference of 0.872. Run a check like that on your own evals, because a fast wrong answer costs more than a slow right one.
- Compute cost per million tokens. Divide the hourly rate by tokens served per hour at your target latency. Use the harness below.
- Canary and roll back. Shift 5% of traffic, watch error rates for a week, then widen. Keep the old pool warm until you trust the new one.
import time
def cost_per_million_tokens(hourly_rate_usd, generate, prompts, max_new_tokens=256):
"""Return (tokens_per_second, usd_per_million_tokens) for one backend."""
start = time.perf_counter()
total_tokens = 0
for prompt in prompts:
output = generate(prompt, max_new_tokens=max_new_tokens)
total_tokens += len(output.token_ids)
elapsed = time.perf_counter() - start
tokens_per_second = total_tokens / elapsed
tokens_per_hour = tokens_per_second * 3600
return tokens_per_second, hourly_rate_usd / tokens_per_hour * 1_000_000
# Run the same prompts against each backend, then compare:
# gpu_tps, gpu_cost = cost_per_million_tokens(GPU_RATE, gpu_generate, PROMPTS)
# tpu_tps, tpu_cost = cost_per_million_tokens(TPU_RATE, tpu_generate, PROMPTS)Replace the rates with the current prices from your provider’s console and your own signed quote. Run the test at several concurrency levels, since a chip that wins at concurrency 1 can lose under load. AMD’s own mono-decode result was measured at concurrency 1, which is exactly the kind of setting that can overstate a real-world gain.
Common Migration Traps
- Custom CUDA kernels. Any hand-written kernel needs a rewrite or a library replacement on TPU or ROCm. List them first.
- Quantization formats. MXFP4 and FP8 support differs by chip and by library version. Confirm support before you commit to a plan.
- Region and quota limits. Rubin capacity is ramping, and one marketplace reportedly treated it as reservation-only on November 5. Ask about quota early.
- Lock-in by convenience. A TPU-specific serving trick can quietly tie you to Google Cloud. Keep a portable fallback path.
Pros and Cons of Each Accelerator
Nvidia Vera Rubin
- Pros: Widest software coverage with CUDA support ready on day one. Available on Oracle Cloud and Google Cloud now, with more clouds ramping. Open rack designs through MGX. Full-stack reference designs shorten deployment.
- Cons: No published purchase or cloud price. Rack-scale systems demand serious power and cooling. Efficiency claims still need outside confirmation. Capacity looks tight, with reservations common.
Google TPU Ironwood
- Pros: Strong reported performance per dollar on large inference. Pod-scale training that Google says scales near-linearly. TorchTPU is now open source, which lowers the PyTorch porting cost. A cloud partner route through Oracle is coming in 2027.
- Cons: Rentable from Google Cloud only today. Named successors, TPU 8t and TPU 8i, make the buying timeline unclear. Fewer public per-chip specs. The reported $12.00 per chip-hour rate is unconfirmed by Google.
AMD Instinct MI355X
- Pros: 288 GB of HBM3E and 8 TB/s of bandwidth per card. Fast support for new open models, with AMD reporting a 1.72x throughput gain from software tuning alone. Real deployments at HUMAIN and with Anthropic. Open Helion rack blueprint.
- Cons: No official price. The MI450 successor is already scheduled, so you buy the outgoing generation. Narrower library coverage than CUDA. Most performance data comes from AMD itself.
Verdict: Who Wins the GPU vs TPU Matchup in 2026
There is no single winner, and anyone who names one from today’s public data is selling something. There is a clear ranking by situation, though, and the evidence supports it.
Nvidia Vera Rubin wins for most teams. It is in full production, generally available on Oracle Cloud since October 27, announced as generally available with Google Cloud on October 21, and backed by the software stack that new models target first. The missing prices and the unproven efficiency claims are real risks, but they are risks of timing, not of capability.
Google TPU Ironwood wins on inference economics inside Google Cloud. The best outside evidence, a single SemiAnalysis comparison showing up to 50% better performance per dollar than B200 and B300, is narrow but credible. TorchTPU removes the biggest porting objection. The catch is that Google has already named TPU 8t and TPU 8i as better-priced successors, so commit on short terms if you buy now.
AMD Instinct MI355X wins on memory per card and on willingness to compete on price. At 288 GB and 8 TB/s it is the only chip here with a clean per-card spec line, and a 97% jump in Data Center revenue shows buyers are signing. Its weakness is that MI450 is coming, so pick it for 2026 workloads and plan for a refresh.
The takeaway: run a two-week bake-off on your own model before you sign a multi-year commitment, and use cost per million tokens as the scoreboard. For more hardware coverage, browse the hardware category.
FAQ: GPU vs TPU in 2026
What is the main difference between a GPU and a TPU?
A GPU is a general parallel processor that Nvidia and AMD have tuned for AI, backed by broad software support. A TPU is Google’s purpose-built tensor processor, designed around the operations its own models use and sold as a cloud service. Both do the same matrix math, but TPUs trade flexibility for scale and cost, while GPUs keep flexibility and a larger software ecosystem.
Is Google TPU Ironwood faster than Nvidia Vera Rubin?
No independent test answers that yet. The best public Ironwood result compares it with Nvidia’s B200 and B300, not Rubin, and shows up to 50% better performance per dollar on one FP8 inference workload. NVIDIA reports leading MLPerf Inference v6.1 results for Vera Rubin NVL72. Both claims are vendor or analyst reports on different tests, so measure your own workload.
How much does Nvidia Vera Rubin cost?
NVIDIA has not published a purchase price for the Rubin GPU, the NVL72 rack or the Vera CPU. One analyst firm estimates about $55,000 per GPU and $7.8 million to $9.1 million per rack, but those are estimates and not NVIDIA prices. Oracle said Rubin pricing and timing may change, so wait for a written quote.
Can I rent Google TPUs outside Google Cloud?
Not yet. Google and Oracle announced on October 27 that TPU capacity will deploy on Oracle Cloud Infrastructure beginning in calendar year 2027, and it is not available there as of November 6. Until then, Google Cloud is the place to rent Ironwood.
Is the AMD Instinct MI355X still worth buying with MI450 coming?
Yes for 2026 workloads that need memory capacity. The MI355X offers 288 GB of HBM3E and 8 TB/s of bandwidth, AMD still supports it actively, and it runs in production at HUMAIN and with Anthropic. The MI450 Series in Helios racks starts large deployments in the first half of 2027, so plan for a refresh cycle.
Are TPU 8t and TPU 8i available today?
Availability is unconfirmed. Google names them as eighth-generation parts and claims up to 2.7 times better training price-performance for TPU 8t and up to 80% better inference price-performance for TPU 8i, both versus Ironwood. A November 4 report still described them as coming soon. Check Google Cloud’s product pages before you plan around them.
Can PyTorch code run on a TPU without a rewrite?
Often with few changes. Google open-sourced TorchTPU on October 20, a native PyTorch stack co-developed with Meta’s PyTorch team that aims to run existing PyTorch workloads on TPUs with minimal code changes. Custom CUDA kernels still need replacements, so audit those first.
