AI

Dual GPU LLM Build: PCIe x8/x8 vs x16/x4, PSU and Cooling

This post contains affiliate links. If you buy through them, we may earn a small commission at no extra cost to you. Learn more.

A second GPU only speeds up a local LLM when the split mode keeps both cards busy. That choice decides whether a dual GPU LLM build needs an x8/x8 board at all. In the rented-host runs behind our dual RTX 3090 build for local LLMs, Llama 3.3 70B generated 31.9 tokens per second with tensor split and 19.5 with the default layer split (llama.cpp b11741, October 2026, Ubuntu 24.04). In tensor mode both cards ran within 10 W of their 290 W cap.

Original content from computingforgeeks.com - post 191488

This guide turns those results into rules for two RTX cards. It covers when a chipset x4 slot is enough and how far apart the slots must sit for your card. It then sizes the power supply and the wall circuit for two cards at full draw, and keeps the upper card cool. It ends with two parts lists priced on Amazon US on November 16, 2026, one near $3,760 for 32 GB of VRAM and one near $17,900 for 64 GB at scarce 5090 prices.

Two builds at a glance

These are the parts that decide the layout. The parts lists further down have the full builds with CPU, memory, SSD and cooler, and the reasoning behind each pick:

PartBuild A: 32 GB VRAM, x16/x4Build B: 64 GB VRAM, x8/x8
Graphics cards2x PNY RTX 5060 Ti 16GB OC Dual Fan2x MSI RTX 5090 32G Ventus 3X OC
MotherboardASRock B850 Steel Legend WiFiASRock X870E Taichi OCF
Power supplyMSI MAG A850GL PCIE5 850 WMSI MEG Ai1600T PCIE5 1600 W
CaseAntec Flux Pro (8 slots)Phanteks Enthoo Pro 2 Server Edition (11 slots)
Full build total$3,759.00$17,896.75

Each part has a longer comparison behind it. The boards come from our AM5 and workstation boards for two or more GPUs, and the power supplies from the best PSU for a multi-GPU AI build. For the cards, our RTX 5070 Ti vs 5060 Ti for local AI tests the budget card, and RTX 3090 vs 5090 for local AI covers the top end.

Dual 3090 results by split mode

Every row comes from the rented-host runs for the dual RTX 3090 build post linked above. The per-card power and temperature come from the nvidia-smi logs recorded during those runs, which that post does not print, and this guide adds them here. GPU0 and GPU1 are listed in that order in the last two columns.

Run (Llama 3.3 70B)SplitGeneration tok/sAverage W per cardPeak Β°C per card
llama.cpp Q4_K_Mlayer (default)19.5248 / 21947 / 50
llama.cpp Q4_K_M, 8K context filledlayer (default)18.3259 / 24451 / 55
llama.cpp Q4_K_Mtensor31.9283 / 28250 / 55
llama.cpp Q4_K_M, 8K context filledtensor29.9277 / 27454 / 59
vLLM AWQ, 1 usertensor parallel 231.3285 / 28955 / 62
vLLM AWQ, 8 userstensor parallel 2148.8 total289 / 28457 / 66

Power is the first thing the table changes. Layer split averaged 219 to 259 W per card, because each card waits while the other runs its half of the layers. Tensor split and vLLM held both cards within 20 W of the 290 W cap at the same time. The split mode that makes the second card worth buying is also the one that loads the power supply hardest, so size for that case.

Heat is the second. GPU1 ran 3 to 9 Β°C hotter than GPU0 in every row, and the gap was widest with eight vLLM users, 66 against 57 Β°C. The host was rented, so which card sat on top is unknown. A gap that always points the same way is what a stacked pair produces, though, and the cooling section below is about closing it.

The bus is the third, and it barely moved. Both cards held a PCIe Gen3 x16 link under load in every two-card llama.cpp run whose log recorded it, which is 15.8 GB/s per direction and the same bandwidth a Gen4 card gets from an x8 slot. Any AM5 board that runs two slots at x8/x8 from the CPU gives each card at least what this rig had while it posted the numbers above.

Where the 3090 numbers come from

The dual 3090 post ran on a rented host, which limits what its numbers can and cannot tell you about slots and cooling:

  • Cards and caps: two RTX 3090s, capped at 290 W by the host against a 350 W default, with no NVLink bridge. nvidia-smi reported PHB topology, so traffic between the cards crossed the CPU host bridge.
  • Engines: llama-bench from llama.cpp b11741 on the Q4_K_M GGUF, and vLLM 0.31.0 serving an AWQ build at one and at eight concurrent users.
  • Power and heat: nvidia-smi sampled once per second. In this guide, averages count only samples at 50 percent GPU utilization or higher, and peaks are the hottest sample in the run.
  • Not measured: an x8 or x4 slot. The rental market does not let a renter pick link width, so the x4 figures in the next section are arithmetic from the model shape.

The power supply sizing, slot spacing and parts below come from maker spec pages, manuals and Amazon listings, each named where it is used.

x8/x8 or x16/x4: the split mode decides

Layer split hands one small activation vector from the first card to the second for every token. Llama 3.3 70B has a hidden size of 8192 and 80 layers, per its model config, so that handoff is 8192 values at 2 bytes, 16 KiB. At 19.5 tokens per second that is about 320 KB/s, which even a Gen3 x1 riser carries without trouble.

Tensor parallel is different. vLLM documents that it implements the Megatron-LM tensor parallel algorithm, which needs two all-reduces per transformer layer on the forward pass. A 70B model therefore syncs 160 times per generated token. At 16 KiB each that is about 2.6 MB per token, or about 390 MB/s for eight vLLM users. If llama.cpp tensor mode, which its help text still marks experimental, syncs the same way, 31.9 tokens per second on the 3090 pair needs about 84 MB/s. Reading a prompt at its measured 763 tokens per second pushes it to about 2 GB/s.

Set those figures against what each link carries. The per-lane rates come from the PCI-SIG FAQ (8, 16 and 32 GT/s per lane per direction for Gen3, Gen4 and Gen5) after 128b/130b encoding:

LinkGB/s per directionWhere a second card meets it
Gen5 x831.5An RTX 50 card in an x8/x8 board fed by the CPU
Gen4 x815.8An RTX 3090 or 4090 in the same x8/x8 board
Gen3 x1615.8The rented host in the dual 3090 post
Gen4 x47.9The chipset slot on both ASRock boards in the parts lists
Gen3 x43.9Older chipset slots and M.2 to PCIe adapters

On paper a Gen4 x4 slot leaves headroom for single-user generation even in tensor mode, and the squeeze shows up in prompt processing and batch serving, where traffic runs in GB/s. The arithmetic also hides latency: 160 small syncs per token through the host bridge add up on GeForce cards, and bandwidth figures do not capture that. The 3090 runs could not isolate it, so treat x4 under tensor parallel as a risk, not a known cost.

That gives three cases, decided by how you plan to run the model:

  • Layer split only: x16/x4 is fine. The second card adds memory, not speed, and the bus never fills. Spend the board savings on VRAM.
  • Tensor split for one user: x16/x4 should hold on the numbers above, but this is the mode that gave the 3090 pair 1.64 times the layer split speed, so buy x8/x8 when the price gap is small.
  • vLLM for several users or long pasted documents: x8/x8 with both slots wired to the CPU. Prefill traffic is where a 7.9 GB/s link would first run out.

Read the board manual, not the chipset name. AMD lists B850, X870 and X870E as supporting “1×16 or 2×8” graphics lanes from the CPU on its AM5 chipset page. Yet the ASRock B850 Steel Legend WiFi wires its second x16 slot to the chipset at Gen4 x4. It also turns that slot off when the M2_4 socket holds a drive. Our guide to PCIe lanes for GPU and NVMe builds covers how M.2 sockets borrow lanes.

Slot spacing against card thickness

Card thickness decides which board works far more often than lane count does. One expansion slot position is about 20 mm, and the air gap between two cards is the distance between the two x16 slots minus the thickness of the upper card. Here are the cards in this guide, with thickness from the maker:

CardThicknessPowerSource
PNY RTX 5060 Ti 16GB OC Dual Fan39.9 mm, 2-slot180 W, 1x 8-pinPNY brochure
RTX 5090 Founders Edition2-slot575 WNVIDIA
MSI RTX 5090 Ventus 3X OC67 mm, about 3.3 slots575 W, 1x 16-pinMSI
RTX 4090 Founders Edition61 mm, 3-slot450 WNVIDIA
RTX 3090 Founders Edition3-slot350 W, 2x 8-pin via the included 12-pin adapterNVIDIA

Slot distances here are read from the layout drawings in the ASRock manuals. The B850 Steel Legend WiFi puts its two x16 slots about 104.6 mm apart, roughly five positions. A pair of 39.9 mm 5060 Ti cards leaves about 65 mm of open air there. The X870E Taichi OCF puts its x8/x8 slots about 82 mm apart, and a 67 mm 5090 Ventus in the upper one leaves about 15 mm, under one slot position.

Then count the case. Both boards place the lower x16 slot at about position seven. A 2-slot card there covers positions seven and eight, and the 3.3-slot Ventus covers seven to ten. A standard 7-slot ATX case closes on neither. That is why Build A uses the 8-slot Antec Flux Pro and Build B uses the 11-slot Phanteks Enthoo Pro 2 Server Edition.

For other AM5 and workstation boards, including the Taichi Lite in the dual 3090 build, the multi-GPU motherboard comparison lists slot positions per model. Check the slot positions there before you choose a card thicker than 3 slots.

PSU sizing for two cards at full draw

Start from sustained draw with both cards at full power, because tensor split held the 3090 pair there for the whole run. Add the GPU board power from NVIDIA, the CPU TDP from AMD and 150 W for the board, memory, drives and fans. Divide by 0.9 for power supply losses to get what the outlet sees. The Intel ATX version 3 power supply design guide allows 100 to 300 W for the rest of the platform, so 150 W is a middle figure.

Pair and CPUGPUsSustained DCAt the wall (90%)PSU
2x RTX 5060 Ti, Ryzen 7 9700X (65 W)360 W575 Wabout 640 W850 W
2x RTX 3090, Ryzen 7 9700X700 W915 Wabout 1,020 W1,200 to 1,500 W
2x RTX 4090, Ryzen 9 9950X (170 W)900 W1,220 Wabout 1,360 W1,600 W
2x RTX 5090, Ryzen 9 9950X1,150 W1,470 Wabout 1,630 W1,600 W with both cards capped

Short spikes sit on top of that. The power excursion table in the Intel ATX version 3 guide (revision 2.1a) requires a unit over 450 W with a 12V-2×6 connector to carry 200 percent of its rating for 100 Β΅s, 160 percent for 10 ms and 120 percent for 100 ms. The same guide says its budgets assume one card using excursions and leaves systems with two to the builder.

That gap matters more under tensor parallel. Both cards work on the same layer at the same moment, so their spikes can line up rather than average out. Keep sustained draw near 70 to 80 percent of the rating, so the excursion window goes to the spikes instead of steady load. That is why the 3090 row lists 1,200 W as the floor.

Then check the outlet. Under the NEC continuous load rule a 15 A, 120 V circuit carries 1,440 W for long runs, and a 20 A circuit carries 1,920 W. Two 5090s at 575 W each need about 1,630 W at the wall. Both cards can be power-limited with the NVIDIA driver tools to about 450 W each, which brings sustained draw to about 1,220 W, or 76 percent of a 1,600 W unit, and the wall figure to roughly 1,360 W, provided that value sits inside the limit range NVIDIA allows for the card. A 20 A circuit lifts the outlet limit but not the PSU margin, so cap the pair on any circuit. CPU socket power can run above TDP, so on a 15 A circuit that capped figure has little margin left under 1,440 W. Our PSU guide for dual RTX 3090 and 5090 builds compares units for both budgets.

Count connectors last. Each 5060 Ti takes one 8-pin, and the MSI MAG A850GL ships four. Each 5090 takes one 600 W 16-pin cable, and the MSI MEG Ai1600T has two native 12V-2×6 outputs, one per card, with no adapters in the path.

Cooling the upper card

Open-air cards pull air in through fans on their underside. In a stacked pair, the upper card’s fans face the back of the lower card and breathe what it just warmed. Part of the 3 to 9 Β°C gap in the 3090 logs was already there at idle. With no load, GPU1 sat about 3 to 8 Β°C warmer in the 70B runs. Read the gap as the direction stacking pushes, not a measure of it. Four habits keep that gap from turning into throttling:

  • One empty slot position: aim for 20 mm or more between the cards. The 5060 Ti pair on the Steel Legend has about 65 mm, while the Ventus pair on the Taichi OCF has about 15 mm and is the tightest fit in this guide.
  • Intake aimed at the gap: front fans level with the space between the cards feed the upper card directly.
  • Thinner cards on top: a 2-slot card such as the 5090 Founders Edition leaves two positions of air on the same Taichi OCF where the Ventus leaves less than one.
  • Cap the hot card first: if one card runs hotter, a lower power cap on it costs less speed in tensor mode than letting it throttle mid-run.

Judge cooling over an hour of real use, not a two-minute benchmark. The hottest reading in the dual 3090 logs, 66 Β°C, came under the busiest run, eight users on vLLM, while the short llama-bench passes never passed 59 Β°C.

Parts lists at two budgets

Prices are Amazon US on November 16, 2026, new and in stock, and they move daily through Black Friday, so check the listing before you buy. Our Black Friday GPU deals for local AI tracks the cards separately. Build A is the x16/x4 layout with 32 GB of VRAM:

PartBuild A: 2x RTX 5060 Ti, x16/x4Price
Graphics cards2x PNY RTX 5060 Ti 16GB OC Dual Fan$799.99 each
MotherboardASRock B850 Steel Legend WiFi$169.99
CPUAMD Ryzen 7 9700X$268.98
MemoryCorsair Vengeance RGB RS 64 GB (2×32) DDR5, up to 6000 MT/s CL40 per the listing$1,039.99
Power supplyMSI MAG A850GL PCIE5 850 W$105.18
CaseAntec Flux Pro (8 slots)$149.99
SSDSamsung 990 PRO 2TB$389.99
CPU coolerThermalright Peerless Assassin 120 SE$34.90
Total$3,759.00

Build A bets on layer split and small models. PNY lists the 5060 Ti as a PCIe 5.0 x8 card, so in the CPU slot it gets 31.5 GB/s and in the chipset slot it drops to Gen4 x4 at 7.9 GB/s. Leave the M2_4 socket empty, or the second slot switches off. Memory is the line to watch: 64 GB DDR5 kits ran $1,030 to $1,300 the day we priced them, more than a quarter of this build.

PNY GeForce RTX 5060 Ti 16GB OC Dual Fan, the 2-slot card in the budget dual GPU LLM build
PNY RTX 5060 Ti 16GB OC Dual Fan: 39.9 mm thick, 180 W, one 8-pin. Image: PNY.

The board is where Build A saves money: one CPU slot takes the first card and a chipset slot takes the second.

ASRock B850 Steel Legend WiFi AM5 motherboard with a CPU x16 slot and a chipset Gen4 x4 slot
ASRock B850 Steel Legend WiFi: CPU Gen5 x16 slot plus a chipset Gen4 x4 slot about five positions lower. Image: ASRock.

Each card takes one 8-pin plug, so the 850 W unit feeds both from its own PCIe cables with no adapters.

MSI MAG A850GL PCIE5 850W ATX 3.1 power supply for two RTX 5060 Ti cards
MSI MAG A850GL PCIE5: 850 W, 80 PLUS Gold, four 6+2 pin PCIe connectors. Image: MSI.

The case comes last, picked for an eighth slot so the lower card does not hang past the bottom edge.

Antec Flux Pro full tower case with 8 expansion slots for a dual GPU build
Antec Flux Pro: 8 expansion slots and room for a 455 mm card. Image: Antec.

Swap any of those four for a thicker card, a board with closer slots or a seven-slot case, and the slot spacing math above has to be redone.

Build B is the x8/x8 layout with 64 GB of VRAM, sized for tensor parallel and vLLM serving:

PartBuild B: 2x RTX 5090, x8/x8Price
Graphics cards2x MSI RTX 5090 32G Ventus 3X OC$7,299.95 each
MotherboardASRock X870E Taichi OCF$499.99
CPUAMD Ryzen 9 9950X$539.00
MemorySame Corsair 64 GB kit$1,039.99
Power supplyMSI MEG Ai1600T PCIE5 1600 W$578.00
CasePhanteks Enthoo Pro 2 Server Edition (11 slots)$179.99
SSDSame Samsung 990 PRO 2TB$389.99
CPU coolerArctic Liquid Freezer III Pro 360$69.89
Total$17,896.75

The cards are 82 percent of Build B, and the Ventus listing was a third-party seller marked scarce, so the total is the least stable number in this guide. The Taichi OCF runs its two CPU slots at x8/x8 Gen5. Its chipset slot, PCIE1, falls to x2 when M2_3 is used, so keep it for a network card at most. Plan to power-limit both cards with the NVIDIA driver tools on any circuit: that keeps the 1,600 W unit near 76 percent load and, on a 15 A circuit, inside the outlet budget from the PSU section.

MSI GeForce RTX 5090 32G Ventus 3X OC graphics card for a dual RTX 5090 LLM build
MSI RTX 5090 32G Ventus 3X OC: 67 mm thick, 575 W, one 16-pin connector. Image: MSI.

Two of these need a board that splits the CPU lanes evenly, which is the whole reason for the Taichi OCF.

ASRock X870E Taichi OCF AM5 motherboard with x8/x8 PCIe 5.0 slots from the CPU
ASRock X870E Taichi OCF: two CPU PCIe 5.0 slots that run x16 or x8/x8. Image: ASRock.

Each Ventus pulls up to 575 W through a single 16-pin plug, so the power supply needs two native cables rather than adapters.

MSI MEG Ai1600T PCIE5 1600W power supply with two native 12V-2x6 outputs for dual RTX 5090
MSI MEG Ai1600T PCIE5: 1600 W, 80 PLUS Titanium, two native 12V-2×6 outputs. Image: MSI.

The case has to clear two cards of about 3.3 slots each plus the gap between them.

Phanteks Enthoo Pro 2 Server Edition case with 11 expansion slots for two 3.3-slot RTX 5090 cards
Phanteks Enthoo Pro 2 Server Edition: 11 expansion slots and room for a 503 mm card. Image: Phanteks.

Build B pays for its width twice: eleven slots so the lower Ventus has somewhere to end, and a native power cable for each card.

The link a card ends up with depends on three things: how the slot is wired, how many lanes the card has, and which M.2 sockets hold drives. The ASRock manuals for the B850 Steel Legend WiFi and the X870E Taichi OCF give the first and third. The 5060 Ti is an x8 card per PNY, so it never uses more than eight lanes even in an x16 slot. That gives these expected links with both cards installed:

Build and slotSlot wiring per the manualExpected link with the listed cardM.2 socket that shares it
Build A, upper card (PCIE1)CPU, PCIe 5.0, x16 modeGen5 x8 (the 5060 Ti is an x8 card)None
Build A, lower card (PCIE2)Chipset, PCIe 4.0, x4 modeGen4 x4M2_4: PCIE2 is disabled when it holds a drive
Build B, both cards (PCIE2, PCIE3)CPU, PCIe 5.0, x16 or x8/x8 modesGen5 x8 eachNone
Build B, spare slot (PCIE1)Chipset, PCIe 4.0, x4 modeNetwork card at mostM2_3: PCIE1 drops to x2 when it holds a drive

Plan the drives around that table before the build. On the Steel Legend, a drive in M2_4 does not slow the second card down, it switches its slot off, so the card simply disappears. Put the boot drive and any second SSD in the other sockets. On the Taichi OCF the CPU slots share nothing with the M.2 sockets, so drive placement only matters for the small PCIE1 slot.

When the board reports its links, read them under load. The nvidia-smi logs from the dual 3090 runs show both cards at Gen1 while idle and Gen3 once a model was running, at width 16 throughout, because the link drops its speed to save power when the card has nothing to do. An idle Gen1 reading is normal. A width lower than the table above under load usually means a card that is not fully seated, a riser cable, or the slot mode set in the BIOS, and on boards that share M.2 lanes with a GPU slot, a drive in the wrong socket. Our nvidia-smi guide covers reading those fields on Linux.

Which build fits which model size

Build choice comes down to what has to fit in memory with room for context. Weight sizes below are the GGUF files from the dual 3090 runs, and our LLM VRAM calculator covers other models and quantizations:

Model at Q4_K_MWeightsBuild A, 32 GBDual 3090, 48 GBBuild B, 64 GB
Qwen3 32B18.4 GiBFits, long contextFits, long contextFits, long context
Llama 3.3 70B39.6 GiBDoes not fitFits, 8K context in the 3090 runsFits, room for long context

Build A is a 32B machine, and that is a fair place to stop if the models you run are in that class. A 70B needs 48 GB or more, and Build B costs nearly five times Build A to get there. Two renewed RTX 3090s at $2,399.99 each on the listing we checked reach 48 GB for far less. Our best GPU for LLMs by VRAM and budget weighs that pair against a single 5090. Whichever pair you pick, the order of decisions stays the same: split mode, then slots, then watts at the wall.

Keep reading

Ollama Models Cheat Sheet: Best Models by VRAM, Tested (PDF) AI Ollama Models Cheat Sheet: Best Models by VRAM, Tested (PDF) Open Source LLM Comparison Table (2026) AI Open Source LLM Comparison Table (2026) Claude Code Cheat Sheet: Commands, Shortcuts and Flags (PDF) AI Claude Code Cheat Sheet: Commands, Shortcuts and Flags (PDF) MCP vs RAG vs Agent Skills: When to Use Each AI MCP vs RAG vs Agent Skills: When to Use Each Install and Run llama.cpp on Ubuntu 26.04 / 24.04 and Rocky 10 AI Install and Run llama.cpp on Ubuntu 26.04 / 24.04 and Rocky 10 agent-browser on Linux: Install and Drive It from Claude Code AI agent-browser on Linux: Install and Drive It from Claude Code

Leave a Comment

Press ESC to close