A second GPU only speeds up a local LLM when the split mode keeps both cards busy. That choice decides whether a dual GPU LLM build needs an x8/x8 board at all. In the rented-host runs behind our dual RTX 3090 build for local LLMs, Llama 3.3 70B generated 31.9 tokens per second with tensor split and 19.5 with the default layer split (llama.cpp b11741, October 2026, Ubuntu 24.04). In tensor mode both cards ran within 10 W of their 290 W cap.
This guide turns those results into rules for two RTX cards. It covers when a chipset x4 slot is enough and how far apart the slots must sit for your card. It then sizes the power supply and the wall circuit for two cards at full draw, and keeps the upper card cool. It ends with two parts lists priced on Amazon US on November 16, 2026, one near $3,760 for 32 GB of VRAM and one near $17,900 for 64 GB at scarce 5090 prices.
Two builds at a glance
These are the parts that decide the layout. The parts lists further down have the full builds with CPU, memory, SSD and cooler, and the reasoning behind each pick:
| Part | Build A: 32 GB VRAM, x16/x4 | Build B: 64 GB VRAM, x8/x8 |
|---|---|---|
| Graphics cards | 2x PNY RTX 5060 Ti 16GB OC Dual Fan | 2x MSI RTX 5090 32G Ventus 3X OC |
| Motherboard | ASRock B850 Steel Legend WiFi | ASRock X870E Taichi OCF |
| Power supply | MSI MAG A850GL PCIE5 850 W | MSI MEG Ai1600T PCIE5 1600 W |
| Case | Antec Flux Pro (8 slots) | Phanteks Enthoo Pro 2 Server Edition (11 slots) |
| Full build total | $3,759.00 | $17,896.75 |
Each part has a longer comparison behind it. The boards come from our AM5 and workstation boards for two or more GPUs, and the power supplies from the best PSU for a multi-GPU AI build. For the cards, our RTX 5070 Ti vs 5060 Ti for local AI tests the budget card, and RTX 3090 vs 5090 for local AI covers the top end.
Dual 3090 results by split mode
Every row comes from the rented-host runs for the dual RTX 3090 build post linked above. The per-card power and temperature come from the nvidia-smi logs recorded during those runs, which that post does not print, and this guide adds them here. GPU0 and GPU1 are listed in that order in the last two columns.
| Run (Llama 3.3 70B) | Split | Generation tok/s | Average W per card | Peak Β°C per card |
|---|---|---|---|---|
| llama.cpp Q4_K_M | layer (default) | 19.5 | 248 / 219 | 47 / 50 |
| llama.cpp Q4_K_M, 8K context filled | layer (default) | 18.3 | 259 / 244 | 51 / 55 |
| llama.cpp Q4_K_M | tensor | 31.9 | 283 / 282 | 50 / 55 |
| llama.cpp Q4_K_M, 8K context filled | tensor | 29.9 | 277 / 274 | 54 / 59 |
| vLLM AWQ, 1 user | tensor parallel 2 | 31.3 | 285 / 289 | 55 / 62 |
| vLLM AWQ, 8 users | tensor parallel 2 | 148.8 total | 289 / 284 | 57 / 66 |
Power is the first thing the table changes. Layer split averaged 219 to 259 W per card, because each card waits while the other runs its half of the layers. Tensor split and vLLM held both cards within 20 W of the 290 W cap at the same time. The split mode that makes the second card worth buying is also the one that loads the power supply hardest, so size for that case.
Heat is the second. GPU1 ran 3 to 9 Β°C hotter than GPU0 in every row, and the gap was widest with eight vLLM users, 66 against 57 Β°C. The host was rented, so which card sat on top is unknown. A gap that always points the same way is what a stacked pair produces, though, and the cooling section below is about closing it.
The bus is the third, and it barely moved. Both cards held a PCIe Gen3 x16 link under load in every two-card llama.cpp run whose log recorded it, which is 15.8 GB/s per direction and the same bandwidth a Gen4 card gets from an x8 slot. Any AM5 board that runs two slots at x8/x8 from the CPU gives each card at least what this rig had while it posted the numbers above.
Where the 3090 numbers come from
The dual 3090 post ran on a rented host, which limits what its numbers can and cannot tell you about slots and cooling:
- Cards and caps: two RTX 3090s, capped at 290 W by the host against a 350 W default, with no NVLink bridge. nvidia-smi reported PHB topology, so traffic between the cards crossed the CPU host bridge.
- Engines: llama-bench from llama.cpp b11741 on the Q4_K_M GGUF, and vLLM 0.31.0 serving an AWQ build at one and at eight concurrent users.
- Power and heat: nvidia-smi sampled once per second. In this guide, averages count only samples at 50 percent GPU utilization or higher, and peaks are the hottest sample in the run.
- Not measured: an x8 or x4 slot. The rental market does not let a renter pick link width, so the x4 figures in the next section are arithmetic from the model shape.
The power supply sizing, slot spacing and parts below come from maker spec pages, manuals and Amazon listings, each named where it is used.
x8/x8 or x16/x4: the split mode decides
Layer split hands one small activation vector from the first card to the second for every token. Llama 3.3 70B has a hidden size of 8192 and 80 layers, per its model config, so that handoff is 8192 values at 2 bytes, 16 KiB. At 19.5 tokens per second that is about 320 KB/s, which even a Gen3 x1 riser carries without trouble.
Tensor parallel is different. vLLM documents that it implements the Megatron-LM tensor parallel algorithm, which needs two all-reduces per transformer layer on the forward pass. A 70B model therefore syncs 160 times per generated token. At 16 KiB each that is about 2.6 MB per token, or about 390 MB/s for eight vLLM users. If llama.cpp tensor mode, which its help text still marks experimental, syncs the same way, 31.9 tokens per second on the 3090 pair needs about 84 MB/s. Reading a prompt at its measured 763 tokens per second pushes it to about 2 GB/s.
Set those figures against what each link carries. The per-lane rates come from the PCI-SIG FAQ (8, 16 and 32 GT/s per lane per direction for Gen3, Gen4 and Gen5) after 128b/130b encoding:
| Link | GB/s per direction | Where a second card meets it |
|---|---|---|
| Gen5 x8 | 31.5 | An RTX 50 card in an x8/x8 board fed by the CPU |
| Gen4 x8 | 15.8 | An RTX 3090 or 4090 in the same x8/x8 board |
| Gen3 x16 | 15.8 | The rented host in the dual 3090 post |
| Gen4 x4 | 7.9 | The chipset slot on both ASRock boards in the parts lists |
| Gen3 x4 | 3.9 | Older chipset slots and M.2 to PCIe adapters |
On paper a Gen4 x4 slot leaves headroom for single-user generation even in tensor mode, and the squeeze shows up in prompt processing and batch serving, where traffic runs in GB/s. The arithmetic also hides latency: 160 small syncs per token through the host bridge add up on GeForce cards, and bandwidth figures do not capture that. The 3090 runs could not isolate it, so treat x4 under tensor parallel as a risk, not a known cost.
That gives three cases, decided by how you plan to run the model:
- Layer split only: x16/x4 is fine. The second card adds memory, not speed, and the bus never fills. Spend the board savings on VRAM.
- Tensor split for one user: x16/x4 should hold on the numbers above, but this is the mode that gave the 3090 pair 1.64 times the layer split speed, so buy x8/x8 when the price gap is small.
- vLLM for several users or long pasted documents: x8/x8 with both slots wired to the CPU. Prefill traffic is where a 7.9 GB/s link would first run out.
Read the board manual, not the chipset name. AMD lists B850, X870 and X870E as supporting “1×16 or 2×8” graphics lanes from the CPU on its AM5 chipset page. Yet the ASRock B850 Steel Legend WiFi wires its second x16 slot to the chipset at Gen4 x4. It also turns that slot off when the M2_4 socket holds a drive. Our guide to PCIe lanes for GPU and NVMe builds covers how M.2 sockets borrow lanes.
Slot spacing against card thickness
Card thickness decides which board works far more often than lane count does. One expansion slot position is about 20 mm, and the air gap between two cards is the distance between the two x16 slots minus the thickness of the upper card. Here are the cards in this guide, with thickness from the maker:
| Card | Thickness | Power | Source |
|---|---|---|---|
| PNY RTX 5060 Ti 16GB OC Dual Fan | 39.9 mm, 2-slot | 180 W, 1x 8-pin | PNY brochure |
| RTX 5090 Founders Edition | 2-slot | 575 W | NVIDIA |
| MSI RTX 5090 Ventus 3X OC | 67 mm, about 3.3 slots | 575 W, 1x 16-pin | MSI |
| RTX 4090 Founders Edition | 61 mm, 3-slot | 450 W | NVIDIA |
| RTX 3090 Founders Edition | 3-slot | 350 W, 2x 8-pin via the included 12-pin adapter | NVIDIA |
Slot distances here are read from the layout drawings in the ASRock manuals. The B850 Steel Legend WiFi puts its two x16 slots about 104.6 mm apart, roughly five positions. A pair of 39.9 mm 5060 Ti cards leaves about 65 mm of open air there. The X870E Taichi OCF puts its x8/x8 slots about 82 mm apart, and a 67 mm 5090 Ventus in the upper one leaves about 15 mm, under one slot position.
Then count the case. Both boards place the lower x16 slot at about position seven. A 2-slot card there covers positions seven and eight, and the 3.3-slot Ventus covers seven to ten. A standard 7-slot ATX case closes on neither. That is why Build A uses the 8-slot Antec Flux Pro and Build B uses the 11-slot Phanteks Enthoo Pro 2 Server Edition.
For other AM5 and workstation boards, including the Taichi Lite in the dual 3090 build, the multi-GPU motherboard comparison lists slot positions per model. Check the slot positions there before you choose a card thicker than 3 slots.
PSU sizing for two cards at full draw
Start from sustained draw with both cards at full power, because tensor split held the 3090 pair there for the whole run. Add the GPU board power from NVIDIA, the CPU TDP from AMD and 150 W for the board, memory, drives and fans. Divide by 0.9 for power supply losses to get what the outlet sees. The Intel ATX version 3 power supply design guide allows 100 to 300 W for the rest of the platform, so 150 W is a middle figure.
| Pair and CPU | GPUs | Sustained DC | At the wall (90%) | PSU |
|---|---|---|---|---|
| 2x RTX 5060 Ti, Ryzen 7 9700X (65 W) | 360 W | 575 W | about 640 W | 850 W |
| 2x RTX 3090, Ryzen 7 9700X | 700 W | 915 W | about 1,020 W | 1,200 to 1,500 W |
| 2x RTX 4090, Ryzen 9 9950X (170 W) | 900 W | 1,220 W | about 1,360 W | 1,600 W |
| 2x RTX 5090, Ryzen 9 9950X | 1,150 W | 1,470 W | about 1,630 W | 1,600 W with both cards capped |
Short spikes sit on top of that. The power excursion table in the Intel ATX version 3 guide (revision 2.1a) requires a unit over 450 W with a 12V-2×6 connector to carry 200 percent of its rating for 100 Β΅s, 160 percent for 10 ms and 120 percent for 100 ms. The same guide says its budgets assume one card using excursions and leaves systems with two to the builder.
That gap matters more under tensor parallel. Both cards work on the same layer at the same moment, so their spikes can line up rather than average out. Keep sustained draw near 70 to 80 percent of the rating, so the excursion window goes to the spikes instead of steady load. That is why the 3090 row lists 1,200 W as the floor.
Then check the outlet. Under the NEC continuous load rule a 15 A, 120 V circuit carries 1,440 W for long runs, and a 20 A circuit carries 1,920 W. Two 5090s at 575 W each need about 1,630 W at the wall. Both cards can be power-limited with the NVIDIA driver tools to about 450 W each, which brings sustained draw to about 1,220 W, or 76 percent of a 1,600 W unit, and the wall figure to roughly 1,360 W, provided that value sits inside the limit range NVIDIA allows for the card. A 20 A circuit lifts the outlet limit but not the PSU margin, so cap the pair on any circuit. CPU socket power can run above TDP, so on a 15 A circuit that capped figure has little margin left under 1,440 W. Our PSU guide for dual RTX 3090 and 5090 builds compares units for both budgets.
Count connectors last. Each 5060 Ti takes one 8-pin, and the MSI MAG A850GL ships four. Each 5090 takes one 600 W 16-pin cable, and the MSI MEG Ai1600T has two native 12V-2×6 outputs, one per card, with no adapters in the path.
Cooling the upper card
Open-air cards pull air in through fans on their underside. In a stacked pair, the upper card’s fans face the back of the lower card and breathe what it just warmed. Part of the 3 to 9 Β°C gap in the 3090 logs was already there at idle. With no load, GPU1 sat about 3 to 8 Β°C warmer in the 70B runs. Read the gap as the direction stacking pushes, not a measure of it. Four habits keep that gap from turning into throttling:
- One empty slot position: aim for 20 mm or more between the cards. The 5060 Ti pair on the Steel Legend has about 65 mm, while the Ventus pair on the Taichi OCF has about 15 mm and is the tightest fit in this guide.
- Intake aimed at the gap: front fans level with the space between the cards feed the upper card directly.
- Thinner cards on top: a 2-slot card such as the 5090 Founders Edition leaves two positions of air on the same Taichi OCF where the Ventus leaves less than one.
- Cap the hot card first: if one card runs hotter, a lower power cap on it costs less speed in tensor mode than letting it throttle mid-run.
Judge cooling over an hour of real use, not a two-minute benchmark. The hottest reading in the dual 3090 logs, 66 Β°C, came under the busiest run, eight users on vLLM, while the short llama-bench passes never passed 59 Β°C.
Parts lists at two budgets
Prices are Amazon US on November 16, 2026, new and in stock, and they move daily through Black Friday, so check the listing before you buy. Our Black Friday GPU deals for local AI tracks the cards separately. Build A is the x16/x4 layout with 32 GB of VRAM:
| Part | Build A: 2x RTX 5060 Ti, x16/x4 | Price |
|---|---|---|
| Graphics cards | 2x PNY RTX 5060 Ti 16GB OC Dual Fan | $799.99 each |
| Motherboard | ASRock B850 Steel Legend WiFi | $169.99 |
| CPU | AMD Ryzen 7 9700X | $268.98 |
| Memory | Corsair Vengeance RGB RS 64 GB (2×32) DDR5, up to 6000 MT/s CL40 per the listing | $1,039.99 |
| Power supply | MSI MAG A850GL PCIE5 850 W | $105.18 |
| Case | Antec Flux Pro (8 slots) | $149.99 |
| SSD | Samsung 990 PRO 2TB | $389.99 |
| CPU cooler | Thermalright Peerless Assassin 120 SE | $34.90 |
| Total | $3,759.00 |
Build A bets on layer split and small models. PNY lists the 5060 Ti as a PCIe 5.0 x8 card, so in the CPU slot it gets 31.5 GB/s and in the chipset slot it drops to Gen4 x4 at 7.9 GB/s. Leave the M2_4 socket empty, or the second slot switches off. Memory is the line to watch: 64 GB DDR5 kits ran $1,030 to $1,300 the day we priced them, more than a quarter of this build.
The board is where Build A saves money: one CPU slot takes the first card and a chipset slot takes the second.

Each card takes one 8-pin plug, so the 850 W unit feeds both from its own PCIe cables with no adapters.
The case comes last, picked for an eighth slot so the lower card does not hang past the bottom edge.
Swap any of those four for a thicker card, a board with closer slots or a seven-slot case, and the slot spacing math above has to be redone.
Build B is the x8/x8 layout with 64 GB of VRAM, sized for tensor parallel and vLLM serving:
| Part | Build B: 2x RTX 5090, x8/x8 | Price |
|---|---|---|
| Graphics cards | 2x MSI RTX 5090 32G Ventus 3X OC | $7,299.95 each |
| Motherboard | ASRock X870E Taichi OCF | $499.99 |
| CPU | AMD Ryzen 9 9950X | $539.00 |
| Memory | Same Corsair 64 GB kit | $1,039.99 |
| Power supply | MSI MEG Ai1600T PCIE5 1600 W | $578.00 |
| Case | Phanteks Enthoo Pro 2 Server Edition (11 slots) | $179.99 |
| SSD | Same Samsung 990 PRO 2TB | $389.99 |
| CPU cooler | Arctic Liquid Freezer III Pro 360 | $69.89 |
| Total | $17,896.75 |
The cards are 82 percent of Build B, and the Ventus listing was a third-party seller marked scarce, so the total is the least stable number in this guide. The Taichi OCF runs its two CPU slots at x8/x8 Gen5. Its chipset slot, PCIE1, falls to x2 when M2_3 is used, so keep it for a network card at most. Plan to power-limit both cards with the NVIDIA driver tools on any circuit: that keeps the 1,600 W unit near 76 percent load and, on a 15 A circuit, inside the outlet budget from the PSU section.
Two of these need a board that splits the CPU lanes evenly, which is the whole reason for the Taichi OCF.
Each Ventus pulls up to 575 W through a single 16-pin plug, so the power supply needs two native cables rather than adapters.
The case has to clear two cards of about 3.3 slots each plus the gap between them.

Build B pays for its width twice: eleven slots so the lower Ventus has somewhere to end, and a native power cable for each card.
Link widths and M.2 lane sharing on these boards
The link a card ends up with depends on three things: how the slot is wired, how many lanes the card has, and which M.2 sockets hold drives. The ASRock manuals for the B850 Steel Legend WiFi and the X870E Taichi OCF give the first and third. The 5060 Ti is an x8 card per PNY, so it never uses more than eight lanes even in an x16 slot. That gives these expected links with both cards installed:
| Build and slot | Slot wiring per the manual | Expected link with the listed card | M.2 socket that shares it |
|---|---|---|---|
| Build A, upper card (PCIE1) | CPU, PCIe 5.0, x16 mode | Gen5 x8 (the 5060 Ti is an x8 card) | None |
| Build A, lower card (PCIE2) | Chipset, PCIe 4.0, x4 mode | Gen4 x4 | M2_4: PCIE2 is disabled when it holds a drive |
| Build B, both cards (PCIE2, PCIE3) | CPU, PCIe 5.0, x16 or x8/x8 modes | Gen5 x8 each | None |
| Build B, spare slot (PCIE1) | Chipset, PCIe 4.0, x4 mode | Network card at most | M2_3: PCIE1 drops to x2 when it holds a drive |
Plan the drives around that table before the build. On the Steel Legend, a drive in M2_4 does not slow the second card down, it switches its slot off, so the card simply disappears. Put the boot drive and any second SSD in the other sockets. On the Taichi OCF the CPU slots share nothing with the M.2 sockets, so drive placement only matters for the small PCIE1 slot.
When the board reports its links, read them under load. The nvidia-smi logs from the dual 3090 runs show both cards at Gen1 while idle and Gen3 once a model was running, at width 16 throughout, because the link drops its speed to save power when the card has nothing to do. An idle Gen1 reading is normal. A width lower than the table above under load usually means a card that is not fully seated, a riser cable, or the slot mode set in the BIOS, and on boards that share M.2 lanes with a GPU slot, a drive in the wrong socket. Our nvidia-smi guide covers reading those fields on Linux.
Which build fits which model size
Build choice comes down to what has to fit in memory with room for context. Weight sizes below are the GGUF files from the dual 3090 runs, and our LLM VRAM calculator covers other models and quantizations:
| Model at Q4_K_M | Weights | Build A, 32 GB | Dual 3090, 48 GB | Build B, 64 GB |
|---|---|---|---|---|
| Qwen3 32B | 18.4 GiB | Fits, long context | Fits, long context | Fits, long context |
| Llama 3.3 70B | 39.6 GiB | Does not fit | Fits, 8K context in the 3090 runs | Fits, room for long context |
Build A is a 32B machine, and that is a fair place to stop if the models you run are in that class. A 70B needs 48 GB or more, and Build B costs nearly five times Build A to get there. Two renewed RTX 3090s at $2,399.99 each on the listing we checked reach 48 GB for far less. Our best GPU for LLMs by VRAM and budget weighs that pair against a single 5090. Whichever pair you pick, the order of decisions stays the same: split mode, then slots, then watts at the wall.





