[yichiche]

Motivation

InferenceX #2693 adds HiCache host-DRAM KV arms to the Qwen3.5 MXFP4 MI355X AgentX sweep (TP4 at concurrency 40/48/56/64 and TP2 at 20/24/28/32). The cookbook currently has no way to generate that configuration: the MXFP4-on-MI355X recipe always emits --disable-radix-cache, and HiCache extends the radix cache, so the two cannot be combined by hand-editing the generated command — _handle_cache_compatibility() raises ValueError when --enable-hierarchical-cache and --disable-radix-cache are both present.

This adds a KV Cache Offloading selector to the Qwen3.5 deployment generator so the HiCache variant is a first-class option rather than something a reader has to reconstruct from the benchmark scripts.

While here, this also corrects a note that #35445 left stale. That PR started emitting --kv-cache-dtype fp8_e4m3 for the MXFP4-on-MI355X recipe but did not update the page, which still told readers "It is not included in the generated commands above."

Modifications

docs/src/snippets/autoregressive/qwen35-deployment.jsx

• New kvOffload option group, KV Cache Offloading, with Disabled (default) and Host DRAM (HiCache). It is shown only for mi355x + fp4, which is the arm HiCache is tuned on; every other combination is untouched.
• When HiCache is selected the MXFP4/MI355X branch emits --enable-hierarchical-cache --hicache-ratio 1.5 --hicache-write-policy write_through --hicache-io-backend direct --hicache-mem-layout page_first_direct and drops --disable-radix-cache. Values match the InferenceX recipe.

docs/cookbook/autoregressive/Qwen/Qwen3.5.mdx

• New Configuration Tips entry describing what the HiCache tier buys, the mutual exclusion with --disable-radix-cache, and the host-DRAM budget implied by --hicache-ratio 1.5.
• FP8 KV Cache warning reworded to say the flag is emitted for the MXFP4-on-MI355X recipe and omitted elsewhere.

TP2/EP1, the other half of InferenceX #2693, needs no change here: the generator's mi355x.fp4 entry carries no ep key, so it never emitted --expert-parallel-size and was already EP1. EP2 only ever applied to the B200 arm.

Accuracy Tests

Not applicable — documentation only, no runtime code paths are touched.

Speed Tests and Profiling

Not applicable — documentation only.

The generator was exercised directly to confirm the emitted commands are valid and that nothing else regressed:

Case	--enable-hierarchical-cache	--disable-radix-cache	Conflict
MI355X FP4 + MTP, HiCache off	no	yes	no (identical to before this change)
MI355X FP4 + MTP, HiCache on	yes	no	no
MI355X FP4 no-MTP, HiCache on	yes	no	no
MI355X FP8 (selector hidden)	no	no	no
B200 FP4 (selector hidden)	no	yes	no

The HiCache flag values are taken from the InferenceX recipe; that sweep has not published results yet, so no throughput numbers are quoted here.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

---

CI States

Latest PR Test (Base): ✅ Run #32800180789
Latest PR Test (Extra): ❌ Run #32800186335
Latest PR Test (AMD ROCm 7.2): ➖ No AMD PR run found for this commit.

[1am9trash]

LGTM

[sogalin]

LGTM