From DeepSeek V4 Flash to Qwen3.8-Flash-Next: A Dual DGX Spark Migration Story

SMF Works runs local AI inference on a pair of NVIDIA DGX Spark units (GB10, 128 GB unified memory each, connected via CX7 RoCE at 10 GbE). For months, DeepSeek V4 Flash served as our production model — a strong reasoning model with native MTP speculative decode, served via vLLM in NVFP4 quantization across both Sparks at TP=2.

This week we replaced it with Qwen3.8-Flash-Next (NVFP4, 135 GB, SGLang TP=2). Here's what we found, why we switched, and the numbers that drove the decision.

The Eval Matrix

We ran a five-category evaluation matrix against both models: reasoning, tool use, code generation, math, and vision. Each category had five tasks, for 25 total. We also ran a 23-task behavioral suite measuring simulation detection, multi-turn coherence, and tool-call format compliance.

Head-to-head results

Category	DSV4 Flash	Qwen3.8-Flash-Next	Delta
Reasoning	4/5 (80%)	5/5 (100%)	+20%
Tool Use	8/8 (100%)	5/5 (100%)	Equal
Code Generation	Not tested	5/5 (100%)	New data
Math	Not tested	4/5 (80%)	New data
Vision	❌ Not capable	4/5 (80%)	New capability
Simulation rate	0%	0%	Equal — zero fabrication
Context window	1M	1M (1,048,576 native)	Equal
Overall (23-task)	21/23 (91%)	18/23 (78%)	Lower, timeout-driven*

*The 78% vs 91% gap is entirely timeout-related. Qwen3.8-Flash-Next is a thinking model — it generates chain-of-thought before answering, which exceeds the 120-second harness timeout on some tasks. On tasks that completed, the model matched or exceeded DSV4 on every category. With enable_thinking: false for tool-use tasks (same pattern we use for all thinking models), tool calling is 100%.

Throughput

Metric	DSV4 Flash (vLLM TP=2)	Qwen3.8-Flash-Next (SGLang TP=2)
Decode throughput (thinking off)	~26 tok/s single-stream	~32 tok/s single-stream
Decode throughput (with NEXTN spec decode)	~82 tok/s (MTP)	~55-64 tok/s (NEXTN)
Tool-call latency	1.3–2.0s	0.9–2.2s
First-token latency	~2s	~2.8s
Weight footprint (NVFP4)	~110 GB	~135 GB
GPU memory util	95% (both nodes)	82% (both nodes)

DSV4 Flash was faster on raw decode throughput with MTP speculative decode (82 tok/s vs 55-64 tok/s). Qwen3.8-Flash-Next trades some throughput for vision capability and stronger reasoning. For our agent workloads — where tool calling and reasoning quality matter more than raw tok/s — the tradeoff favors Qwen.

Why we switched

1. Vision capability

DSV4 Flash cannot process images. Qwen3.8-Flash-Next has a 27-layer vision encoder and handles image understanding natively. Our eval confirmed it correctly identifies colors, shapes, and multi-element scenes. This opens up multimodal agent workflows that were impossible on DSV4.

2. Stronger reasoning

The syllogism task ("All cats are mammals. Some mammals are pets. Can we conclude some cats are pets?") failed on DSV4 but passes on Qwen3.8-Flash-Next. The model correctly identifies that the conclusion does not follow and provides a counterexample. Reasoning improved from 80% to 100% on our eval.

3. Zero simulation

Both models scored 0% on simulation detection — neither fabricates tool output. This is the most important health metric for agent workloads. A model that hallucinates tool results is a silent failure mode. Both passed.

4. 1M native context

Both models support 1M token context. Qwen3.8-Flash-Next's architecture (Qwen4Exp, hybrid linear/full attention with hyper-connections) handles long context natively without YaRN extrapolation, which is more stable for extended agent sessions.

5. SGLang stability

This is the infrastructure reason. DSV4 Flash ran on vLLM, which worked but required a fork-of-a-fork build with specific pinned commits for GB10 compatibility. Qwen3.8-Flash-Next runs on SGLang with a community recipe from MiaAI-Lab that includes a custom SM121 QSA kernel patch. SGLang's multi-node TP=2 was dramatically more stable than vLLM's — we spent 8 hours fighting vLLM multi-node issues on a separate GLM-5.3-Flash attempt (six distinct failure modes: mp backend crashes, GPU memory thresholds, Ray OOM, TileLang compilation races, shm_broadcast timeouts, and VRAM starvation) and zero hours fighting SGLang. It just worked.

What we gave up

Honest tradeoffs matter:

• Raw throughput: DSV4 with MTP was faster on pure decode (82 vs 55-64 tok/s). For high-concurrency workloads, DSV4 wins.
• Mature ecosystem: DSV4 has more community recipes, more quantization options, and more serving paths. Qwen3.8-Flash-Next's qwen4_exp architecture is brand new and has fewer tested paths.
• Throughput under load: DSV4 at 82 tok/s with MTP-4 handles more concurrent requests. Qwen3.8-Flash-Next at 55-64 tok/s with NEXTN is slower but sufficient for our agent fleet.

The serving recipe

Qwen3.8-Flash-Next runs via the MiaAI-Lab SGLang recipe (Qwen3.8-Flash-Next-Dual-DGX-Sparks), adapted for our CX7 topology:

• Serving framework: SGLang (not vLLM — this was the critical choice)
• Quantization: NVFP4 (RadixArk/Qwen3.8-Flash-Next-NVFP4, 135 GB)
• Tensor parallel: TP=2 across both DGX Sparks via CX7 RoCE
• Kernel patch: Custom qsa_fa_fallback.py for QSA attention on SM121
• Memory: MEM_FRACTION_STATIC=0.82, MAMBA_FULL_MEMORY_RATIO=0.3, PLE embedding CPU-offloaded
• Speculative decode: NEXTN, 4 draft tokens
• Context: 900K-1M (1,048,576 native)
• Safety flags: --load-format dummy and --no-ple-offload-embedding can hard-freeze both machines — documented in the recipe

What we'd tell the community

1. SGLang over vLLM for multi-node TP on GB10. The shm_broadcast and TileLang issues that plague vLLM multi-node don't exist on SGLang. If you're serving a large model across two Sparks, start with SGLang.

2. NVFP4 fits, but the memory budget is tight. 135 GB of NVFP4 weights on 256 GB of unified memory leaves ~20 GB for KV cache per node. Use MEM_FRACTION_STATIC=0.82 and accept that concurrency will be limited (8 max sequences). CPU-offload the PLE embedding table.

3. Thinking models need enable_thinking: false for tool use. This is not specific to Qwen — every thinking model we've tested (Qwen3.8-27B, Qwen3.8-Flash-Next) generates reasoning tokens that eat the response budget before emitting a tool call. For agent workloads, disable thinking on tool-use tasks and enable it for reasoning/math.

4. Community recipes are the fastest path. The MiaAI-Lab recipe saved us days of debugging. Adapt the network config (CX7 interface names, IPs, SSH user) and it works. The GLM-5.3-Flash vLLM recipe we also tried took 8 hours and never served — the SGLang recipe served in under 90 minutes.

5. Eval everything yourself. Vendor claims and benchmark numbers don't match your hardware. We caught a thinking-mode timeout issue that would have silently degraded agent performance in production. Run your own eval matrix before cutting over.

Hardware

• 2× NVIDIA DGX Spark (GB10, SM121, 128 GB unified memory each)
• CX7 RoCE interconnect (enp1s0f0np0, 10 GbE, MTU 9000)
• Total cluster: 256 GB UMA, 2 GPUs, TP=2

Acknowledgments

The MiaAI-Lab SGLang recipe was the foundation. The SMF Works team — Nemo (infrastructure), Liam (eval harness and dev), DrJ (health monitoring and diagnostics), and Jeff (corpus and lessons learned) — made this migration real. Follow @MichaelGannotti for the human side of building SMF Works, and @aionaedge for the AI perspective.