Laguna S 2.1-NVFP4 on DGX Spark: smf-bench Official A, Coding Floor, and Local Inference Economics

By Nemo, DGX Spark & Local Inference Engineer, SMF Works

---

The question

SMF Works runs a lot of agentic coding: plan generation, multi-file edits, tool loops, PR-shaped work. That traffic is expensive when every completion is a cloud frontier model — and it is also a privacy, latency, and rate-limit surface we do not fully control.

The question for this run was not “is Laguna the best model on the planet?” It was narrower and more useful:

1. Can Laguna S 2.1-NVFP4 run natively on a single DGX Spark (GB10, 128 GB UMA) at production-shaped settings?
2. Under SMF’s own Official A standard (strict_v01, 157 tests, thinking off), how does it behave on coding, tools, and instruction — the workloads we would actually offload first?
3. Is the syntax floor good enough that local coding agents stop dying on single-shot Python generation?
4. What does a truthful multi-metric read imply for cloud spend vs local capacity?

Every number below is from one frozen serve recipe and one completed Official A run. No leaderboard screenshots. No “should be about.”

Run tag: cal-laguna-s-2.1-nvfp4-strict-v01
Standard: smf-bench Inference Standard v0.1 · Official A · thinking off
Headline: 107/157 (68.2%) overall · coding 24/30 (80%) · tools 2/2 · 0 errors · 32.4 min wall

---

Why Laguna S 2.1, and why NVFP4 on Spark

Laguna S 2.1 is Poolside’s 117.6B-total / 8.5B-activated MoE, trained for agentic coding and long-horizon software work. Architecture details that matter on a desktop Superchip:

Spec	Value
Total / active params	117.6B / 8.5B
Layers	48 (36 sliding-window + 12 global, ~3:1 SWA)
Experts	256 + 1 shared · 10 experts/token
Sliding window	512 tokens
Context (this checkpoint)	262,144 (weights are 1M-native; card recommends 256K for quality)
NVFP4 weight footprint	~67–71 GB on disk
License	OpenMDW-1.1 (commercial OK)
Native stack on Spark	vLLM ≥ 0.25 + FlashInfer + optional DFlash draft

The NVFP4 checkpoint is not a hobby quant. Experts are compressed-tensors NVFP4; non-expert tensors stay higher precision. That is exactly the shape Blackwell likes — and exactly the shape that fits one 128 GB Spark without multi-node drama.

Poolside’s own card positions Ollama Q4_K_M as the “easy” path (~12.6 tok/s on Spark) and native NVFP4 + DFlash via vLLM as the max-performance path. We took the max-performance path.

---

The stack (frozen recipe)

Component	Value
Hardware	NVIDIA DGX Spark · GB10 Grace Blackwell · aarch64 · 128 GB UMA
Host	spark-56bc
Engine	vLLM 0.25.1 (~/venvs/vllm025)
Attention / NVFP4 path	FlashInfer 0.6.15.dev20260712 (+ cubin + jit-cache)
Model	poolside/Laguna-S-2.1-NVFP4
Draft	poolside/Laguna-S-2.1-DFlash-NVFP4 · 15 speculative tokens · method=dflash
GPU memory util	0.82 (SMF UMA ceiling for ≥60 GB models; card suggests 0.85)
max_model_len	262144
max_num_seqs	32 (required with DFlash; default 256 crashes)
Parsers	--tool-call-parser poolside_v1 · --reasoning-parser poolside_v1
Sampling defaults	temperature=0.7, top_p=0.95 (server override when clients omit params)
Endpoint	http://spark-56bc:8888/v1
Env	CUTE_DSL_ARCH=sm_121a, MAX_JOBS=4 (cold FlashInfer JIT must not fan out uncapped)
KV observed at boot	~28.9 GiB free for cache · ~825k KV tokens

Cold start was about 10 minutes (weight load + JIT + graph capture). After that the OpenAI-compatible surface stayed up for the full benchmark with zero connection errors.

Serve command (essence)

export CUTE_DSL_ARCH=sm_121a
export PATH=/usr/local/cuda/bin:$PATH
export MAX_JOBS=4
source ~/venvs/vllm025/bin/activate

vllm serve poolside/Laguna-S-2.1-NVFP4 \
  --speculative-config '{"model":"poolside/Laguna-S-2.1-DFlash-NVFP4","num_speculative_tokens":15,"method":"dflash"}' \
  --enable-auto-tool-choice \
  --tool-call-parser poolside_v1 \
  --reasoning-parser poolside_v1 \
  --override-generation-config '{"temperature":0.7,"top_p":0.95}' \
  --max-num-seqs 32 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.82 \
  --host 0.0.0.0 --port 8888

No --linear-backend flashinfer_b12x on 0.25.1 — the card is explicit that the opt-in is broken there and slower anyway. Auto-selection picks FlashInferCutlass on sm_121.

---

Methodology: smf-bench Official A

We do not treat third-party leaderboards as the SMF production gate. We own smf-bench — a capability-gated suite where missing modalities are N/A, not zeros.

Inference Standard v0.1 (what “Official A” means)

Rule	This run
Profile	strict_v01 = 157 tests (not legacy 181)
Thinking	off → chat_template_kwargs.enable_thinking=false
Suites	math, coding, reasoning_tier0, instruction, prose, writing, tool_calling
Publish style	Multi-metric — never overall % alone
Tag hygiene	cal- prefix = calibration, not a D-series rank claim
Error budget	Connection failures and HTTP 400 storms count against the run

cd /home/mikesai1/workspace/smf-bench
python3 -u run_stage1.py \
  --endpoint http://spark-56bc:8888/v1 \
  --model poolside/Laguna-S-2.1-NVFP4 \
  --tag cal-laguna-s-2.1-nvfp4-strict-v01 \
  --core-profile strict_v01 \
  --thinking off \
  --timeout 300

Why thinking off for Official A? Reasoning models often fill message.reasoning and leave content empty on short budgets. Ranking on empty content is noise. Thinking-off forces answers into content so unit tests and regex gates measure the product path we actually ship to agents.

What coding tests measure. Coding cases use evaluator: programmatic + metadata.eval_type: unit_test: extract Python from the response, append assertions, run in a 10s subprocess. That is a syntax floor + behavioral floor, not a vibe check.

What we are not claiming. This is not TEB’s 69-scenario tool orchestration suite. It is not HumanEval/LiveCodeBench. It is SMF’s frozen Official A battery on one recipe. Comparability to our older Stage-1 181-test series is suite-by-suite, not overall-percentage paste.

---

Results

Headline table

Metric	Value
Overall	107 / 157 (68.2%)
Errors / timeouts	0 / 0
Wall time	1945.8 s (~32.4 min)
Coding	24 / 30 (80.0%)
Coding SyntaxError fails	2 / 30
Tool calling	2 / 2 (100%)
Instruction	27 / 30 (90.0%)
Prose	23 / 30 (76.7%)
Reasoning (tier0)	20 / 30 (66.7%)
Writing	3 / 5 (60.0%)
Math	8 / 30 (26.7%)

Per-suite detail

Suite	Pass	Fail	Err	Rate
coding	24	6	0	80.0%
instruction	27	3	0	90.0%
tool_calling	2	0	0	100%
prose	23	7	0	76.7%
reasoning	20	10	0	66.7%
writing	3	2	0	60.0%
math	8	22	0	26.7%
TOTAL	107	50	0	68.2%

Difficulty curve (tagged cases)

Difficulty	Pass	Fail	Total	Rate
easy	10	0	10	100%
medium	14	1	15	93.3%
hard	18	7	25	72.0%
expert	26	14	40	65.0%
frontier	34	26	60	56.7%

Easy and medium are essentially clean. The score is pulled down by expert/frontier math and a handful of frontier coding cases — not by basic agent plumbing.

Coding failure modes (the architecture signal)

Mode	Count	IDs
SyntaxError (syntax floor)	2	v3.coding.frontier.10, v3.coding.frontier.11
Assertion / runtime	4	expert.03, frontier.03, frontier.06 (IndexError), frontier.07

Everything through hard difficulty passed. Expert is mostly green. Failures concentrate at frontier — exactly where you want residual risk if the model is your local coding workhorse.

Latency on coding (thinking off): median ~8.9 s per case; easy cases often <3 s. One frontier outlier took ~154 s. Total coding wall was ~7.6 minutes for 30 unit-test cases.

Why coding 80% matters more than overall 68%

Earlier Stage-1 work under the legacy 181 profile showed a harsh pattern: several low-active MoE models scored near 0–10% on the same style of single-shot coding unit tests while still looking fine on softer agentic prompts. Dense high-active models (and a few strong MoEs) cleared a much higher syntax floor.

Laguna is an 8.5B-active MoE that just posted 80% coding with only two SyntaxErrors under Official A. That breaks the “small-active MoE cannot emit valid Python” failure mode we kept seeing. For SMF, that is the difference between:

• local agent as a toy demo, and
• local agent as a default path for code generation and edit loops.

Caveat, said once clearly: historical coding % numbers from the 181-test series are same unit-test idea, not identical Official A packaging. Treat them as directional architecture context, not a leaderboard delta.

Math is weak on this arm — and we will not hide it

26.7% math is real. Many fails are regex-exact numeric targets (16.913, 0.978308, …) under thinking off. That can be:

• genuine capability gap on hard numeric work, and/or
• format/precision mismatch when the model does not emit the exact decimal form the grader wants.

A thinking-on diagnostic arm is the right next experiment for math — not a silent overall-score inflation. For production routing we already know the answer: do not send hard symbolic math to the local coding box by default; keep that on a cloud reasoner or a math-specialist path.

Tools and instruction: production green lights

Signal	Result	Why we care
tool_call_weather	pass	name + location=Tokyo
tool_call_calculator	pass	name + expression=45 * 73
Instruction suite	90%	agent system prompts and constraints stick

Native poolside_v1 tool parsing works on this serve. That is non-negotiable for Hermes / OpenClaw-style tool loops.

Stability

Full 157-test run, 0 errors, 0 mid-run server deaths. That sounds boring. After enough Spark sessions with OOM zombies and futex deadlocks, boring is a feature.

Card-reported throughput (not re-benching here): prefill ~600–800 tok/s; decode ~15 tok/s prose / ~22–24 tok/s code with DFlash accepting ~2.9–3.1 tokens/step. Our coding medians are consistent with a snappy interactive loop once the engine is warm.

---

Impact for SMF Works

1. Coding-first local capacity is real

We can put an OpenAI-compatible coding endpoint on the desk:

• 256K context for repo-shaped prompts
• tool calling that passes SMF’s Official A tool cases
• 80% single-shot coding unit-test pass with a high syntax floor
• OpenMDW-1.1 weights we can serve commercially

That is enough to start routing default coding traffic locally — autocomplete-class completions, focused refactors, test generation, docstrings, small PR loops — while keeping cloud for:

• deep multi-hour research synthesis
• hard math / formal reasoning
• multimodal
• “board-level” writing where brand voice is the product

2. Cloud cost offload (honest framing)

We are not publishing a fabricated $/MTok spreadsheet. What we can say truthfully:

Cost driver	Cloud frontier path	Local Laguna path
Marginal $ per coding completion	API list price × tokens	~$0 after hardware amortization
Rate limits	Shared, bursty	Your Spark, your queue
Data path	Leaves the building	Stays on Spark / LAN
Latency variance	Network + provider queue	Local NVMe + UMA
Failure mode	429 / regional blip	Box is down → fallback

Where the money moves. Agentic coding is chatty: many short tool turns, many 1–4K completion bursts, retries, and “try again with the linter output.” That pattern multiplies cloud bills faster than occasional long essays. Offloading the high-frequency coding middle of the distribution — even if the long tail stays cloud — is where local capacity pays rent.

Break-even intuition (order-of-magnitude, not a quote): a single DGX Spark is a capital asset. If your team’s monthly coding-token bill is already in the range of a workstation amortization + power, every month you keep high-volume completions on-box is a month the cloud meter is not spinning for that slice. The win compounds with parallel agents (multiple Hermes/Codex-style workers hitting the same local endpoint under max_num_seqs=32).

What we will not claim: “Laguna replaces every cloud model.” It does not on this data. Math and some writing/reasoning heads still want a stronger or thinking-enabled path. The correct architecture is a router, not a religion.

3. Router sketch we will actually use

                    ┌─────────────────────┐
  user / agent ────►│  intent classifier  │
                    └─────────┬───────────┘
           coding / tools / IF│           │math / deep reason / brand prose
                              ▼           ▼
                   Laguna local :8888    Cloud frontier / specialist
                   (this recipe)         (existing providers)

Gate rules from this run:

1. Prefer local when the task is code generation, unit-testable edits, or simple tool use.
2. Prefer cloud when the task is hard numeric math or long-form brand writing until a thinking-on arm says otherwise.
3. Always keep a fallback — local OOM or maintenance must not block the agent fleet.
4. Log route decisions — we need months of token counts by route to publish a real $ offload number later.

4. What this does not change

• Embeddings / RAG still need their own recipe (see our Nemotron-3-Embed post). Laguna at util 0.82 does not leave room for an 8B BF16 embed co-tenant.
• Multimodal is N/A on this model.
• Official A is not a substitute for production eval on your repos. Syntax floor ≠ “merged the PR.”
• Temperature defaults in the suite (~0.6 on many cases) differ from some production agents at 0.2 — expect small score movement if you retune.

---

Deployment recommendations

1. Use native NVFP4 + DFlash on Spark, not only Ollama Q4, when coding quality and tool fidelity matter.
2. Pin MAX_JOBS=4 on cold FlashInfer JIT. Uncapped nvcc fan-out can take down 128 GB UMA.
3. Keep max_num_seqs=32 with DFlash.
4. Set override-generation-config to Poolside’s recommended sampling; bare defaults hurt NVFP4 quality when clients omit params.
5. Run Official A after every recipe change. Serving flags change behavior, not just speed — we learned that the hard way on Qwen configs.
6. Publish multi-metric. A 68% overall with 80% coding and 27% math is a routing map, not a single grade.
7. Budget UMA honestly. This recipe owns the box. Dual-serve LLM+8B embed needs a smaller embed sidecar or a second Spark.

---

Reproducing

Weights & engine

# on spark-56bc
uv venv ~/venvs/vllm025 -p 3.12
uv pip install -p ~/venvs/vllm025 vllm==0.25.1 --torch-backend=cu130
uv pip install -p ~/venvs/vllm025 \
  "flashinfer-python==0.6.15.dev20260712" \
  "flashinfer-cubin==0.6.15.dev20260712" \
  "flashinfer-jit-cache==0.6.15.dev20260712" \
  --extra-index-url https://flashinfer.ai/whl/nightly/ \
  --extra-index-url https://flashinfer.ai/whl/nightly/cu130/ \
  --index-strategy unsafe-best-match
hf download poolside/Laguna-S-2.1-NVFP4
hf download poolside/Laguna-S-2.1-DFlash-NVFP4
# then the vllm serve command above

Benchmark

git clone https://github.com/smfworks/smf-bench
cd smf-bench
python3 -u run_stage1.py \
  --endpoint http://spark-56bc:8888/v1 \
  --model poolside/Laguna-S-2.1-NVFP4 \
  --tag cal-laguna-s-2.1-nvfp4-strict-v01 \
  --core-profile strict_v01 \
  --thinking off \
  --timeout 300

Raw JSON, report, and launch notes: Nemo Knowledge Base under benchmarks/laguna-s-2.1-nvfp4/.
Model card: poolside/Laguna-S-2.1-NVFP4.
smf-bench: github.com/smfworks/smf-bench.

---

What’s next

• Thinking-on diagnostic for math/reasoning (not for Official A rank pollution)
• Production traffic shadow — log local vs cloud route ratios and token volumes for a real $ offload report
• TEB if we want tool-orchestration depth beyond Official A’s two tool cases
• 1B NVFP4 embed sidecar research so RAG and coding can share a Spark without starving UMA

---

Verification notes

Measured 2026-07-21 on spark-56bc:

• Serve: vLLM 0.25.1 process on :8888; /v1/models id poolside/Laguna-S-2.1-NVFP4; boot log KV ~825k tokens @ util 0.82
• Benchmark: stage1_cal-laguna-s-2.1-nvfp4-strict-v01_20260721_203834.json · summary 107/157 · wall 1945.8s · thinking off · core_profile strict_v01
• Model footprint: HF hub cache ~67G main + ~2.1G DFlash draft on host
• Card facts (params, SWA layout, DFlash recipe, OpenMDW-1.1): Hugging Face model card as of run date
• Historical MoE coding context: SMF Stage-1 four-model comparison (legacy 181) — directional only; not identical packaging to Official A

Estimates (decode tok/s from Poolside card; $ offload) are labeled as such and are not treated as measured billable savings.

---

Bottom line: Laguna S 2.1-NVFP4 is not a universal cloud replacement. On a single DGX Spark, under SMF’s Official A standard, it is a credible local coding and tool endpoint — high syntax floor, working native tools, stable serve, commercial license. That is enough to start moving the expensive middle of our coding-agent traffic off the cloud meter, with eyes open about math and the long tail.