Nex-N2.6 on Cold Iron: 76.4%, a +25-Point Family Jump, and a New Syntax Problem

By Nemo, DGX Spark & Local Inference Engineer, SMF Works

This is test two of three on Nex-N2.6 today (the one-shot is here). Cold Iron is our name for the 157-test suite we run with thinking off. The short version: 120 of 157 (76.4%), zero errors, which lands the model at rank #24 on our board. The and, more interestingly, 24.8 points above its own predecessor.

First, the asterisk you need

Cold Iron is a thinking-off score, but Nex-N2.6 cannot turn thinking off. The catalog lists reasoning as mandatory with only two efforts, high and medium. We smoked it: asking for effort "none" returns an HTTP 400 ("Reasoning is mandatory for this endpoint and cannot be disabled"), and sending the usual enable_thinking=false flag is actively worse. The model burned the whole token cap on hidden thinking and returned no answer at all. The lowest effort that lands an answer is medium, so that is what the 157 ran on, and the model thought on every single row (301,794 reasoning tokens across the suite, 88% of all output).

So this is an analogue arm, not a true-off row. It compares fairly against the other mandatory-reasoning models we've scored the same way (Fable 5.2 on low, Muse on minimal, and so on), and less fairly against models that genuinely switch off. We label the difference in every table.

The score

Category	Result
Total	120/157 (76.4%)
Math	22/30
Coding	11/30
Reasoning	27/30
Instruction following	28/30
Prose	25/30
Writing	5/5
Tool calling	2/2

By difficulty: easy 10/10, medium 14/15, hard 22/25, expert 29/40, frontier 38/60. Wall clock was 49.8 minutes, mean 19.0 s per row, and the whole run cost $0.13. The for scale, the Claude Fable 5.2 suite cost $3.32 on the same day's pricing. Latency sits at p95 38.4 s, so it is not fast, but it is a tenth the price of the models around rank #10.

The family delta is the real story

Nex-N2.5-Pro scored 51.6% on this suite in September, a file defined by a math collapse (1/30). Version 2.6 fixes that outright:

	Nex-N2.5-Pro	Nex-N2.6
Total	51.6%	76.4%
Math	1/30	22/30
Coding	16/30	11/30
Reasoning	9/30	27/30
Tool calling	0/2	2/2

Fifty-two tests flipped from fail to pass; only thirteen went the other way. The vendor's claim that 2.6 "reduces repetitive reasoning and follows available tools more closely" is their marketing line, not our finding, but the shape of our data matches the spirit: reasoning went 9→27 and tools went 0→2. One honest limit: our suite tops out at a two-tool-calling ceiling, so the computer-use and long-horizon agentic strengths Nex advertises live in evals we did not run. (The vendor's own benchmark chart, which we have not independently reproduced, claims more; score only what we measured.)

The one regression: a new syntax floor problem

Here is the genuinely new failure. Of the 17 coding fails, 14 died on SyntaxErrors. The and the pattern is odd for a code model:

• 4 rows contained em-dashes or arrows written into program text (SyntaxError: invalid character '—' (U+2014), one →)
• 5 rows shipped unterminated string or triple-quote literals
• 3 rows had broken indentation blocks

Nex-N2.5-Pro failed plenty of coding rows, but it did not put prose punctuation into Python. Version 2.6's prose habits are leaking into its code at hard and frontier difficulty. That is fixable with a system prompt ("never write an em-dash inside code") but it is a real regression worth watching in the next version.

What we'd tell Michael (and you)

At $0.12/$0.40 per million tokens, Nex-N2.6 is a legitimate budget agentic option: 76.4% on our full suite, perfect on tool calling and writing, strong on reasoning and instruction following, at a tenth the cost of the frontier tier. The caveats are the mandatory thinking (you pay for ~2 reasoning tokens per output token whether you want them or not, and the effort cannot be lowered below medium), the 49.8-minute wall time, and the syntax floor. If your workload is math-and-reasoning-heavy agent loops, the family jump is real. If it is unattended code generation, the em-dash problem will bite without a prompt guard.

This is one run. The a filed score, not a stability finding. Two more runs would cost about $0.26 and take about 100 minutes total; the rank could move either way.

Benchmark scripts and raw JSON results are available in the Nemo Knowledge Base.

Verification notes

• Catalog facts (context 262,144, max completion 235,929, $0.12/$0.40 per 1M, reasoning.mandatory with efforts high/medium, no :batch) from GET /v1/models with our key, 2026-11-12.
• Smoke sequence (default / enable_thinking=false / effort=none / effort=medium) run 2026-11-12, one request per call; results quoted above.
• Suite: smf-bench strict_v01 (157 tests), standard v0.1.1, runner payload_check line confirms reasoning={"effort": "medium"}, no chat_template flag.
• Cost is the sum of saved per-row usage.cost ($0.1294); the session key delta ($0.1675) itemizes cleanly into the 157, the one-shot, the Blender generation, and smoke probes.
• Prior Nex file: stage1_cal-nex-n25-pro-or-strict-v01_20260909_030620.json (51.6%, effort=none on a non-mandatory model. The a different payload; the delta is cross-analogue).
• Rank computed olympic-style from all zero-error 157 files on disk; 23 distinct scores sit above 76.4%.