Showing 20 of 20
Beyond the Leaderboard: Claude Opus 4.8 Fast
Anthropic's speed-optimized flagship delivered the best code generation in the series — and the worst instruction-following score.
Beyond the Leaderboard: DeepSeek-V4-Pro
A thinking model that negotiates constraints and writes clean code — but carries an 8× speed penalty versus Kimi.
Beyond the Leaderboard: Gemini 2.5 Pro
Google's flagship delivered the best coding benchmark in the series — then timed out on the 10,000-word long-context test.
Beyond the Leaderboard: Gemma 4 12B Local vs. 31B Cloud
Same model family, two different operational universes: a 12B local Ollama run that hung on complex prompts, and a 31B cloud endpoint that completed the full suite.
Beyond the Leaderboard: Gemma 4 26B
Google's MoE flagship won the series with the fastest time-to-first-token and strong precision — but still over-produces on word counts.
Beyond the Leaderboard: Gemma4:e4b on Local Hardware
A 9.6GB local model outperformed cloud titans on code generation and complex reasoning — but crashed on structured output.
Beyond the Leaderboard: GPT-5.5 on OpenRouter
OpenAI's newest flagship matched Claude Opus on overall score and led the series on JSON — but hallucinated a fictional G7 summit.
Beyond the Leaderboard: Kimi K2.6
We put our daily-driver model through a 15-test production gauntlet. Fast, reliable, and great at JSON — but precision tasks are not its strength.
Beyond the Leaderboard: MiniMax-M3
MiniMax-M3 matched DeepSeek on instruction following and produced clean code — but hallucinated detailed recent events with confidence.
Beyond the Leaderboard: Multimodal MiniMax M3
MiniMax M3 passed physics reasoning but hallucinated a red light in an autonomous-driving frame — a safety-critical failure pattern.
Beyond the Leaderboard: Nemotron 3 Ultra
NVIDIA's 550B-parameter free-tier model solved the logic puzzle that defeated Kimi and DeepSeek — then over-thought the constraint puzzle into paralysis.
Beyond the Leaderboard: Nex N2 Pro
A surprise free-tier model matched leaders on JSON and code execution — then hallucinated an entire article summary.
Beyond the Leaderboard: One Week of Testing
Ten models, fifteen tests each, 150 production evaluations. The brutal truth: there is no best model, only a best-fit model.
Beyond the Leaderboard: Qwen 3.7-Max
Alibaba's reasoning flagship scored near the top of the series — but took 31 seconds on average to produce its first token.
Beyond the Leaderboard: StepFun Step-3.7-Flash
A reasoning-first model that outputs its chain of thought through a separate API field. The harness saw partial answers, but the raw reasoning was coherent.
Beyond the Leaderboard: The Harness Is Open
How SMF Works tests AI models in production: the 15-test methodology, the rubrics, and the open-source harness behind the series.
Local LLM Shootout on a Mini PC
We tested Llama 3.1, Qwen 2.5, and Mistral on a sub-$400 mini PC to find the best local coding assistant.
Ollama Cloud Model Stack Benchmark 2026
We benchmarked 5 Ollama Cloud models under parallel and sequential load to fix a cron collision that cost $240 in a single week.
Running Claude Code on a Steam Deck
An experiment in portable agent development: Anthropic's terminal agent on Valve's handheld.
Self-Hosted Open WebUI for a 3-Person Team
We ran Open WebUI as an internal chat and RAG workspace for a week. Here's the setup, cost, and friction.