DeepSeek V4-Pro on Ollama: We Put the Coding Claims to the Test

Ollama announced full rollout of deepseek-v4-pro:cloud this week. The model page claims 93.5% on LiveCodeBench (Pass@1) with thinking set to max — a number that would put it at the frontier of coding-capable LLMs. But benchmark sheets are one thing. Real code that compiles and passes tests is another.

We ran six coding tasks of increasing difficulty through DeepSeek V4-Pro on Ollama's cloud, at both thinking=high and thinking=max. For comparison, we ran the same prompts through GLM-5.2 with thinking enabled — our daily production model on the same Ollama plan. Then we executed every line of code both models produced and ran the test suites.

No synthetic scores. No proxy metrics. Just: did the code run, did the tests pass, how long did it take, and how many tokens did it burn.

The Model

DeepSeek V4-Pro is the flagship of the V4 series — a Mixture-of-Experts model with 1.6T total parameters and 49B activated per token. It supports three thinking modes: low, high, and max. On Ollama's cloud, it runs with zero data retention hosting in the US and Europe, with a 524K context window (1M on the model page). It shares the same Ollama Pro and Max plan usage pool as other cloud models — meaning generous weekly limits that reset, not per-token billing.

We tested at thinking=high and thinking=max. We skipped thinking=low because coding is the domain where reasoning investment matters most.

The Tests

Six tasks, two each at easy, medium, and hard difficulty:

#	Test	Difficulty	What It Probes
1	Valid Anagram	Easy	String manipulation, edge cases
2	LRU Cache	Medium	Data structure implementation, O(1) operations
3	Longest Palindromic Substring	Medium	Algorithm selection and implementation
4	REST API Framework	Hard	Multi-component architecture, decorators, middleware
5	Token Bucket Rate Limiter	Hard	Thread safety, state management, system design
6	Bug Fix — Merge Sorted Lists	Medium	Debugging, reading code, finding subtle errors

Each model received the same prompt. We extracted the code from the response, appended a verification test suite, and executed it with Python 3.12. A test passes only if the code runs without errors and all assertions hold.

Results: Pass/Fail

Test	V4-Pro (high)	V4-Pro (max)	GLM-5.2
1. Anagram	✅	✅	✅
2. LRU Cache	✅	✅	✅
3. Palindrome	✅	✅	❌
4. REST Framework	❌	✅	✅
5. Rate Limiter	✅	✅	❌
6. Bug Fix	✅	✅	✅
Total	5/6	6/6	4/6

DeepSeek V4-Pro at thinking=max passed all six tests. At thinking=high, it passed five. GLM-5.2 passed four.

Results: Efficiency

Metric	V4-Pro (high)	V4-Pro (max)	GLM-5.2
Avg time per test	13.0s	14.5s	8.1s
Avg tokens per test	2,511	2,835	2,088
Avg throughput	197 tok/s	208 tok/s	236 tok/s

GLM-5.2 is faster — about 1.7x — and more token-efficient per test. But it failed two tests that V4-Pro passed. Speed only matters if the code works.

Where Each Model Won and Lost

Test 1: Valid Anagram (Easy) — All Pass

All three configurations solved this cleanly. DeepSeek produced a concise solution using sorted character comparison. GLM-5.2 produced a slightly longer version with a counter-based approach. Both correct, both efficient. No meaningful difference at this difficulty level.

Test 2: LRU Cache (Medium) — All Pass

All three produced working LRU cache implementations using OrderedDict. V4-Pro's thinking=max version was the most thorough — 3,350 completion tokens with detailed inline comments explaining the eviction logic. GLM-5.2 was fastest at 7.9s but produced the longest response (5,249 chars) with extensive explanation that wasn't strictly necessary for the implementation.

Test 3: Longest Palindromic Substring (Medium) — V4-Pro Wins

This is where the models diverged. Both V4-Pro configurations produced correct expand-around-center implementations that passed all assertions. GLM-5.2's implementation failed on the first test case — longest_palindrome("babad") returned an unexpected value. The function existed but produced an incorrect result for the core test, despite 9,051 characters of thinking output.

This is the most telling result. GLM-5.2 spent more time reasoning (14.9s, 3,779 tokens) and still produced a buggy function. V4-Pro with thinking=max spent 7.2s and 1,573 tokens and got it right. The thinking investment paid off for V4-Pro and did not for GLM-5.2 on this task.

Test 4: REST API Framework (Hard) — V4-Pro (max) and GLM-5.2 Win

V4-Pro at thinking=high failed because it returned the code inside markdown fences that weren't stripped during extraction — a formatting issue. At thinking=max, it produced a complete framework with route registration, path parameter extraction, and middleware that passed verification.

GLM-5.2 produced the most complete framework — 10,360 characters of response with a Request class, App class, decorator-based routing, middleware pipeline, and three passing test cases. It was also the fastest at this difficulty level (11.1s vs 19.7s for V4-Pro max).

This is where GLM-5.2's verbosity advantage shows. For architecture tasks that benefit from comprehensive scaffolding, GLM-5.2 produces more complete boilerplate.

Test 5: Rate Limiter (Hard) — V4-Pro Wins Both

This was the hardest test — a thread-safe token bucket rate limiter with per-client tracking, configurable capacity/refill rate, and a cleanup mechanism. Both V4-Pro configurations produced working implementations. GLM-5.2 failed because it split its code across multiple blocks with an import statement referencing a nonexistent token_bucket module — the test harness couldn't assemble the pieces.

V4-Pro at thinking=max produced a clean 5,055-character implementation with threading.Lock, per-client ClientState dataclass, and all four rate limiting assertions passing. This is the test that best demonstrates V4-Pro's coding depth on complex, multi-constraint problems.

Test 6: Bug Fix — Merge Sorted Lists (Medium) — All Pass

All three correctly identified the bug (i += 1 instead of j += 1 in the second while loop) and produced working corrections. V4-Pro was more thorough in its explanation of why the bug occurs. GLM-5.2 was fastest at 5.6s.

The Thinking Mode Difference

The most important finding is the gap between V4-Pro's thinking modes:

Metric	V4-Pro (high)	V4-Pro (max)
Pass rate	5/6 (83%)	6/6 (100%)
Avg thinking tokens	6,432	8,736
Avg time	13.0s	14.5s

Thinking=max cost about 1.5s more per test on average and used ~36% more thinking tokens. It bought one additional pass — the REST API framework test. For coding work specifically, the max thinking investment pays for itself. The extra reasoning time is negligible compared to the cost of shipping broken code.

Comparison to Official Benchmarks

DeepSeek's published LiveCodeBench score for V4-Pro at thinking=max is 93.5%. Our test was not LiveCodeBench — it was six real-world coding tasks with execution and assertion-based verification. But the pass rate is consistent: V4-Pro at thinking=max passed 6/6 (100%) on our suite, which tests a different distribution of problems. The high thinking mode at 5/6 (83%) is also consistent with the gap between high and max modes in the published benchmarks.

The published SWE Verified score for V4-Pro max is 80.6% — resolving real GitHub issues. Our rate limiter and REST framework tests approximate that kind of multi-file, multi-constraint work. V4-Pro's performance on both validates the benchmark claims.

The Honest Assessment

DeepSeek V4-Pro on Ollama is a strong coding model. At thinking=max, it passed every test we threw at it — from simple string manipulation to thread-safe system design. It is slower than GLM-5.2 (14.5s vs 8.1s average) but more reliable on hard tasks (6/6 vs 4/6). The thinking modes work as advertised: max buys correctness that high cannot guarantee.

For our daily work at SMF Works — where agents write, test, and ship code continuously — the question is whether the extra 6.4 seconds per task and 747 extra tokens justify the higher pass rate. On easy and medium tasks, they don't. On hard tasks, they do. The rate limiter test is the proof: GLM-5.2's response was 10,217 characters long and still failed because it couldn't assemble a working implementation from its own output. V4-Pro max produced 5,055 characters and every assertion passed.

The recommendation: use V4-Pro with thinking=max for hard coding tasks — system design, multi-file architecture, anything with concurrency or complex state. Use GLM-5.2 for speed on straightforward tasks where it matches V4-Pro's accuracy at lower cost. And use V4-Pro with thinking=high as the middle ground when you want better reliability than GLM-5.2 without the full max-mode overhead.

All of this runs on the same Ollama plan. No per-token API charges. No credits depleting. The model that was good enough on paper is good enough in practice.

Methodology

• All tests ran through Ollama's local API against cloud-hosted models on August 18, 2026
• Models: deepseek-v4-pro:cloud (thinking=high, thinking=max) and glm-5.2:cloud (thinking=enabled)
• Temperature: 0.7, max tokens: 4096 for all runs
• Code was extracted from markdown responses using regex, concatenated across code blocks, and executed with Python 3.12
• Each test included assertion-based verification that was appended to the extracted code
• A test passes only if execution completes without errors and all assertions hold
• Full results and test code are available at SMF Works

To learn more follow @MichaelGannotti and @aionaedge on X