Designing Agent Harnesses for Local Models: The Perplexity Portable Computer Lesson

The problem

Most agent harnesses are designed for frontier models. They assume a model that can absorb enormous contexts, navigate sprawling tool surfaces, and plan over long horizons. When you point those same harnesses at a local 27B model, the model buckles. It loses track of the task after 100K tokens. It picks the wrong tool from a list of 50. It forgets instructions that a frontier model would hold effortlessly.

This is not a model problem. It is a harness problem. And Perplexity's August 2026 launch of Portable Computer — a fully local agent platform running on Nvidia DGX Spark and RTX GPUs — provides a concrete blueprint for fixing it.

The core insight

Perplexity published a research paper alongside the launch arguing that effective local agents require the model and the agent harness to be designed together. The harness must be deliberately minimal because small models cannot handle the same cognitive load as frontier models.

Their empirical findings:

• Models like Qwen 3.8 27B advertise 260,000-token context windows but begin to struggle beyond 100,000 tokens
• A minimal harness — succinct system prompt, small tool set, on-demand skill loading — dramatically improves local model performance
• Converting token-hungry MCP servers into compact CLI tools reduces context pollution
• Self-verification hooks that monitor task health catch errors the model would otherwise miss
• OS-level sandboxing must be always-on; if unavailable, the harness disables itself rather than running unprotected

The five design principles

1. Keep the system prompt short

Frontier-model harnesses often have sprawling system prompts with dozens of rules, examples, and tool descriptions. A 27B model's attention budget is smaller. Every token in the system prompt competes with task context for attention.

Actionable: Limit the system prompt to role definition, core constraints, and the current task. Move everything else to on-demand skills that load only when needed.

2. Minimize the tool surface

A frontier model can choose correctly from 50 tools. A 27B model cannot. Tool selection accuracy degrades as the tool list grows, and the tool definitions themselves consume context.

Actionable: Expose 5-10 core tools at most. Convert specialized tools into CLI commands or skills that load on demand. Perplexity converted Gmail and GitHub connectors from MCP servers into compact command-line tools, cutting their context footprint by 80%+.

3. Load capabilities as skills, not permanent context

Instead of putting every capability in the system prompt, structure them as skills that the agent loads when it needs them and unloads when it doesn't. This keeps the active context lean while preserving access to a rich capability set.

Actionable: Use a skill system (like Hermes skills or Claude Code skills) where each skill is a self-contained markdown file. The agent loads the relevant skill, uses it, and the context is freed when the skill is no longer active.

4. Add self-verification hooks

Local models make mistakes that frontier models don't. A self-verification hook checks the agent's output against expected criteria before acting on it. Perplexity's harness runs a PII check before any cloud escalation and a self-verification check after each significant step.

Actionable: After each tool call or significant step, run a lightweight verification: "Did the output match the expected format? Did the file actually get written? Does the code compile?" These checks catch the kind of errors that local models make frequently.

5. Enforce always-on sandboxing

Open-source harnesses often run commands with the user's full permissions by default. This is dangerous with any model, but especially with smaller models that may hallucinate commands. Perplexity's harness disables itself if the sandbox is unavailable — a hard stop, not a warning.

Actionable: Run all agent-executed code in an isolated sandbox (Docker container, Firecracker microVM, or equivalent). If the sandbox cannot be established, the harness should refuse to execute, not fall back to running with user permissions.

The cost argument

On Terminal-Bench 2.1, Perplexity's fully local Qwen 3.8 27B scored 59.6% at essentially zero marginal cost. Escalating to a Claude Opus 5 "advisor" in the cloud raised the score to 73.0% at $0.415 per task. Running the frontier model alone scored 82.4% at $0.65 per task.

The escalation pattern recovered roughly three-fifths of the gap to frontier performance at about two-thirds of the cost. The user decides when the trade is worth making.

Actionable: Design your harness with a local-first, cloud-escalation pattern. The local model handles 80% of tasks for free. The cloud model is invoked only when the local model hits its limits, and the user (or a confidence threshold) decides when to escalate.

Building your own

If you're building a local-first agent harness:

1. Start with a minimal system prompt — under 500 tokens
2. Expose 5-10 core tools — file read/write, terminal, web search, and 2-3 domain-specific tools
3. Convert MCP servers to CLI tools where possible — reduces context footprint
4. Implement skill loading — capabilities load and unload on demand
5. Add self-verification — check outputs before acting
6. Enforce sandboxing — always-on, hard stop if unavailable
7. Design for escalation — local-first, cloud when needed, user-controlled

Why this matters

The shift from cloud-only agents to local-first agents is accelerating. Open models are getting better (Granite 4.2, Qwen 3.8, GLM-5.2), hardware is getting more capable (DGX Spark, RTX 5090), and the cost and privacy advantages of local execution are significant. But the harness — not the model — is the bottleneck. A well-designed harness makes a 27B model feel like a frontier model for 80% of tasks. A poorly designed one makes a frontier model feel like a 7B model.

Perplexity's Portable Computer is the proof point. The lesson is not "use Perplexity" — it's "design the harness for the model you're actually running."