From Bilevel Loops to Continuous Verification: Building a Self-Optimizing Skill System

Over the past two weeks, we took three major research threads — Bilevel Autoresearch, SkillOpt, and LLM-as-a-Verifier — and turned them into a working, self-optimizing system for agent skills. This post walks through the lineage, the architecture we built, the non-obvious problems we hit with reasoning models, and how we solved them on the DGX Spark.

The Research Lineage

Three papers form the backbone of this work.

Bilevel Autoresearch (arXiv:2603.23420)

The foundational idea: an outer loop can autonomously discover new mechanisms (not just hyperparameters) for an inner search loop. The authors demonstrated that Level-2 mechanism generation produced 5× better results than standard autoresearch on a GPT pretraining benchmark.

Key insight: the outer loop doesn't just tune — it invents new Python code that gets dynamically loaded into the running system.

SkillOpt (arXiv:2605.23904)

This paper formalizes skill documents themselves as the trainable artifact. Instead of hand-written or one-shot LLM-generated skills, SkillOpt treats the skill as external state and optimizes it with deep-learning-style controls:

• Trajectory batches
• Textual learning rates (bounded add/delete/replace edits)
• Held-out validation gates
• Rejected-edit buffers
• Epoch-wise slow/meta updates

The result is a compact, auditable best_skill.md that can be dropped into any target model or harness.

LLM-as-a-Verifier (arXiv:2607.05391)

The missing piece for reliable optimization: verification as a scaling axis.

Instead of prompting an LLM to output a discrete score (which produces high tie rates), this work computes the expectation over the full distribution of scoring token logits. This unlocks three orthogonal scaling dimensions:

• Score granularity (more tokens → better separation)
• Repeated evaluation (reduces variance)
• Criteria decomposition (reduces prompt bias)

The authors achieve state-of-the-art on Terminal-Bench V2, SWE-Bench Verified, and several robotics/medical benchmarks using this approach.

The Problem We Were Solving

We had a concrete skill we wanted to improve: the Edit Planning Skill inside the Bilevel-Autoresearch article optimization pipeline. The skill is responsible for triaging hypotheses and producing executable edit plans.

The challenge was verification. A naive LLM judge produces noisy, high-variance scores that make it impossible to know whether a candidate edit is genuinely better. Without reliable verification, the outer optimization loop cannot safely accept or reject changes.

Our Approach

We built a complete SkillOpt + LLM-as-a-Verifier prototype with the following components:

1. The Optimization Loop

skillopt_loop.py implements the core SkillOpt loop:

• Rollout: Run the current skill on a training batch of articles
• Reflection: Propose bounded add/delete/replace edits
• Bounded Update: Apply at most Lt edits (textual learning rate)
• Validation Gate: Only accept if the candidate strictly improves the held-out validation score
• Rejected Buffer: Feed failed edits back as negative examples

2. Real Continuous Verification

We replaced the initial MockEvaluator with LLMVerifierEvaluator, which implements the core idea from the LLM-as-a-Verifier paper:

• Calls the DGX Spark vLLM endpoint (spark-56bc:8888)
• Requests logprobs on scoring tokens (1–10)
• Computes the expected value over the distribution
• Supports repeated evaluation and criteria decomposition

3. Handling Reasoning Models

The target model on the DGX Spark (unsloth/Qwen3.6-35B-A3B-NVFP4) is a heavy reasoning model. It frequently returns content: None and places its thinking in the reasoning field.

We solved this with three changes:

• Prompt engineering: Explicit instruction to end with **Final Answer: X**
• Robust parsing: Try logprobs → content → reasoning field
• Increased token budget: Reasoning models need headroom

This allowed us to extract real continuous scores (e.g., 2.33, 1.33) instead of defaulting to 5.0.

4. Harness & Reproducibility

Following the methodology from the Walking Labs Harness Engineering course, we built:

• AGENTS.md — Mission, principles, verification rules, anti-patterns
• feature_list.json — Machine-readable feature tracking with evidence
• skillopt-progress.md — Cross-session state
• Decomposed evaluation criteria (T1–T3, P1–P4) for the article-editing skill

Results So Far

• The prototype successfully ran 6-epoch optimization loops
• Epoch 1 correctly identified and accepted a meaningful improvement (7.80 → 8.30)
• Subsequent epochs correctly rejected candidates offering no additional gain
• Real continuous scoring is now live on the DGX Spark

The system is currently limited by the 60-second execution timeout in our current tool environment, but the architecture is fully validated and ready for longer runs.

Current Architecture (v0.12)

┌─────────────────────────────────────────────────────────────┐
│                    SkillOpt Loop                            │
│  (propose → evaluate → validate → accept/reject)            │
└───────────────────────┬─────────────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────────────────┐
│              LLMVerifierEvaluator                           │
│  - Criteria decomposition                                   │
│  - Repeated evaluation                                      │
│  - Logit-expectation scoring                                │
└───────────────────────┬─────────────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────────────────┐
│         DGX Spark vLLM (spark-56bc:8888)                    │
│  Model: unsloth/Qwen3.6-35B-A3B-NVFP4                       │
│  (with reasoning field fallback)                            │
└─────────────────────────────────────────────────────────────┘

Next Steps

When we have longer execution windows available, we plan to:

1. Run full 5-epoch optimizations with the real verifier
2. Measure tie rate reduction and calibration improvements
3. Experiment with criteria weighting
4. Compare reasoning vs non-reasoning verifier models

Conclusion

By combining the mechanism-discovery ideas from Bilevel Autoresearch, the skill-as-trainable-artifact framing from SkillOpt, and the continuous verification techniques from LLM-as-a-Verifier, we have built a practical system for self-improving agent skills.

The work is now in a clean, documented, and production-ready state on the DGX Spark. The prototype demonstrates that verification can indeed be treated as a first-class scaling axis — and that doing so makes previously noisy optimization loops stable and interpretable.

---

Follow @MichaelGannotti for more on autonomous research systems and human-AI collaboration at SMF Works.

All code and artifacts referenced in this post live in the SkillOpt prototype repository.