CI for Hybrid Routing: ollama-cloud Is Not Local
hermes-plugin-hybrid-routing already had 400+ tests and a fail-closed egress model. It had no CI. This post adds the matrix and locks the ollama-cloud false-local contract.
Jasmine Naderi
Principal Engineer, Agent Systems
smfworks/hermes-plugin-hybrid-routing at kickoff SHA ddf6e7647caf was already a serious plugin: blank-by-default models, operator-attested model_egress, fail-closed sensitive routes, and a large pytest suite. What it did not have was CI.
A production routing path with no pull-request gate is a demo that happens to live on main.
Praxis and Swarm hardening belong to a separate official program. This card did not open a second PR on those remotes.
Original state
| Fact | Evidence |
|---|---|
| Version | 1.1.0 |
| Tests | tests/test_router.py, test_plugin.py, test_metadata.py |
| CI | None |
SECURITY.md |
Missing |
| Locality rule | Exact model_egress: local only. Names do not imply local. |
The packaged default already documents the trap:
# "local" is an operator attestation; verify the provider's real endpoint.
model_egress:
ollama-cloud/glm-5.2: external
There was no test that set local_only_model: ollama-cloud/glm-5.2 with an empty registry and asserted a block.
Decision
- Add
.github/workflows/ci.yml: pytest on Ubuntu 3.10/3.12, Windows 3.12, macOS 3.12; ruff; mypy; bandit. - Add
SECURITY.mdandCONTRIBUTING.md. - Lock the false-local contract in pytest.
The new test does not change the router. It prevents a future "helpful" inference from provider names.
def test_ollama_cloud_name_is_not_inferred_local_for_sensitive_routes(tmp_path):
def configure(config):
config["tiers"]["balanced"]["model"] = "ollama-cloud/glm-5.2"
config["sensitivity"]["local_only_model"] = "ollama-cloud/glm-5.2"
config["model_egress"] = {}
decision = configured_router(tmp_path, configure).classify(
"password=private-value"
)
assert decision.sensitivity == "sensitive"
assert decision.model == ""
assert decision.disposition == "block"
assert decision.egress != "local"
What changed
PR: smfworks/hermes-plugin-hybrid-routing#6
CI, SECURITY.md, and packaging gates already landed in #2 (16a2e06, v1.1.1) while this wave was in review. The remaining unique change is the locality regression:
f313c8b test: ollama-cloud names are not inferred as local
Verification on this machine:
pytest -q
# 428 passed in 8.88s
That count includes the new test. It is not a README number.
Remaining limits
- CI does not run a live Hermes
PluginManagerinstall. Dual-surface plugin verification stays a release-gate procedure, not this PR. model_egress: localis still an operator attestation, not a network probe. The plugin cannot prove the endpoint is loopback.- Bandit on the plugin package is a static baseline, not a substitute for the egress tests.
Lesson
A routing plugin that already fails closed still needs a gate that will fail a future PR that "just treats ollama as local." Tests without CI are a local habit. CI without that one test is a green check that misses the production bug.