Integrate Devin backend with ACP and update documentation - #34
Open
chenillen wants to merge 133 commits into
Open
Integrate Devin backend with ACP and update documentation#34chenillen wants to merge 133 commits into
chenillen wants to merge 133 commits into
Conversation
Add LLMTR as a model provider
…source adapter Pi (earendil-works pi coding agent, MIT) becomes a backend the way the runner's header always promised: one BACKENDS entry, a builder, a normalizer. `pi -p --mode json` emits a JSONL event stream that maps cleanly onto the canonical claude stream-json — session header to init, text deltas to assistant deltas with the codex-style self-healing tail on message_end, tool_execution_start/end to tool_use/tool_result, agent_end to the result. Two of pi's own behaviors carry real weight here: --session-id resumes a project session and CREATES it when the file is missing — the fresh-start fallback claude and codex each implement by hand is pi's documented default, so the resume path has no existence check to get wrong. The CLI exits 0 when the provider call fails. Verified against 0.84.2 with a bad key: the failure is stopReason="error" on the assistant message, nothing else. Status therefore derives from the synthesized result event, never the exit code, and the verbatim capture of that run is now a test fixture. Providers mirror hermes's multi-family shape: native env auth for anthropic/openai, a ~/.pi/agent/models.json custom provider for anything with a base_url, api picked by model family (claude -> anthropic-messages, gpt-5/codex -> openai-responses for the same reason hermes routes them there, else openai-completions). The /v1 suffix is normalized per api — pi's anthropic client appends /v1/messages itself, its openai clients want /v1 present. auth.json and models.json are checkpoint-excluded: both can carry the literal key. Pi ships without MCP by design, so MCP arrives as pi's own kind of answer: an extension. pi-mcp-adapter (MIT, github.com/nicobailon/pi-mcp-adapter) installs next to the CLI at first run, and the runner mounts it with -e only on turns that configure MCP servers, writing mcp.json at the agent-dir level — the one location outside pi's project-trust gate. Discovery stays off (--no-extensions): a task could otherwise drop .pi/extensions/ into the workspace and have the next turn execute it. Disabled tools are enforcement on pi, not instruction: -xt is a real per-tool switch, so the catalog entry says "hard" and means it. Skills land in the user-global skills dir (project-local files sit behind the trust gate); AGENTS.md is a context file and loads regardless. The image moves node 20 -> 22: pi's engine floor is 22.19, node 20 has been EOL since April, and claude/codex run unchanged. The release smoke test now asserts all four backends install on a fresh volume, on both architectures. Gateway: wiring rows for the seven integrations pi can serve, a multi-family model catalog (gpt + claude families to start), base catalog entry, and pi in every backend-name set. UI: the "coming soon" card becomes real — logo was already shipped. 15 new runner tests on the real captured stream; 337 gateway + 38 runner + tsc all green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
The fresh-volume run caught it: 'installing Pi' printed, /data/agent-tools/bin/pi existed and answered --version, and the summary still read 'backends available: claude codex hermes' — the probe loop iterated a hardcoded three-name list. The install and the probe now walk the same set. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
…e them first The console's Keys page mints sk-hr-… keys "to make one and call the API", the /api/harness proxy forwards Authorization through, and the gateway resolves the org from the key. None of that could ever run on self-host: the middleware checked only the session cookie and answered 401 before the proxy did, so every bearer-only call died at the door. Measured before the fix: cookie 200, cookie+garbage-bearer 200, valid-key-no-cookie 401 — the key was the one credential that did not work. Found by the UHP conformance suite, which authenticates exactly this way and skipped or failed 50 of 52 checks on HTTP 401. Requests carrying a bearer in the key's own format now pass the gate on the API proxy path only; the gateway stays the authority and 401s bad keys itself. Pages and every other route remain cookie-gated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
…eeds no key at all Follow-through on the API-key fix, driven by the UHP suite's remaining failures. Letting bearer calls past the cookie gate exposed the next assumption down: the BFF proxy stamped internal trust and the pinned local identity on EVERY self-host request, because when it was written the cookie gate in front of it was the only door. Measured: a syntactically valid but nonexistent key read /v1/harnesses as the local org. Trust now follows the SESSION — a caller with a valid cookie (the signed-in console) gets the internal path exactly as before; anyone else forwards their Authorization bare and the gateway's _apikey_resolve decides, with its revocation check and its canonical authentication_error body. The gate also passes ANY bearer through, not just well-formed keys: an invalid credential must earn the gateway's own 401 (error.type, UHP-Version header — the shape check A-02 requires), not this gate's sign-in JSON. And /v1/uhp joins the public list, because discovery being unauthenticated is itself a spec requirement (D-02): a client must be able to learn whether this is a UHP server before deciding what credential to present. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
…Version The gateway stamps UHP-Version on every response — the protocol requires exactly that (V-01) — and the BFF forwarded only its content/caching allow-list, so no client behind the console port could ever see which contract it got. Forwarded now on both branches, SSE included. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
…ENTS.md label Three things the e2e run surfaced. The Add Harness dialog described Pi with Claude Code's copy — literally the same sentence, which even broke test automation that tried to select a card by its description. The base filter dropdown stopped at three backends. And a pi harness's instructions field was labelled CLAUDE.md when the runner seeds AGENTS.md for pi. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
…re it is listed "Why does pi only support gpt and claude?" — it never did; the CLI runs anything OpenAI/Anthropic-compatible. The catalog was narrow because the catalog is a verification record, not a capability statement, and only two families had been through a checked turn. So the other nine earned their rows the same way hermes's did: probed through the pi CLI on the TokenRouter connection (openai-completions custom provider), every reply echoed exactly, every response reporting the requested model id — the substitution check that exists because nine models once "passed" a test that was claude answering nine times. gemini-3.6-flash, deepseek-v4-pro, deepseek-v4-flash, kimi-k3, kimi-k2.7-code, qwen3.7-max, qwen3.8-max, mistral-medium-3.5, step-3.7-flash Five hermes models stay off pi's list, and the comment says exactly why: TokenRouter lists no channel for them, hermes reaches them via OpenRouter, and no OpenRouter credential was available to probe pi with. Unprobed is unlisted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
…replay The failing session was a genuinely multi-model conversation — claude-sonnet-5 with tools (including a screenshot image in one tool result), then gpt-5.4-mini, deepseek, kimi — and every follow-up worked until qwen3.7-max, which answered 400 InvalidParameter: Unexpected item type in content. A fresh session on the same model twice was fine, which is the tell: the poison was in the HISTORY, not the model. Isolated by direct probes on the TokenRouter connection: qwen3.7-max rejects image input outright (a user message with an image part reproduces the exact 400); qwen3.8-max complains about the 1x1 probe's DIMENSIONS (so it takes images), and gemini/deepseek/kimi/mistral/step all accept. One text-only channel, one deny-list entry. The mechanism was then verified against a capturing sink, not inferred: with the runner's models.json declaring input ["text","image"] (what it always wrote), pi serializes a replayed image tool-result as a separate user message carrying an image part — the rejected shape. Declared ["text"], pi drops the image and the same resume completes. So pi's own capability field is the fix: the gateway now says which models are text-only (measured, one entry), the turn body carries it, and the runner writes `input` accordingly. A degraded answer that lost one screenshot beats a conversation that can never continue. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
Seen live on a kimi channel: the streamed deltas and the message's final text disagreed, and the self-healing re-emit painted the whole answer a second time in the transcript. The deltas are already on screen, so the re-emit is suppressed for that one case; the no-deltas path still emits the full text once, and the result event carries the authoritative final either way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
…rmes, Pi The list sorted every row by last activity, so whichever base you had just tried jumped the queue — Pi sat above Codex two minutes after its first task. Built-ins now keep a fixed catalog order with Pi last; custom harnesses stay activity-sorted, above them, because the things you made are what you manage on this page. Same order in the Add Harness dialog and the gateway catalog. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
Pi, the fourth harness — with MCP via the open-source adapter, e2e- and UHP-verified
…a Pi harness Same suite, same instance class as the 0.3.0 report; this one exercised the task checks through the new pi backend (gpt-5.4-mini on a TokenRouter integration), which is the release's headline change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
Named assuming the next version was 0.5.0; 0.5.x and 0.6.0 had already shipped from alongside this branch. Same report, same run, right name. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
… with a relay that unbreaks aggregator tool calls dsh (deepseek-ai/deepseek-harness, MIT, developer preview) becomes a backend along the lines its own audit drew (harnessrouter-management, 2026-08-13): the official Python SDK (`deepseek-harness-sdk==0.1.0rc7`, pinned exactly, never `latest`) drives the bundled JSON-RPC runtime executable (`deepseek-harness-runtime-bin`, same pin — no system Node involved), and a per-turn driver re-emits every session.event as NDJSON for the runner's normalizer. Cancel stays a process-group kill; the runtime dies with the group. The Phase-0 spike the audit prescribed found the gap it predicted, just one layer lower: dsh's LLM adapter accumulates streamed tool calls guarded by `!== undefined`, and aggregators like TokenRouter send continuation deltas with EMPTY-STRING id/name where OpenAI omits the fields — so the second delta clobbers the real name and every tool call died as UNKNOWN_TOOL (captured live against deepseek/deepseek-v4-flash, raw SSE on file). The audit's rule was to bridge outer-layer gaps without a fork, so the driver runs a loopback relay that drops exactly those empty fields — and, as a consequence the credential design gets for free, the REAL key lives only in the driver process: the runtime is launched at the relay with a placeholder key, so the credential never enters the dsh process env, its session log, or anything a checkpoint could carry. Mapping follows the audit's table: reasoning/text deltas stream, committed assistant/message emits only the un-streamed tail (the pi lesson), tool/call + tool/result pair up by callId, usage is keyed by (turn, step) and REPLACED — the retry ladder re-reports a step, and summing would double-bill — then summed at the end. turn/end error/max-tokens/completed map to error / error_max_turns / success, and the terminal event is the only status signal, because the driver exits 0 either way. Single-family by design in this phase: the audit defers multi-provider composition until one path proves model/usage/tool/error consistency, so the catalog is deepseek-v4-pro and deepseek-v4-flash over the four OpenAI-compatible integrations. No first-party DeepSeek Platform integration yet — nobody here holds a platform key, and unprobed is unlisted. Instructions land in AGENTS.md (dsh reads it via dsh-agent-instructions); MCP servers compose into the bundled cordis as dsh-mcp-client entries. Logo is upstream's own MIT mark. 10 new runner tests on the captured fixtures; 337 gateway + 51 runner + tsc green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
…d for the one gap it left shut
The deployed e2e hit it immediately: a follow-up turn died with `session "…" already has a
persisted log on disk that does not match this live session (id collision)`. Upstream's SDK
JSON-RPC server only ever calls ctx.agents.create(); the persistence coordinator's adoption
path needs an in-memory seed a fresh process cannot have, and the wire has no resume method.
Exactly the continuation gap the audit predicted — with its remedy: bridge it out-of-tree,
version-locked, without forking the runtime.
The bridge is hr_dsh_server.cjs, loaded as a CONFIGURATION-RELATIVE plugin (a documented
packaged-bin feature) from a cordis composition that is the wheel's own bundled default with
one entry swapped. It subclasses the exported HarnessSdkJsonRpcServer and overrides only
createSession: try ctx.agents.resume({resumeSessionId}) first, fall back to the parent's
create — so a fresh id costs one failed resume and a stored id loads its history. Everything
else (prompt handling, notifications, shutdown) stays inherited, verbatim upstream.
Two mechanical facts the file documents because they will bite whoever bumps the pin: the
plugin lives outside the runtime's pkg snapshot, so bundled modules resolve through a
createRequire anchored inside the snapshot's node_modules (derived from process.argv[1]);
and the override reaches TS-private fields, so it hard-refuses to boot against any bundled
version other than the pinned 0.1.0-rc.7.
Proven locally against the real runtime executable: three drives of one session id from
three separate processes — persisted zstd log on disk, zero collisions, every turn reaching
the LLM stage. The deployed instance reproduces the before; the e2e that follows proves the
after through the product path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
… say so The MCP e2e wedged a task at Working forever. Root cause in two parts: the pinned runtime wheel (0.1.0rc7) does not bundle dsh's MCP client — upstream's sdk-runtime README documents it as included, but the wheel predates that — and naming the entry fails the plugin tree, which leaves the runtime process alive with a mute stdout, so the SDK's unbounded initialize request waits forever. The driver now sets request_timeout_seconds=180 (a mute boot becomes a loud RequestTimeoutError instead of a wedge) and, per the pi precedent, announces-and-skips configured MCP servers on this runtime build: the turn still runs, the line says why the tools are absent, and DSH_RUNTIME_HAS_MCP flips when the pin moves to a wheel that ships the client. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
…s a dsh plugin The pleasing discovery first: dsh's multi-provider layer, dsh-llm-pi-ai, IS pi's unified LLM library (@earendil-works/pi-ai) wrapped as a Cordis plugin — and probing the pinned runtime wheel showed it bundled (unlike the MCP client, whose README-vs-wheel gap taught that lesson). So the expansion is mostly routing the pi backend already proved: a hand-declared "hr" route at the driver's loopback relay, api picked by model family (claude -> anthropic-messages, gpt/codex -> openai-responses, else openai-completions), model capability declared per turn — including input:["text"] for the one channel measured to refuse images (the qwen lesson travels with the flag the gateway already sends). Family decides the route, not the integration's name: deepseek models keep the verified dsh-llm-deepseek launch path whichever endpoint serves them. The relay grows two duties it did not have: forward protocol headers verbatim (anthropic-version), inject the credential in BOTH spellings (authorization + x-api-key), and rewrite the empty-string tool-call quirk only on chat/completions — the other protocols pass through byte-faithful. The real key still never enters the runtime: the pi-ai route's apiKeyEnv resolves a placeholder. Wiring mirrors pi's rows; the catalog carries the family set pending the e2e probes that follow on this branch — any row the probes refuse comes back out before merge. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
…client appends it Same lesson the pi backend already paid for, re-learned at the relay: pi-ai's anthropic-messages client appends /v1/messages to its base, so composing the hr route with …:port/v1 produced /v1/v1/messages and every claude-family turn died invalid_request_error. Bare base for anthropic-messages, /v1 for the openai apis, with a composition test pinning both spellings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
DeepSeek Harness, the fifth backend — official SDK transport, aggregator tool calls unbroken
…epSeek Harness harness Task checks ran on claude-haiku-4.5 via the dsh backend's pi-ai route — the release's headline change exercised inside the suite itself. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
Retaken from a pristine 0.8.0 container so the frame matches what a new user actually sees on first sign-in: Codex, Claude Code, Hermes, Pi, DeepSeek Harness. Alt text was still counting three. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
…he old one Safari on a first visit was still served the three-harness image from an edge cache — same URL, stale bytes, beyond any client's control. A new filename is the one cache-bust that works everywhere. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GknTsrWVoAbJe8qX7i8QA3
Main grew its own CONTRIBUTING.md (and protocol/GOVERNANCE.md with the UEP process) after this draft was written, so the add/add conflict is resolved on main's version, with what this PR adds beyond it folded in: the two-layer UHP/implementation framing, a pull-request checklist, and the development-check commands. The README hunk is dropped — main's README already links CONTRIBUTING.md from its community section, so the added section would say the same thing twice. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ibuting-governance docs: add UHP contribution and governance guide
…', and the report dates itself The README states the rule and the human renderer honours it, but to_json() set conformant: true whenever nothing failed or errored — a run in which most checks never executed serialised green, in the artifact GOVERNANCE.md expects to be published as evidence, read by people who did not run it. Now: - conformant is strict: every check ran and none failed or errored. One skip makes it false. - conformant_with_skips keeps the old meaning under an honest name, and skipped_not_verified enumerates the check ids the report establishes nothing about. - suite_version and generated_at (UTC) tie the report to the suite revision and the moment that produced it; the 0.3.0 report had to be dated in prose because the file could not date itself. Reports without these fields predate suite 2026.8.11.post1. - highest_class() goes strict the same way: fails, errors, skips, and classes with no results at all break the ladder — before, a run that only exercised core reported highest_class_passed: "full", crediting classes that never ran. - The human summary speaks the same vocabulary: CONFORMANT WITH SKIPS, in yellow, instead of a green CONFORMANT with a footnote. The checked-in reports are artifacts of the older suite and are left as they are; the README says how to read their shape. Closes HarnessRouter#7. Reported by @asj305, with the fix shape as proposed there. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
release.yml gates what ships, but gateway/tests, runner/tests, protocol/conformance/tests and the console's checks only ran when someone remembered to. One job per suite so a failure names the area that broke; ffmpeg is installed for the gateway job because the media tests gate themselves on have_ffmpeg() and silently skip without it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jest exits 1 on 'No tests found' and the ui has no tests yet; the flag makes an empty suite pass while a failing test still fails. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rip it for the turn The deepseek-official adapter sends reasoning_effort on every request. DeepSeek's own API takes it; aggregators serving deepseek/* over the OpenAI shape refuse the whole request (LLMTR: 'The "reasoning_effort" parameter is not supported by "deepseek/deepseek-v4-pro"', captured 2026-08-20 by conformance T-01 — four full-class checks failed on that one rejection). The relay now retries such a rejection once without the parameter and strips it for the rest of the turn, so a provider that accepts it keeps it, and one that refuses it costs one extra round trip, once. Verified live: dsh on deepseek-v4-pro through LLMTR went 47/52 (4 failed, 1 skipped) to 52/52 full-class conformant, including X-07 — the artifact the failed turn never produced now downloads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… Error shape
The spec's Error object requires `type` from the closed enum, with `code` left
for the specific condition. The gateway emitted {"code": "harness_error",
"message": ...} — the type in the code slot and no type at all — so EVERY failed
turn produced a spec-invalid Response. Nothing noticed until a conformance run
actually failed a turn (TokenRouter rejecting hermes's empty text block, LLMTR
rejecting dsh's reasoning_effort, both 2026-08-20): T-01/S-03/S-07/X-05 then
flagged the envelope on top of the failure.
Now: harness-reported failures carry type=harness_error code=turn_failed, an
exhausted connection ladder carries type=harness_error code=connections_exhausted
with the tried-ledger in message, and an unhandled exception is honestly
type=server_error code=unhandled_exception. The SSE error event mirrors
error.code so the two never disagree.
gateway/tests/test_response_error_envelope.py validates fail()'s envelope and
the full failed Response against the schema, and asserts at the source that
every .error literal leads with a spec type.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The console showed opencode.json as a produced file on every turn. Produced files are `git status` of the session workspace, and the config was written at the workspace root — so internal configuration (the relay URL, the key env var name, data-volume paths) was collected and presented to the user as if the agent had produced it. That is exactly the internal-architecture leak the product rules forbid, and the same reason every other backend keeps its config under .harness/. Moved to .harness/opencode.json, with OPENCODE_CONFIG pointing the binary at it — verified against the shipped 1.18.23: the config at that path is loaded and reaches the provider. Regression test pins both the location and the env var. 137 tests pass, pyflakes clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X7VoW2QzfDkxEoudktbjVY
The square mark from opencode.ai (its 180x180 apple-touch icon), replacing the generic glyph fallback. Same mechanism as the other five: one file in /public/logos plus one row in HarnessLogo's map. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X7VoW2QzfDkxEoudktbjVY
…x hours Seen live: the container was SIGKILLed mid-turn, and after restart the task sat at 'Working' with a finished-looking transcript. The reconcile sweep found the stale heartbeat but could not settle it: the trace had no terminal status, adoption is gated on the control store (absent self-hosted), so the only exit was the six-hour _GW_MAX_TURN_S cap. The record was honestly reporting a process that no longer existed, forever-ish. The sweep now asks the RUNNER whether the recorded turn still exists before considering adoption. Turn records are in-memory there, so a restart answers 404 — and 404 means the owning process is gone and the turn can never finish. That settles it as failed on the next sweep cycle. An unreachable runner is NOT treated as gone: transient network means retry later, never fail. SaaS behaviour is unchanged: a live sandbox answers 200 and the adoption path proceeds exactly as before; a reaped per-session sandbox now also settles promptly, which is strictly better than the cap. Verified by reproducing the incident end to end on the deployed container: started a 40-second turn, docker kill mid-flight, restart — the session settled to 'failed' within one sweep cycle instead of showing 'Working' for six hours. This is the recovery-pass-on-startup argument from issue HarnessRouter#25 landing in practice: if turn ownership lives in a process, repairing state after a crash is part of the feature. pyflakes clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X7VoW2QzfDkxEoudktbjVY
…invent 'completed' Follow-through on the restart-orphan fix, from the same live incident. Settling only the session vertex left the turn's RESPONSE record at 'running', so the conversation view kept a live 'Working…' spinner on a turn whose process was gone — the sweep removed the lie from one record and left it in the other. The sweep now settles the session's running_response_id in the same pass through _reconcile_response. And a response settled from session-terminal state with NO output of its own now lands as 'incomplete', not 'completed'. Mapping the session's 'done' straight onto the response invented a success: a 'continue finish the work' that never produced a byte read as completed in the turn list. 'completed' is a claim about this response, not the session. pyflakes clean. Verified on the live instance: the stuck thread settled to four terminal turns with no spinner. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X7VoW2QzfDkxEoudktbjVY
…imply collected Watched the same deck get stranded twice in one evening, two different ways (an OOM kill at finalize, then a cancel landing after the file was written). Root cause is structural: /produced was uncommitted-vs-HEAD while /checkpoint moves HEAD unconditionally, so HEAD doubled as the collection cursor. Any terminal path that skipped collection followed by any checkpoint buried the turn's files: on disk, invisible to /produced, forever. The cursor is now its own ref (refs/hr/collected): - /produced lists everything since the last ACKNOWLEDGED collection — committed changes past the ref plus untracked files — so checkpoints in between change nothing - POST /produced/ack commits and advances the ref; the gateway calls it ONLY after the files are captured to blobs, which is the one thing that makes collected mean collected - the ref initializes to HEAD on first touch (hydrate semantics: what arrived in the checkpoint was not produced by any turn here), and .git is already inside the checkpoint tarball, so the cursor survives sandbox recycling - the gateway now collects on EVERY terminal outcome — a cancelled or failed turn may already have written the deliverable, and skipping collection threw it away from the record while it sat in the workspace A dead turn now strands nothing: whatever it left behind is simply produced by the next turn that completes collection. 4 new tests pin the semantics; 141 pass; pyflakes clean both trees. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X7VoW2QzfDkxEoudktbjVY
…backend feat: OpenCode backend
…ld automation Spec is now served per version at /spec/<date>/<chapter> (spec-first, mirroring MCP's /specification/<date>/…), with a header version dropdown and a /spec → latest redirect. Only the specification is versioned; conformance, governance, versioning and changelog stay unversioned. VERSIONS is the single source; build asserts every listed version has a source tree and was built (no dropdown 404s). Homepage reads title → definition → quick-links (neutral-standard order) and carries the same on-this-page rail as every other page. The rail gains a scroll-spy (32%-anchor / last-passed-heading, aria-current) and a flatter design. Implementations page renamed Examples, grouped Example servers / Example clients. README drops the named-implementations list and the commercial-product framing; it points at the examples page (SSOT) and states no hosted service is required. Automation: one host config at protocol/vercel.json (framework:null, build via site/build.sh, output site/dist) that build.py also copies into dist, so the Vercel Git build and a manual deploy share one source. build.sh is the single build-step source Vercel and CI both call. requirements.txt pins Markdown; a tests.yml `site` job runs the same build.sh via .python-version and asserts the config stays wired. build.py self-checks internal links, #anchors, versions and the description manifest, so a broken build fails the PR, not production. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ations-and-search site: version-addressable spec URLs, TOC scroll-spy, and repo→web build automation
ACP-per-turn integration design for adding Devin as a runner backend in HarnessRouter Community Edition. OpenCode is already supported; this spec focuses on the proprietary Devin CLI via the Agent Client Protocol. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- Model the ACP driver on _run_codex_appserver_bg (JSON-RPC over stdio). - Add a pattern-alignment section comparing Devin to Codex app-server, Hermes, OpenCode, and Claude Code. - Clarify custom dispatch in turn(), resume guard, provider catalog/wiring, and tool_enforcement. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Switch the Docker install section to the official curl-bash installer, with HOME/XDG_DATA_HOME redirected to the data volume and a post-install symlink into the existing $TOOLS/bin path. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Seven tasks covering runner normalizer, ACP driver, registry/dispatch, gateway catalog/provider wiring, UI harness, Docker install, and conformance. Modeled on the existing codex app-server driver and aligned with the approved design spec. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ller - Add 'devin' to the OobHarness backend union and list - Add a devin logo mapping and a simple SVG mark - Add devin to HR_BACKENDS and backend_bin in the entrypoint - Add install_devin using the official curl-bash installer - Redirect HOME/XDG_DATA_HOME to the data volume and link devin into $TOOLS/bin
- Add `ignoreDeprecations: "6.0"` to `ui/tsconfig.json` to silence the `moduleResolution=node10` and `baseUrl` deprecation warnings emitted by TypeScript 6.x in this workspace. - Replace the placeholder Devin logo SVG with the official mark.
Devin's ACP server requires an authenticate JSON-RPC call with the advertised auth method and the API key in _meta.api_key before session/new or session/resume. The driver was skipping this step, so session creation failed with "ACP host has not authenticated". - Send authenticate after initialize/initialized. - Wait for auth success before creating or resuming a session. - Update the fake ACP server to advertise authMethods and accept authenticate. Generated with Devin (https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…o DEVIN_API_KEY The user will configure the API token as `DEVIN_API_KEY` in their environment, so the runner now uses that name consistently: - `turn()` sets `env["DEVIN_API_KEY"]` from `auth.api_key`. - `_run_devin_acp_bg` reads `DEVIN_API_KEY` for the ACP `authenticate` `_meta.api_key` field. - The fail-closed missing-credential message now references `DEVIN_API_KEY`. - Update the design spec and implementation plan to match. Generated with Devin (https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- ui/tsconfig.json: migrate from the deprecated `moduleResolution: "node"` and `baseUrl`/`ignoreDeprecations` combo to `moduleResolution: "bundler"`, so the Next.js production build passes under the project's TypeScript 5.x. - docker/entrypoint.sh: the Devin install script runs an interactive `devin setup` at the end that fails in a headless container. Ignore the installer exit code and verify the binary exists before declaring the install successful, so the runner can use the user-supplied `DEVIN_API_KEY` at turn time. This lets `docker build` succeed and `docker run -e HR_BACKENDS=devin -e DEVIN_API_KEY=...` install and run Devin on the data volume, like the other backends. Generated with Devin (https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Replace the static six-model Devin catalog with live discovery against
the Codeium Connect-RPC GetCliModelConfigs endpoint. The gateway now
fetches a fresh model list per configured Devin API key, caches it for
five minutes, and exposes the latest SWE-1.x family id (currently
swe-1-7) plus the legacy devin-swe alias.
Discovery is fallible: missing credentials or network failures fall back
to the existing static catalog. The model labels from Devin are surfaced
in /v1/models and /v1/bases, and the vendor table is updated so the
runner receives an authorized model.
Local verification:
- GET /v1/models returns devin: {default: swe-1-7, models: [swe-1-7, devin-swe]}
- Turn with swe-1-7 and devin-swe both complete and write workspace files.
- gateway/tests: 368 passed, runner/tests: 145 passed.
Generated with Devin
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
@chenillen is attempting to deploy a commit to the Future HR Team on Vercel. A member of the Team first needs to authorize it. |
Contributor
|
Hi @chenillen, thank you for contributing this PR. We sincerely appreciate the time and effort you put into it. We reviewed Cognition’s published terms and could not find language clearly authorizing a third-party Apache-2.0 project to integrate Devin in this manner. The terms limit the use of Devin to internal business purposes and restrict making the service available to others or developing competing products without Cognition’s written approval. Are you submitting this integration on Cognition’s behalf and authorized to do so? If so, please provide written confirmation from Cognition covering this open-source integration. Otherwise, we will need to substantially revise the PR or place it on hold until we receive that approval. We are taking this step out of respect for the intellectual property rights of Devin’s creators. We would be happy to discuss this privately and work with you toward a compliant path forward. You can find us on Discord, or contact@harnessrouter.ai Thank you again. We are grateful to have you contributing to HarnessRouter. |
Author
Thanks for your reply, I am actually not on the behalf of Cognition, just want to integrate to HarnessRouter. This project looks awesome. |
Contributor
|
Thank you, @chenillen. We truly appreciate your contribution. If you find other agent harnesses that you believe UHP and HarnessRouter should support, and whose licensing permits integration, we would warmly welcome those contributions. We value the work you have put into this PR, so we will leave it open while we reach out to Cognition and invite the community to help establish contact. We hope to obtain either written approval for this integration or clarification of licensing terms that would permit it. Thank you again for helping expand the UHP and HarnessRouter ecosystem. |
richard-epsilla
force-pushed
the
main
branch
from
September 4, 2026 04:43
b4fdaa3 to
6876cea
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.