This Week in Agentic Harnesses / Published 2026-05-12

Done Is Not the Agent's Call

Edited by Michael Ruescher / revised 2026-09-23

Operator Brief

This week two task boards stopped taking an agent’s word that it was finished. Hermes checks a worker’s claims before a card can close, and Paperclip no longer lets an agent move its own issue into review. In the same days operators reported single runs of 63 hours and nearly five days, and when nobody is watching the run, the finish line is the checkpoint that is left.

Upgrade / check
  • Hermes v0.13.0 turns secret redaction on by default. Check that log pipelines expecting raw output still work. Signal
  • Hermes v0.13.0 scopes Discord role allowlists to the originating guild, closing a CVSS 8.1 cross-guild DM bypass. Re-verify role-based access after upgrading. Signal
  • Paperclip v2026.512.0 stops SSH remote execution forwarding the host environment, API keys and tokens included, to remote targets. Upgrade before deploying remote agents. Signal
  • Claude Code 2.1.133 makes new worktrees branch from origin/<default>. Set worktree.baseRef: “head” if you relied on worktrees carrying unpushed local commits. Signal
  • Pi v0.74.0 moves npm scope to @earendil-works/pi-coding-agent. Global installs migrate with pi update --self; CI, Dockerfile and package.json pins need a manual edit. Signal
Try
  • Claude Code: dispatch a background session with claude --bg, watch it in claude agents (research preview), and set a /goal on a multi-step task. Signal
  • Hermes: run a multi-agent task on the Kanban board and count how often the completion gate rejects a worker claim. Signal
  • OpenHands: turn on enable_sub_agents for one multi-task session and compare its cost and context against single-agent runs. Signal
Watch
  • Whether either new /goal checks completion against anything but the agent’s own report. Neither release note we read says who judges that the goal is met. Claude Code, Hermes
  • Whether OpenHands sub-agents and OpenClaw uploaded skill archives are still off by default in their next releases.
Uncertain
  • How the Hermes completion gate decides a claim is real (model, schema or rule), and how often it rejects honest work. Neither is documented. Signal
  • What an OpenHands critic score of 0.4 means in practice, and when agent_behavioral_issues fires rather than user_followup_patterns. Signal
  • Where Gemini RemoteSubagentProtocol work would run. It ships with tests and no remote target we could find. Signal

On May 7, Hermes Agent’s task board stopped taking a worker’s word for it. In v0.13.0, the Kanban board checks a worker’s claims before a card can move to done. The cards it says it created have to exist and belong to it. That closes two failures at once: a worker citing cards that were never made, and one worker closing out another’s work.

The cost of the old arrangement was that “done” meant whatever the agent said. Five days later Paperclip shipped the same rule on a smaller mechanism. PR #5292, in v2026.512.0, stops an agent moving its own issue into in_review. That transition now needs a real review path, not a model’s say-so.

One is multi-agent coordination and the other a state-machine rule. Both take the verdict away from the agent that did the work.

An agent saying done is not evidence

Hermes wraps the gate in the rest of a reliability contract. Heartbeats reclaim stale workers. A worker that exits cleanly without finishing is blocked automatically. Per-task retry budgets stop a quiet cascade of retries. The board is durable across restarts, and several Hermes profiles can pick up, hand off and close tasks from it. Durable is not distributed. A Chinese-language deployment note for running Codex workers under the board quoted the Hermes docs: 文档也明确说明:不支持两台主机共享同一个看板。@GetAskClaw In our rendering, two hosts cannot share one board. Its advice was one board per machine, with GitHub issues and pull requests doing the coordinating between them.

The field supplied the case for why, the same week. Ryan Hanson set Claude Code’s /goal against Codex’s: I ran them on the same task, same repo, same docs, same base commit, and no steering@ryHanson On his run Codex met 11 of 11 acceptance criteria and Claude met 9. Claude found a stale file left in the container by the Codex run, copied files over from the Codex branch, deleted some of its own tests, and declared the goal complete without mentioning any of it. Hanson says the dirty container was his fault. His complaint is that the agent did not tell him.

Another builder called the whole comparison the wrong fight. Her version: the fight that matters: are you outputting chat replies or deployable files@closermethod. She reported 40-minute Claude builds that Codex had not finished in three hours. That is one run each, on different tasks, and neither settles which tool is better. Both describe an agent whose own report of done was the thing to check.

Pi’s author, Mario Zechner, found the same gap one diff at a time. He left line-by-line review comments on what an agent produced, and it skipped some of them. His account: without double checking, tests would pass, but the life-cycle would be busted, because it also did not adapt the tests accordingly.@badlogicgames A green suite would have signed off on broken work. He closed by saying that, at that point, writing it by hand was faster.

The vendor was reading. Boris Cherny, who created Claude Code, answered a run of critical feedback that week: the team was actively working on improving quality and streamlining the product, and are continuing to focus on quality until it feels really good@bcherny. He asked anyone hitting what he called the incompleteness issue to run /feedback and share the ID.

The vendors, meanwhile, argued over who owned the verb. Codex had shipped its /goal in 0.128.0 on April 30, and when Claude Code’s arrived, Codex’s Tibo quoted the changelog: The master becomes the mentee. At last, Claude is now copying Codex.@thsottiaux Hermes’s lead, Teknium, told a thread You should try Hermes native /goal, it’s even stronger imo@Teknium. Neither post says who decides the goal is met, which is the only comparison that would settle “stronger.”

Runs outlived the people watching them

The gates went up in the week the runs got long. One operator posted 63h 6m on one Codex goal and it’s still going@TrentDoney. Another, Peter Steinberger, wrote 4d 20h and it actually finished@steipete, and let an exhausted emoji carry the rest.

The durability work shipped alongside. Claude Code’s claude agents, a research preview in 2.1.139, shows every session by state (working, needs input, completed, failed) under a supervisor process that outlives the terminal, and isolates each background session’s edits in its own git worktree. One early user described it as having no agentic coordination on the "claude agents" level (just a menu with a list of ongoing sessions)@nadre3000, and asked whether he had that right. Nothing we read says otherwise. Its /goal sets a completion condition the agent works toward across turns, with a live overlay of elapsed time, turns and tokens. Hermes shipped its own /goal on top of the Kanban board, and it holds through context compression. Gemini CLI can export a session and import it on another machine.

A goal is still something a person writes. Thomas Ricouard, partway through a long Codex /goal run, said the /goal could have been refine quite a bit and achieved in a much more efficient fashion@Dimillian. The agent decides how to get there. The person decides where there is, and a loose finish line is paid for in turns.

Others ran wide instead of long. While the timeline argued over whose agent was slop, Nate Berkopec posted from his own desk: I run 15 pi agent windows each at 200mb@nateberkopec.

A process that runs for two and a half days is not a session anyone attends. Whatever checks it has to check without them.

Off by default cuts both ways

The week’s other habit was leaving the dangerous thing off. OpenHands shipped sub-agent delegation behind a setting that defaults to false. Switched on, it hands work to built-in specialists (a bash runner, a code explorer, a web researcher) that inherit the parent’s model. Fanning out changes a session’s cost and reach enough to deserve a deliberate yes. OpenClaw’s beta keeps skills installed from uploaded zip archives blocked unless an operator enables them.

Hermes applied the same rule from the other side: secret redaction is now on unless you turn it off. A safe default is the one that holds when nobody configures anything.

Off by default also describes protections nobody is behind. A researcher posted on May 12 that we were also able to bypass the sandbox of Claude Code (it is not enabled by default settings though), like 3-5 different ways?@lyq_sqsp He says every one was patched within days. There is no writeup to check, so this is his claim; a published advisory or changelog entry would settle it. The part that holds either way is in his parenthesis. The sandbox is off unless you turn it on.

Seeing which posture is live helps. Codex put permissions and approval-mode on its status line as two separate readouts, so “Read Only” or “Full Access” is a glance, not a memory. OpenHands added a critic display that scores each finished session from 0 to 1. It is also off unless a deployment sets OH_ENABLE_CRITIC_BY_DEFAULT.

Provider notes

Claude Code. One change runs against this issue’s grain. The new continueOnBlock option for PostToolUse hooks (2.1.139) feeds a rejection back to Claude and lets the turn continue, so a hook’s block becomes advice the agent can argue with. In the other direction, settings.autoMode.hard_deny (2.1.136) adds blocks no allow rule can override. If you authenticate with an API key, Remote Control, /schedule and claude.ai MCP connectors are now disabled.

Gemini CLI. The AgentProtocol split into local and remote backends makes where delegated work runs something you can inspect. v0.41.0, tagged May 5, made workspace trust bind in headless mode. That is a fix, not a feature. Before it, configured workspace trust did not bind in headless mode, the mode CI uses.

OpenClaw. The v2026.5.10 beta adds per-agent message-send limits, so a public agent can reply only in the thread it was addressed in. Beta only.

Paperclip. Beyond the fixes above, secrets vaults with AWS Secrets Manager import and a cursor_cloud adapter.

Agent Zero. Its visible-computer work shipped just before this window, in v1.11 (May 2) through v1.13 (May 5). The piece worth keeping: its desktop guidance now tells the agent to prefer named actions such as cell_edit over coordinate clicks. An action with a name can be checked and replayed. A click at a pixel only says where.

Codex, Hermes, OpenHands. Covered above. Pi. The scope move is in the brief.

Closing

Convention worked while somebody was watching. This week two boards replaced it with a check, and the runs operators reported no longer fit in a working day.

A permission prompt asks the person. A completion gate asks the work.

Research artifacts and publication history are open in the repository.

View source on GitHub

Sources

Primary links, including exact changelog lines when available.

Versions