Skip to content

About

Homegrown durable multi-agent message bus (Redis Streams) for coordinating Claude/Codex agent sessions via a shared bus + GitHub-issue state machine

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

messagebus

A tiny, durable message bus that lets multiple AI coding agents — different Claude Code sessions, Codex, different models — talk to each other, ask questions, coordinate a shared task list, and co-author the same code. You inject the first prompt; the agents discuss, claim work, build in isolation, and (via the huddle) write one file together with peer review and a done-gate.

Everything is one Python file (bus) over Redis, plus a few shell scripts. No server, no framework, no auth — a single-operator local tool.

Why Redis Streams (not pub/sub)

Agent sessions are turn-based: they aren't sitting on an open socket. Redis pub/sub is fire-and-forget — a message published while an agent is mid-turn is gone. Streams are a durable log: every message persists and each agent keeps its own read cursor, so an agent that was busy (or offline) catches up on wake. The wait command uses a blocking XREAD for a push-like feel without the loss.

Coordination layers

Layer Mechanism Role
Chatter Redis Stream per room discussion, questions, answers
Task ownership Redis SET NX lock atomically decides who owns an issue
Task state GitHub issue status:* labels durable, human-visible state machine
Isolation per-agent git worktree agents build without colliding on files
Co-authorship the huddle (write-pen + done-gate) many agents, one branch, one writer at a time

Prerequisites

  • Python 3.9+ with redis-py (pip install -r requirements.txt; the file pins redis>=5.0).
  • Redis server on localhost. scripts/start-redis.sh starts one via Homebrew or redis-server if none is running.
  • git — required only for the worktree (ws) and huddle features.
  • GitHub CLI (gh), authenticated (gh auth login) — required only for the issue commands (claim/status/board/init/huddle). Plain messaging (send/poll/wait/…) needs Redis only.

bus is a self-contained executable (#!/usr/bin/env python3). Run it as ./bus from the repo directory, or symlink it onto your PATH.

Install & setup

git clone https://github.com/LJK-Consulting-Services/messagebus.git
cd messagebus
pip install -r requirements.txt        # redis-py

./scripts/start-redis.sh               # start local Redis if not already up
./bus doctor                           # verify Redis (+ gh, if you use issues)

If you want the GitHub-issue task features, also:

export BUS_GH_REPO=owner/repo           # which repo the issue commands target
./bus init                              # create the six status:* labels there (idempotent)

bus usually runs from this directory, which is not your project's git repo — so issue/label commands need BUS_GH_REPO=owner/repo to know where to act.


Usage

1. Messaging

./bus join   --as alice                         # register presence (cursor = "now")
./bus send   --from alice --to bob "ping"        # direct message
./bus send   --from alice --to all "hello team"  # broadcast
./bus send   --from alice --to bob --kind question --topic issue-42 "who takes the API?"
./bus poll   --as bob                            # read new messages for bob, advance cursor
./bus wait   --as bob --timeout 30               # block until a message for bob arrives
./bus watch                                      # live-follow the room (read-only, Ctrl-C to stop)

--from/--as are self-asserted agent ids ([A-Za-z0-9._-]). send takes the body as the last argument, or - to read stdin.

Ordering: start agents BEFORE injecting the opening prompt. join sets an agent's cursor to "now", so a message sent before an agent has joined is not delivered to it. Bring agents online (confirm with ./bus agents), then send.

2. Task coordination (GitHub issues)

Requires BUS_GH_REPO set and gh authenticated.

./bus claim  --as alice --issue 42               # atomic claim: Redis lock + status:claimed label
./bus status --as alice --issue 42 --set status:pr-open
./bus board                                      # table: issue | status | lock holder | present?
./bus renew  --as alice --issue 42               # extend the claim TTL (default 8h) on a long task
./bus release --as alice --issue 42              # give up the claim
./bus reap                                       # list stale locks (holder gone); --release <N> frees one

A claim is a Redis SET NX lock (the atomic race arbiter) plus a durable gh label. A second agent claiming the same issue is rejected. claim also cross-checks the status:claimed label so an expired lock can't silently allow a double-claim.

3. Isolated parallel building (worktrees)

Agents building in the same working directory collide on files and the git index. ws gives each claimed issue its own git worktree:

./bus claim --as alice --issue 42 --worktree     # claim + create an isolated worktree
cd "$(./bus ws path --issue 42)"                 # build here — the main tree is untouched
#   ... edit, commit, git push, gh pr create ...
./bus ws list                                    # all worktrees: dirty / unpushed / present
./bus ws remove --as alice --issue 42            # after merge (refuses if dirty/unpushed)

Each worktree is a sibling dir ../<repo>-worktrees/issue-N-<agent> on branch feat/issue-N-<agent>, cut from a freshly fetched origin/<base> (default dev; --base main for hotfixes) — never a stale local ref. ws remove refuses to destroy uncommitted or unpushed work without --force; claim --worktree is fail-closed (rolls the claim back if the worktree can't be created). Config: BUS_WORKTREE_ROOT overrides the location, BUS_REPO_DIR the target repo.

4. Co-authoring one file (the huddle)

A huddle lets several agents author one shared branch together. A single write-pen (a Redis mutex) means only its holder commits at any moment; the pen changes hands, so the code has many authors but never two writers at once. A done-gate requires everyone to sign off at the final commit before close.

# alice opens a huddle on issue 42 and holds the pen
./bus huddle open  --as alice --issue 42
./bus huddle join  --as bob   --issue 42          # admit bob as a participant

# alice writes in her per-driver worktree, then checkpoints (commit + push, one command)
cd "$(ls -d ../*-worktrees/huddle-42-alice)"      # (created by the first checkpoint)
./bus pen checkpoint --as alice --issue 42        # creates the worktree on first run
#   ... edit files ...
./bus pen checkpoint --as alice --issue 42        # commit + push your work to the shared branch
./bus signoff        --as alice --issue 42        # sign off at the current tip
./bus pen pass       --as alice --issue 42 --to bob   # hand off (commits first)

# bob's worktree fast-forwards to alice's work; he builds on it, then signs off
./bus pen checkpoint --as bob --issue 42          # ff to alice's tip; edit; checkpoint
./bus signoff        --as bob --issue 42          # (this push makes alice's signoff stale — she re-signs)
./bus signoff        --as bob --issue 42 --block "the error path is untested"   # or hard-block instead
./bus unblock        --as bob --issue 42          # lift your own block

# close is GATED: every present participant must have signed off at the CURRENT tip, no open block
./bus huddle close   --as alice --issue 42        # advances the issue to status:pr-open

Dynamic lead: ./bus pen take --as bob --issue 42 --reason "<evidence>" challenges for the pen. It force-takes only from an absent driver (and only after a grace window); a present driver keeps the pen until it pen passes (concede) or pen denys. Pen/huddle ops refresh presence, so an actively-driving agent is never force-taken. Any new push dismisses stale sign-offs, so a huddle can't be closed over code nobody re-approved.

5. Orchestrating from a Claude Code session (the coordinator)

The easiest way to run a team is to open a Claude Code session in this repo and talk to it in plain language. That session becomes the coordinator — it runs the bus commands for you. The worker agents (claude-2, codex-1, …) run in their own terminals under agent-loop.sh; your Claude session dispatches to them, watches, and gates their work. You never type a bus command yourself — you describe the outcome and Claude drives.

Option A — install the coordinator plugin (recommended). This repo ships a Claude Code plugin, messagebus-coordinator, and is its own plugin marketplace. From any Claude Code session:

/plugin marketplace add LJK-Consulting-Services/messagebus
/plugin install messagebus-coordinator@messagebus

The plugin adds a coordinate-message-bus skill (it auto-loads the coordinator role — no prompt to paste) and three slash commands: /bus-dispatch <task>, /bus-huddle <issue|task>, and /bus-board. It drives the ./bus CLI directly; no MCP server is required. To load it straight from a local checkout while developing, skip the marketplace and run claude --plugin-dir /path/to/messagebus.

Run the coordinator session from inside this repo (so ./bus and gh are on hand), or set BUS_GH_REPO=owner/repo in its environment.

Option B — no plugin, just paste the role. Start a Claude Code session in this repo and give it its role as your first message (or put it in CLAUDE.md):

You are the coordinator for a team of agents on the message bus in this repo.
The workers claude-2 and codex-1 are already running under scripts/agent-loop.sh.
Use the ./bus CLI (BUS_GH_REPO=owner/repo): create gh issues for the work, dispatch
tasks with `./bus send --from coordinator --to <agent> --topic issue-<N> "..."`,
watch with `./bus watch` and `./bus board`, pace with `./bus wait --as coordinator`,
and gate + merge the agents' PRs. Never poll --as an agent (it eats their messages).

Optional structured tools (MCP). scripts/bus-mcp.py is a thin MCP server that exposes the bus as typed tools (bus_send, bus_board, bus_tail, bus_huddle_status, …) if you'd rather the coordinator call typed tools than shell out. It's off by default. To turn it on, register it with an absolute path — from the repo root:

claude mcp add messagebus -- python3 "$PWD/scripts/bus-mcp.py"

The coordinator skill drives the ./bus CLI regardless, so the MCP server is a convenience, never required.

Then just type what you want, exactly like you'd talk to any Claude session:

Build a rate limiter for the API. Break it into gh issues, hand them to
claude-2 and codex-1 on the bus, then watch and gate their PRs.
Where are the agents on #42? Show me the board and the recent bus traffic.
Have claude-2 and codex-1 co-author the fix for #42 in a huddle — one drives,
the other reviews and calls out weak code. Gate it and merge once both sign off.
codex-1 looks stuck on #42. Check its worktree, and if it's dead, reap the lock
and hand the issue to claude-2.

Behind each of those prompts, your coordinator session runs the real commands — ./bus send, ./bus watch, ./bus board, ./bus wait --as coordinator, gh pr merge, etc. (the ## Full command reference below). You describe the goal; Claude translates it into bus + gh calls, waits for the agents, reviews their PRs, and reports back.

This entire repo was built exactly this way: a human typing prompts to a coordinator Claude session, which dispatched, watched, and gated a team of worker agents — including the agents co-authoring the bus's own features via the huddle.

Two things the coordinator relies on (and you can lean on in your prompts):

  • Watch, don't poll. bus watch is read-only; bus wait --as coordinator uses a non-agent id — neither consumes a worker's messages.
  • Truth lives in git + gh, not chat. Across a /clear or restart, the coordinator trusts git refs, then gh issue status:* labels, then the latest issue comment — bus chatter is ephemeral. Point a fresh coordinator at those.

Full command reference

# messaging
bus send     --from A [--to all] [--topic T] [--reply-to ID] [--kind msg] "text" | -
bus poll     --as A [--topic T]              # new messages for you; advances your cursor
bus wait     --as A [--timeout 30] [--topic T] [--reply-to ID]   # block until one arrives
bus tail     [-n 20]                         # recent traffic; cursor untouched
bus history  [-n 1000]                       # full room log
bus watch    [-n 10] [--topic T]             # live-follow (read-only observer)
bus thread   ID                              # print the full reply_to thread of a message
bus inbox    --as A                          # peek unread directed msgs / questions (no cursor move)
bus prune    [--keep 5000] [--force] [--dry-run]  # trim the room to newest N (refuses if it drops unread)
bus join     --as A                          # register presence; cursor to "now"
bus agents                                   # who is present
bus drain    --as A [--force]                # graceful exit: release your pens + presence (claims kept)

# task coordination (needs BUS_GH_REPO + gh)
bus claim    --as A --issue N [--ttl 28800] [--worktree] [--base dev]
bus renew    --as A --issue N [--ttl 28800]  # extend your claim TTL
bus release  --as A --issue N                # release your claim
bus reap     [--release N]                   # list stale locks; free one you've verified
bus status   --as A --issue N --set <status:open|claimed|pr-open|merged|deployed|verified>
bus board                                    # gh issue + status + lock holder + presence
bus init                                     # create the status:* labels in BUS_GH_REPO

# worktree isolation (needs git)
bus ws create --as A --issue N [--base dev] [--type feat] [--allow-stale] [--allow-nested]
bus ws path   --issue N                      # print the worktree path
bus ws list                                  # all worktrees: dirty / unpushed / present
bus ws remove --as A --issue N [--force]

# huddle — co-authored code (needs git + gh)
bus huddle open   --as A --issue N [--base dev] [--ttl 28800] [--allow-stale]
bus huddle join   --as B --issue N
bus huddle status --issue N
bus huddle close  --as A --issue N [--force]
bus pen status     --issue N
bus pen checkpoint --as A --issue N          # holder: commit + push WIP to the shared branch
bus pen pass       --as A --issue N --to B   # hand off (commits first; aborts if the push fails)
bus pen take       --as B --issue N --reason "<evidence>"   # challenge / force-take from absent driver
bus pen deny       --as A --issue N --reason "<why>"        # driver rejects a challenge
bus signoff  --as A --issue N [--block "<reason>"]   # sign off at the current tip, or hard-block
bus unblock  --as A --issue N                        # lift your own block

# diagnostics
bus doctor                                   # check Redis + gh

Global flags (before the subcommand): --room <name> (default main), --url <redis url> (default redis://127.0.0.1:6379/0), --json (machine output).

--topic <t> (poll/wait/watch) scopes reading to one thread by exact match — cut the noise while working one issue (--topic issue-42). The cursor still advances past non-matching messages. A __SHUTDOWN__ always passes the poll/wait topic filter, so a topic-scoped agent can't miss the kill-switch.


Autonomy (agents run without you between messages)

Two ways to keep an agent reacting to the bus:

Stop hook (Claude Code). Wire it into the agent's settings.json so a session re-invokes itself when a message for it lands:

{
  "hooks": {
    "Stop": [
      { "hooks": [ { "type": "command",
        "command": "BUS_AGENT=claude-1 /ABS/PATH/messagebus/hooks/stop-hook.sh" } ] }
    ]
  }
}

agent-loop.sh (Codex / any hookless CLI). It blocks on bus wait, pipes each delivered message to your agent command on stdin, and — crucially — re-invokes the agent until it signals a terminal handoff (touch "$BUS_DONE_MARKER", i.e. work pushed + review requested, or a blocker posted), so an agent completes a multi-step task in one wake instead of stalling after step 1:

# Codex:
./scripts/agent-loop.sh codex-1 codex exec --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check "$(./scripts/agent-bootstrap.sh codex-1)"
# headless Claude:
./scripts/agent-loop.sh claude-2 claude -p --append-system-prompt "$(./scripts/agent-bootstrap.sh claude-2)"

scripts/agent-bootstrap.sh <agent-id> renders the agent's system prompt (operating rules + the MB-SPEED speed protocol) and registers its presence.

Codex needs --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check (both flags, as shown above). Without them Codex runs in its default network + filesystem sandbox: Redis rejects with Operation not permitted, gh reports an invalid token, and github.com won't resolve. The agent can still edit and commit locally but silently cannot push or touch the bus — which looks exactly like it is "stalling." Claude Code does not need these flags.

  • Stop an agent for good: touch .bus-state/stop-<agent-id> (or stop-all), or send it a message whose body is exactly __SHUTDOWN__.
  • examples/echo-agent.sh is a trivial stdin→bus-reply agent to prove the loop drives a real external process before wiring a real CLI.

Prove the driver first

# terminal 1: drive the echo agent + watch the bus
./scripts/agent-loop.sh echo-1 ./examples/echo-agent.sh echo-1 &
./bus watch
# terminal 2 (after echo-1 shows in `./bus agents`):
./bus send --from operator --to echo-1 "ping"     # its reply appears live in the watch
touch .bus-state/stop-echo-1                       # stop it

Security / trust model

This is a single-operator local tool. It assumes every agent on the bus and every local user of the machine is trusted. There is deliberately no authentication — the "homegrown simple" tradeoff. Sharp edges:

  • Identity is self-asserted. --from/--as are whatever the caller types. Any agent can impersonate operator or a peer, bus join --as <victim> to reset another cursor, or bus poll --as <victim> to advance it. Don't put an untrusted agent on the bus.
  • --to all __SHUTDOWN__ is a fleet kill-switch. One message stops every agent. Intended for the operator; available to any agent on the bus.
  • Maintenance commands are not an auth boundary. bus prune, bus reap --release and bus drain are explicit manual commands (never automatic), but any local caller can invoke them. drain --as <victim> drops that agent's write pen and presence, so it is exactly as trustworthy as the self-asserted --as above — it takes no claim locks, but it is not a permission check. Note what clearing presence means: the huddle done-gate only requires present participants to sign off, so draining an agent also drops it out of that check. Every drain emits an EV_DRAIN event precisely so it can't be a quiet way to vanish from a gate — bus events --type drain.
  • With AOF on, message bodies are written to disk. Durability (below) means every message the bus has carried — untrusted agent chatter included — lands in the append log under BUS_REDIS_DIR (default ~/.bus/redis). start-redis.sh creates that directory 0700, but a Redis you started some other way will use whatever its own config says.
  • Message content is untrusted input to your agents. A peer's body is fed into each agent's context. The Stop hook fences it in an "UNTRUSTED" boundary and prompts/agent-system.md tells agents not to obey embedded instructions, but a capable model can still be prompt-injected — run wired agents with restricted tool permissions if any message source isn't fully trusted.
  • Redis is unauthenticated on loopback. start-redis.sh binds 127.0.0.1. Any local user can read/forge bus traffic. Don't expose the Redis port off-host.

Injection is not a gap: bus shells out to gh/git via argv lists (no shell), the Stop hook escapes message bodies through json.dumps, untrusted content printed to a terminal is control-char escaped, and agent ids / issue numbers are charset/type validated.


Config

Env Default Meaning
BUS_REDIS_URL redis://127.0.0.1:6379/0 Redis connection (also read by start-redis.sh)
BUS_REDIS_DIR ~/.bus/redis where start-redis.sh's own daemon keeps its append log
BUS_ROOM main default room/stream
BUS_GH_REPO — owner/repo the issue/label/huddle commands target
BUS_REPO_DIR cwd's repo the git repo worktrees/huddles operate on
BUS_WORKTREE_ROOT ../<repo>-worktrees where per-agent worktrees live
BUS_AGENT — agent id, used by the Stop hook
BUS_WAIT_SECS 20 (hook) / 60 (loop) how long each bus wait blocks
BUS_MAX_CONTINUE 6 agent-loop.sh self-continue cap (re-invocations per message)
BUS_MAX_TURNS 200 Stop-hook consecutive auto-continue cap (runaway guard)
BUS_DONE_MARKER set by the loop file an agent touches to signal its turn is complete

Redis keys (for the curious): bus:stream:<room>, bus:cursor:<room>:<agent>, bus:presence:<room>:<agent> (90s TTL), bus:lock:issue:<N>, bus:worktree:issue:<N>, bus:huddle:issue:<N>, bus:pen:issue:<N>, bus:signoff:issue:<N>, bus:block:issue:<N>.

Durability

Redis holds live coordination state, not just a replayable chat log — claim locks, the write pen, huddle metadata, every agent's cursor. A volatile server loses all of it on restart, which is how you get an issue claimed twice or a pen dropped mid-huddle. start-redis.sh therefore turns on AOF persistence (appendfsync everysec, bounding loss to ~1s where an RDB snapshot would lose everything back to the last save point), and bus doctor reports the mode it finds. A WARN there means a restart will drop that state.

start-redis.sh only ever reconfigures a loopback Redis — if BUS_REDIS_URL points at a shared or remote server, it reports the mode and leaves that server's config alone.

Clients ride out a blip on their own (redis-py retries with backoff), and the long-lived loops — bus wait, bus watch — rebuild the connection and resume from where they were when the server actually restarts under them. bus wait still honours its --timeout: reconnect time is capped by the caller's budget, so a Redis that never comes back can't hang a turn.

Retrying makes writes at-least-once: a command whose reply is lost is re-sent, so a rare duplicate message is possible. That is the right trade for a coordination log, but it has a sharp edge — a re-sent SET NX finds the key its own first attempt created and reports that it lost the race. bus claim and bus huddle open both handle that explicitly (they check whether the value in the lock is their own) rather than half-completing.

Retention is cursor-aware: bus prune trims by a boundary checked against every agent's cursor and refuses to drop anything unread (--dry-run reports the lag and writes nothing; --force overrides and says so on the event stream). The stream is deliberately never bounded with XADD MAXLEN, which drops the oldest messages by count with no idea what anyone has read yet.

Status transitions

bus status moves an issue along the linear pipeline open → claimed → pr-open → merged → deployed → verified. Forward moves (any distance) are always allowed; a no-op re-set of the current label is skipped (no duplicate comment); a backward move is refused unless you pass --force. Two "back to work" edges are always legal without --force: claimed → open (abandon a claim) and pr-open → claimed (PR closed, back to building). If the current label can't be read, the move is refused (even with --force — --force overrides legality, not an unreadable state; retry when gh is reachable). This stops a fat-finger from sending a verified issue back to open, without constraining real forward progress.

bus claim and huddle-open also set status:claimed, but they run inside agent loops where a hard refusal could wedge a driver mid-task. So on those acquire paths a backslide (claiming an issue already past claimed) is a warning, not a block — the explicit bus status command is the one place a backward move is refused. huddle-close advances to pr-open (forward by construction).

Tests

Pure-function tests (no Redis or gh needed) for the status-transition gate:

python -m unittest discover -s tests    # from the repo root; needs redis-py importable

About

Homegrown durable multi-agent message bus (Redis Streams) for coordinating Claude/Codex agent sessions via a shared bus + GitHub-issue state machine

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages