Gamebrew

Let AI agents playtest your Unity game — while you build it.

Gamebrew is a Unity MCP that plugs AI agents into your game-dev loop as hands-on playtesters. They drive a running Unity Editor — walk the player, orbit in to inspect a prop, capture what they see, read live scene state, run tests — so bugs and rough edges surface as you develop. One loopback HTTP port. No cloud, no account, no SDK lock-in.

_{Unity Editor ⇄ MCP ⇄ your agent — brew a playtester out of any Unity 6 project.}

---

✨ What an agent can do

🚶 Move & navigate	walk the player, NavMesh-path to any object by name
👁 Look & orbit	aim the camera, orbit a free camera around a prop to inspect it
📸 See	capture the game view to a PNG the agent then Reads
🔍 Inspect	dump any live component's private fields to JSON; read the scene tree
⌨️ Interact	send Input-System keyboard / mouse in Play Mode
🧪 Test	run EditMode tests, gate on compilation
🧩 Extend	add your own play.* verbs — auto-allowlisted, one case away
🎟️ Share safely	a lease dispatcher lets several agents take turns on the one Editor

🚀 Quickstart

Two halves: the Unity Editor bridge, and a Node MCP server your agent talks to.

1 · Unity — drop unity-package/ into your project, install the two UPM packages (Input System + Newtonsoft JSON), then Tools → Unity Bridge → Start.

2 · MCP

cd mcp && npm install

Point your MCP client (Claude, Cursor, …) at mcp/src/index.mjs — config below.

3 · Drive it — enter Play Mode, then:

cd mcp
./browse.sh my-agent play.navTo     '{"target":"Environment/Player","standoff":2}'
./browse.sh my-agent play.orbitView '{"target":"Environment/Player","pitch":18,"yaw":45,"distance":2.5}'
./browse.sh my-agent capture shot     # → Read the printed PNG path to SEE it

Your agent is now looking around your game.

🧠 Why Gamebrew

• Self-hosted & tiny — ~20 Editor C# files + one Node server. No cloud service, no account, loopback only.
• Agent-native — speaks MCP, so it drops straight into Claude, Cursor, or any MCP client as bridge_* tools.
• Multi-agent by design — a cooperative lease lets several agents share one Editor without desyncing the world.
• Safe by default — a default-deny allowlist keeps browsing agents to gameplay + inspection; lifecycle stays with you.
• Yours to extend — game-specific verbs are one case away, and the movement layer sits behind clean, subclass-able seams.

🤖 For AI agents

Driving Gamebrew yourself? Read AGENTS.md — the effective-use playbook, written in agent-imperative style. The three rules that matter most:

1. See by capturing and Reading the PNG; trust dump over screenshots for anything numeric — a dim frame lies, live component fields don't.
2. Orbit props to judge them. One head-on capture hides the silhouette — and play.orbitView writes Logs/orbit.png (a free camera), not what a plain capture reads. Use the orbit verb.
3. Take turns and release. Acquire the lease, drive, release — never sit on it.

The full loop, copy-paste command patterns, and a symptom→fix table are in AGENTS.md.

Organizing a playtest crew (recommended, optional)

Gamebrew is unopinionated about your agent stack — but here's a structure that works well for AI-assisted playtesting. It's how the tool was dogfooded: an orchestrator plus two browse specialists found real gameplay + visual bugs in a running game.

Role	Drives the bridge?	Job
Orchestrator	directly, :8787	Owns lifecycle — enter/exit Play Mode, load scenes, run tests, apply fixes — and synthesizes + verifies findings. Never lets a browsing agent do lifecycle.
Playtester agent	lease window, :8788	Evaluates gameplay logic · legibility · edges: can a new player tell what to do? do interactions read? any soft-locks or exploits?
Visual / tech-art agent	lease window, :8788	Evaluates silhouettes · materials · lighting · HUD against your art bible — orbits props, dumps material state.
Implementer agent(s)	no	Fix what's confirmed (edit code); the orchestrator re-verifies with a fresh browse.

The loop

1. The orchestrator enters Play Mode and dispatches the browse specialists — one lease turn each (one game world = one driver at a time).
2. Each agent runs the browse loop: acquire → navigate / orbit / capture / dump → report → release.
3. The orchestrator verifies every finding against the code or capture before acting — agents can be wrong (in dogfooding, an orbit refuted a "plain table" verdict that came from a bad head-on angle). Confirm, don't trust.
4. Implementers fix what's confirmed; the orchestrator recompiles and re-browses to prove the fix.
5. Feel is the human's. Agents settle understandable and correct; "does it look / feel good" routes to you.

Start small — one orchestrator + one playtester is already useful; add the visual agent when art matters, and scale to a full crew for a deep eval. Per-agent driving rules are in AGENTS.md.

---

Architecture

Two halves. The Unity Editor side hosts an HTTP listener; the Node side speaks MCP to your agent client and optionally serializes multiple agents through a lease dispatcher.

   ┌─────────────┐   MCP/stdio    ┌───────────────────────┐
   │  AI agent   │ ─────────────▶ │  Gamebrew MCP server  │   (mcp/src/index.mjs)
   │ (Claude,    │                │  bridge_* tools       │
   │  Cursor, …) │ ◀───────────── │                       │
   └─────────────┘                └───────────┬───────────┘
                                              │ POST { command, args }
                                              ▼
                                   http://127.0.0.1:8787/           ← Editor HTTP bridge
                                   ┌───────────────────────┐          (unity-package/…/BridgeServer.cs)
                                   │  Unity Editor (Play)  │
                                   │  CommandRouter switch │ ── drives ──▶  your running game
                                   └───────────────────────┘

   ── Multiple agents? put the lease DISPATCHER in front of 8787 ──────────────────────────

   agent A ─┐                        ┌──────────────────────┐
   agent B ─┼── POST /lease, /cmd ─▶ │ dispatcher  :8788    │ ─ only the lease HOLDER ─▶ :8787
   agent C ─┘   (take turns)         │ lease + queue +      │
                                     │ default-deny allow   │
                                     └──────────────────────┘

• Editor bridge (:8787) — a HttpListener bound to 127.0.0.1. A CommandRouter dispatches {command, args} to coordinators (scene, gameobject, component, play-mode input, navigation, orbit camera, capture, component dump, tests).
• MCP server (mcp/src/index.mjs) — exposes the bridge as bridge_* MCP tools; POSTs a bare {command,args} body to :8787. This is the orchestrator / lifecycle path.
• Lease dispatcher (mcp/src/dispatcher.mjs, :8788) — optional, additive. Sits in front of :8787 and serializes turns so parallel agents don't interleave and desync the one shared world. Enforces a default-deny browse allowlist.
• mcp/browse.sh — a thin shell wrapper an agent uses to acquire/drive/release a lease.

---

Install — the Unity half

1. Copy the whole unity-package/ into your project (e.g. Assets/Gamebrew/), keeping the Editor/ and Runtime/ subfolders intact. Unity regenerates the .meta files. Both assemblies (Gamebrew.Bridge.Editor, Gamebrew.Bridge.Runtime) are self-contained — the Editor asmdef references the Runtime asmdef by name, which resolves once both folders are present.

2. Requirements (UPM packages) — install both via Window → Package Manager:

   • Input System — com.unity.inputsystem (assembly Unity.InputSystem). Referenced by both assemblies; drives play.sendKey / play.mouse* and the input relay. Set Edit → Project Settings → Player → Active Input Handling to Input System (or Both).
   • Newtonsoft Json — com.unity.nuget.newtonsoft-json (Newtonsoft.Json.dll). Used by the Editor assembly's JSON command envelope. (The Runtime assembly needs neither Newtonsoft nor any other package.)

   The built-in AI (NavMesh) module (UnityEngine.AIModule) is used by play.navTo for path planning and needs no package. Baking a runtime NavMesh is your game's job — see the BridgeNavMeshBaker seam in Status.

3. Wire the movement seams (optional, only if you use play.move/play.navTo/play.aimAt). The play-driver verbs locate game-agnostic abstract components — BridgeLocomotor, BridgeCameraRig, BridgeNavMeshBaker (in Runtime/BridgeSeams.cs) — via FindAnyObjectByType. Subclass each on your player / camera / navmesh baker and forward to your own controllers (a 3-line adapter each; example in BridgeSeams.cs). If a seam is absent, the matching verb returns a clean "No <seam> in the scene" error — it never throws, and every other verb still works.

4. In Unity: Tools → Unity Bridge → Start. The listener comes up on http://127.0.0.1:8787/. It survives domain reloads (remembers your Start/Stop intent in SessionState).

Install — the MCP / Node half

cd mcp
npm install

Register the server with your MCP client (Cursor .cursor/mcp.json, Claude Code MCP config, …):

{
  "mcpServers": {
    "gamebrew-bridge": {
      "command": "node",
      "args": ["/absolute/path/to/Gamebrew/mcp/src/index.mjs"]
    }
  }
}

Restart / reconnect the client after config changes. If the client shows a stale tool count after you add tools, regenerate descriptors with npm run export-mcp-tools and toggle the server off/on.

Environment variables

Variable	Default	Purpose
UNITY_BRIDGE_URL	http://127.0.0.1:8787/	Editor bridge endpoint (also read by the dispatcher).
GAMEBREW_BRIDGE_URL	—	Legacy alias for UNITY_BRIDGE_URL.
UNITY_BRIDGE_CONTROL_FILE	.unity-bridge/control.json	Start/stop the bridge when HTTP is down (Editor polls this file).
UNITY_BRIDGE_MENU	Tools → Unity Bridge → Start	Shown in error messages.
DISPATCHER_PORT	8788	Lease dispatcher port.
DISPATCHER_LEASE_TTL	90 (seconds)	Idle auto-release timeout.
GAMEBREW_BROWSE_RUN	default	Per-session nonce so two parallel runs of the same agent id don't fight over one lease.

---

The agent browse workflow

The bridge cannot stream video. An agent sees the world by capturing a still and Reading the PNG; it reads exact state with a component dump. A typical single-agent loop:

cd mcp
# 1. ACQUIRE the lease (only granted if free; otherwise you're queued)
./browse.sh my-agent acquire

# 2. NAVIGATE / LOOK  (play.* verbs; require the Editor to already be in Play Mode)
./browse.sh my-agent play.navTo     '{"target":"Environment/Player","standoff":2}'
./browse.sh my-agent play.orbitView '{"target":"Environment/Player","pitch":18,"yaw":45,"distance":2.5}'

# 3. CAPTURE → then Read the printed path to SEE the frame
./browse.sh my-agent capture before        #  → Logs/browse-my-agent-before.png

# 4. DUMP live state as JSON (beats squinting at a low-contrast capture)
./browse.sh my-agent dump Environment/Player Rigidbody

# 5. RELEASE promptly when done — do NOT sit on the lease
./browse.sh my-agent release

⚠️ The orbitView-vs-capture gotcha

There are two different cameras and it trips people up:

• play.orbitView moves a free inspection camera and writes its own frame to Logs/orbit.png (a fixed path). It does not move the player rig.
• capture (editor.captureGameView) reads whatever the player / main game camera currently sees, to the path you name.

So after an orbitView, Read Logs/orbit.png to see the orbit result — a plain capture will show the player's view, which orbitView never moved. To judge a prop's silhouette, orbit it from a few yaws (yaw:0 front, yaw:60 3/4, yaw:120 side) and Read each Logs/orbit.png. One head-on navTo capture is not enough to say a prop "reads".

---

Safety model

• Loopback only. The Editor bridge binds 127.0.0.1:8787; the dispatcher binds 127.0.0.1:8788. Nothing listens on a routable interface.
• No authentication. This is a local dev tool. Reflection can invoke component methods — run it only on projects you trust.
• Default-deny browse allowlist. The dispatcher's /cmd forwards a command only if it is a play.* verb (whole namespace) or an exact member of a small read/inspect/perceive set. Everything else — Play-Mode toggle, scene load/save, tests, structural mutation, killing the bridge — is rejected without reaching the Editor. (See mcp/src/browseAllowlist.mjs.)
• Orchestrator owns lifecycle. Doctrine: the orchestrator drives Play enter/exit, scene I/O, tests, compiles, and structural mutation on the direct :8787 path. Agents get lease windows for gameplay only on :8788. An agent physically cannot enter Play Mode or mutate the scene graph — so ask the orchestrator to enter Play Mode before your window.
• The lease is COOPERATIVE, not enforced. The dispatcher gives advisory turn-taking + observability, not access control. :8787 stays open, so a client that talks to it directly bypasses the lease entirely. The lease works only because everyone agrees to go through :8788. The orchestrator must GET :8788/status and not drive while an agent holds the lease. Do not repoint the MCP client's UNITY_BRIDGE_URL at :8788 — the client POSTs a bare body to /, which the dispatcher doesn't route (it only knows /lease/*, /cmd, /status); keep the MCP path on :8787.

Lease etiquette (the short version)

1. Acquire before driving; you're queued if it's held.
2. Drive, then release promptly — never hold more than 1–2 minutes.
3. Idle leases auto-reclaim at DISPATCHER_LEASE_TTL (90s) and pass to the next agent.
4. Two-phase promotion: on release the next queued agent is reserved and must re-acquire within ~12s to confirm, so a dead agent can't block the queue for the full TTL.
5. Same agent id in two parallel runs → give each a distinct GAMEBREW_BROWSE_RUN nonce.

Validate the whole lease/allowlist machine with the self-contained test (spins up the dispatcher + a stub Editor, asserts turn-taking + allowlist):

cd mcp && npm run test:dispatcher     # → "37 passed, 0 failed"

---

Command vocabulary (generic verbs)

<target> is a scene path like Environment/Player. play.* verbs require Play Mode.

Command	Args	Purpose	Browse-safe?
ping	{}	liveness / Unity version	✅
editor.getBridgeStatus	{}	bridge health	✅
editor.getPlayMode / editor.setPlayMode	{} / {playing}	read / set Play Mode	read ✅ / set 🚫
editor.captureGameView	{path}	write a PNG of the game camera	✅
editor.getConsoleLogs	{level?, count?}	recent console output	✅
editor.executeMenuItem	{path}	run a menu item	🚫
editor.stopBridge	{}	stop the listener	🚫
scene.getHierarchy	{}	active scene tree	✅
scene.open / scene.save	{path} / {}	scene I/O	🚫
gameobject.find / gameobject.get	{name} / {path}	read a GameObject	✅
gameobject.create / .rename / .reparent / .setActive	varies	mutate the scene graph	🚫
component.add / .setProperty / .getProperty / .callMethod	varies	mutate / invoke components	🚫
debug.dumpComponent	{type} or {path, component}	live component fields → JSON	✅
play.move / play.moveTo	varies	move the player	✅
play.navTo / play.navTo.status	{target, standoff?, …} / {}	NavMesh-walk the player to an object	✅
play.viewObject	{target, standoff?}	frame an object from the player's eye	✅
play.orbitView	{target, pitch, yaw, distance}	orbit a free camera → Logs/orbit.png	✅
play.aimAt / play.aimAtObject / play.setLook	varies	aim / look	✅
play.sendKey	{key, action?, shift?, …}	Input-System keyboard in Play Mode	✅
play.mouseLook / play.mouseButton	varies	mouse look / click	✅
test.spawnWall / test.despawnWall	varies	test obstacles	🚫
editor.runTests / editor.getTestResults / editor.waitForCompile	varies	EditMode tests / compile gate	🚫 (orchestrator)

Adding your own game-specific verbs

New play.* verbs are auto-allowed by the dispatcher — you do not touch the allowlist. To add one:

1. Write a static coordinator in the Editor assembly (e.g. MyGamePlayCoordinator.cs).
2. Add a case "play.myVerb": return MyGamePlayCoordinator.Do(args); in CommandRouter.Execute (there is a clearly-marked extension point where the game verbs used to be).
3. Optionally add a matching bridge_* MCP tool in mcp/src/index.mjs.

A minimal, self-contained worked example (a single play.example.* verb) lives in examples/custom-verb/ — copy the pattern for adding your own play.* verbs.

---

Repo layout

Gamebrew/
├── README.md
├── LICENSE                          ← MIT
├── assets/gamebrew-logo.svg
├── unity-package/                   ← drop this whole folder into your project (keep both subfolders)
│   ├── Editor/Bridge/               ← the Editor-only assembly (Gamebrew.Bridge.Editor)
│   │   ├── Gamebrew.Bridge.Editor.asmdef   ← refs Gamebrew.Bridge.Runtime by name
│   │   ├── BridgeServer.cs          ← HttpListener on 127.0.0.1:8787 + Tools menu
│   │   ├── CommandRouter.cs         ← the command switch (+ game-verb extension point)
│   │   ├── MainThreadDispatcher.cs, CompileCoordinator.cs, ConsoleLogBuffer.cs …
│   │   ├── GameObjectResolver.cs, ComponentResolver.cs, JsonCoercion.cs
│   │   ├── DebugDumpCoordinator.cs (debug.dumpComponent), GameViewCapture.cs
│   │   ├── Play{Move,Nav,Orbit,View,Look}Coordinator.cs, PlayModeInputCoordinator.cs
│   │   └── EditorPlayModeCoordinator.cs, Test{Run,Obstacle}Coordinator.cs
│   └── Runtime/                     ← the companion runtime assembly (Gamebrew.Bridge.Runtime)
│       ├── Gamebrew.Bridge.Runtime.asmdef   ← refs Unity.InputSystem; all platforms
│       ├── BridgeInputRelay.cs      ← generic Play-Mode keyboard injector (play.sendKey)
│       ├── BridgeSimTime.cs         ← generic deterministic-dt seam (#if UNITY_EDITOR)
│       └── BridgeSeams.cs           ← BridgeLocomotor / BridgeCameraRig / BridgeNavMeshBaker
├── mcp/
│   ├── package.json                 ← "gamebrew-mcp"
│   ├── browse.sh                    ← agent lease wrapper
│   ├── src/index.mjs                ← MCP stdio server (bridge_* tools)
│   ├── src/dispatcher.mjs           ← lease dispatcher on :8788
│   ├── src/browseAllowlist.mjs      ← default-deny browse-safe command set
│   ├── src/playModeUtils.mjs        ← enter/exit Play Mode helpers
│   └── scripts/{test-dispatcher,export-mcp-tool-descriptors}.mjs
└── examples/
    └── custom-verb/                 ← "how a game adds its own play verbs" (worked example)
        ├── README.md
        └── ExampleVerbCoordinator.cs

---

Status — what's decoupled vs TODO

Snapshot. Gamebrew was extracted from a real Unity game. The unity-package/ is now self-contained — an Editor assembly plus its bundled Gamebrew.Bridge.Runtime — and every referenced symbol resolves to the package, UnityEngine/UnityEditor, Newtonsoft.Json, System.*, or the Input System, so it's expected to compile on drop-in once the two UPM packages above are present. The remaining game hooks below are optional and marked TODO(decouple) in-source.

Decoupled and generic: the whole transport (HttpListener, CommandRouter, main-thread dispatch), scene/gameobject/component verbs, debug.dumpComponent, all play.* navigation / orbit / view / look coordinators, capture, tests, the console buffer — and the entire Node half (MCP server, lease dispatcher, allowlist, browse wrapper). The bundled runtime assembly holds the three game-agnostic pieces the Editor coordinators need: BridgeInputRelay (keyboard injection), BridgeSimTime (deterministic dt), and the BridgeLocomotor / BridgeCameraRig / BridgeNavMeshBaker seams. The Node lease + allowlist logic passes its self-test 37/37.

Marker / seam	What it is	What you do
Runtime/BridgeSeams.cs	Abstract MonoBehaviour seams standing in for a game's player controller / camera rig / navmesh baker. The play-driver verbs invoke them directly.	Subclass each on your player / camera / baker (3-line adapters) only if you use play.move / play.navTo / play.aimAt / test.spawnWall. Absent seam → clean error, no crash.
CommandRouter.cs (extension point)	The origin game's own play.* verb cases were removed; a marked extension point remains.	Add your own play.* cases here (see examples/).
CommandRouter.cs (time.advance)	Drove a game clock. Body stubbed to return an error.	Wire to your own time system, or delete the case.
CommandRouter.cs (perceive.aimHit / perceive.locate)	Relied on a game perception/interaction helper. Bodies stubbed.	Port a small perception helper, or delete the cases.
CommandRouter.cs (editor.getPlaytestReport, playtest.clear)	Read a game acceptance-test log. Bodies stubbed.	Wire to your own test log, or delete the cases.
PlayNavCoordinator.cs	Default nav standoff was a game interaction range; hardcoded to 3.25f * 0.9f.	Repoint at your own range if you have one.
ComponentResolver.cs & DebugDumpCoordinator.cs	Simple-name type resolution preferred a game namespace; replaced with a PreferredNamespacePrefix const defaulting to "" (off).	Set it to your game's root namespace for the convenience, or leave empty.

---

_{Built for AI agents that need to see and play, not just read code. • MIT}