Skip to content

Commit 1f47239

Browse files
rgfaberrgfaber
committed
serve/serve -daemon: add -exec, a per-call computed reply
-reply/-echo both answer from something already known at registration time -- a fixed payload, or the caller's own payload bounced back. Neither this control protocol nor the daemon itself had any way to hand a registration a live answer. -exec closes that gap: the given shell command runs once per inbound CALL, gets the call's payload as one JSON document on stdin, and whatever it writes to stdout becomes the reply (empty stdout replies null). Shared between serve's direct path and serve -daemon via a new BuildReplyHandler in internal/daemon/exec_handler.go, so both compute a handler identically rather than two copies drifting apart. A non-zero exit, a timeout (-exec-timeout, 10s default), or invalid JSON on stdout all become a normal BOLT#4 ERROR reply to the caller -- connection.CallHandler's own contract already guarantees a handler error (or even a panic) never crashes the shared serve loop, confirmed by reading macula-go's own ServeForever/buildCallReply before relying on it. That guarantee matters more here than for -reply/-echo: every procedure a daemon serves shares one serveSession, so a hung exec would otherwise block every OTHER registered procedure too. Verified live against the real demo fleet, not just typechecked: a Node script doubling its input, called three times with three different values, each producing a genuinely per-call-computed reply; three sibling registrations deliberately made to fail three different ways (non-zero exit, invalid JSON on stdout, a timeout) each correctly answered their own caller with an ERROR while the working procedure kept computing correctly throughout, and the daemon's own status afterward showed all four still registered and healthy; the one-shot (non-daemon) serve -exec path separately confirmed too. Also verified the -exec/-reply/-echo mutual-exclusion check rejects at parse time, before ever registering anything.
1 parent a9bdcda commit 1f47239

6 files changed

Lines changed: 297 additions & 34 deletions

File tree

‎README.md‎

Lines changed: 33 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,9 @@ is that throwaway program, built once and kept.
5353
the provider-role counterpart to `call`. Same `-direct`/`-cert-chain`
5454
flags as `call`, plus `-require-ucan-issuer` to gate the procedure. `
5555
-daemon` registers it with a running `daemon` instead (see below) —
56-
persistent, many calls, no exit after the first.
56+
persistent, many calls, no exit after the first. `-exec <command>`
57+
computes the reply per call (JSON in on stdin, JSON out on stdout)
58+
instead of a fixed `-reply`/`-echo`.
5759
- **`pubsub watch` / `pubsub publish`** — subscribe and stream events as
5860
newline-delimited JSON, or publish one event and exit. `watch -daemon`
5961
taps into a daemon's own subscription instead of subscribing itself;
@@ -213,6 +215,13 @@ macula-cli daemon start station-de-frankfurt.macula.io:4433 &
213215
# Register a procedure -- answers as many calls as arrive, not just one.
214216
macula-cli serve -daemon -reply '{"pong":1}' my.echo
215217

218+
# Or -exec: a real, per-call computed reply instead of a fixed one --
219+
# the script gets the call's JSON payload on stdin, its stdout becomes
220+
# the reply. A non-zero exit, a timeout, or invalid JSON on stdout all
221+
# become a normal ERROR reply to the caller, not a crash of the daemon
222+
# or any other procedure it's serving.
223+
macula-cli serve -daemon -exec './double.sh' my.double
224+
216225
# From anywhere else: ordinary "call" reaches it exactly like any other
217226
# advertised procedure -- the daemon is invisible to callers. Or route
218227
# through the daemon's own connection instead of dialing fresh:
@@ -260,9 +269,29 @@ assumed from `os.MkdirAll` succeeding — refused live against a
260269
deliberately world-writable planted directory during testing.
261270

262271
`serve -daemon`'s registration flags (`-direct`, `-cert-chain`,
263-
`-require-ucan-issuer`, `-reply`/`-echo`) are the same ones the one-shot
264-
`serve` takes; it just sends them to the daemon instead of dialing the
265-
mesh itself and takes no `<host[:port]>` (the daemon already has one).
272+
`-require-ucan-issuer`, `-reply`/`-echo`/`-exec`/`-exec-timeout`) are the
273+
same ones the one-shot `serve` takes; it just sends them to the daemon
274+
instead of dialing the mesh itself and takes no `<host[:port]>` (the
275+
daemon already has one).
276+
277+
**`-exec` is the only registration mode that computes anything per
278+
call** — `-reply`/`-echo` both answer from something already known at
279+
registration time (a fixed payload, or the caller's own payload bounced
280+
back), since neither this control protocol nor the daemon itself has any
281+
other way to hand a registration a live answer. `-exec` runs the given
282+
shell command once per inbound CALL (`sh -c` on Linux/macOS, `cmd /C` on
283+
Windows), writing the payload as one JSON document to its stdin and
284+
reading its entire stdout back as the reply (empty stdout replies
285+
`null`). Every procedure a daemon serves shares its one `serveSession`
286+
(see "Three Sessions, not one" above) — a hung exec would block every
287+
OTHER registered procedure too, not just its own, so `-exec-timeout`
288+
(10s default) kills it and turns the call into a normal ERROR reply
289+
instead. A non-zero exit or invalid JSON on stdout do the same — none of
290+
the three can crash the shared serve loop, verified live: three
291+
procedures registered to fail three different ways (non-zero exit,
292+
non-JSON stdout, a `sleep` past its timeout) all correctly answered
293+
their callers with an ERROR while a fourth, working procedure kept
294+
computing real per-call replies throughout.
266295

267296
---
268297

‎cmd/macula-cli/serve.go‎

Lines changed: 57 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -36,11 +36,18 @@ import (
3636
// *configured* -reply value even though an "unauthorized" ERROR frame,
3737
// not that payload, is what actually went out over the wire.
3838
type serveResult struct {
39-
Procedure string `json:"procedure"`
40-
Refused bool `json:"refused,omitempty"`
41-
Payload any `json:"payload,omitempty"`
42-
Replied any `json:"replied,omitempty"`
43-
DurationMs int64 `json:"duration_ms"`
39+
Procedure string `json:"procedure"`
40+
Refused bool `json:"refused,omitempty"`
41+
Payload any `json:"payload,omitempty"`
42+
Replied any `json:"replied,omitempty"`
43+
// HandlerError is set when -exec's command failed, timed out, or
44+
// produced invalid JSON -- a normal outcome for -exec (it's a real
45+
// caller-visible ERROR reply, sent successfully), not a bug, so
46+
// this is still report.Ok, same reasoning as Refused above. Replied
47+
// is omitted in this case: there's no reply value to show, only an
48+
// error.
49+
HandlerError string `json:"handler_error,omitempty"`
50+
DurationMs int64 `json:"duration_ms"`
4451
}
4552

4653
func runServe(args []string) int {
@@ -50,6 +57,8 @@ func runServe(args []string) int {
5057
realmHex := fs.String("realm", "", "32-byte realm as hex (default: all-zero realm)")
5158
replyJSON := fs.String("reply", "null", "the RESULT payload to send back, as a JSON document")
5259
echo := fs.Bool("echo", false, "reply with the caller's own payload instead of -reply")
60+
execCmd := fs.String("exec", "", "compute the RESULT per call instead of a fixed -reply: run this shell command, writing the call's payload as one JSON document to its stdin and parsing its entire stdout as the reply (empty stdout replies null); a non-zero exit, a timeout, or invalid JSON on stdout all become a normal ERROR reply to the caller, not a crash. Takes precedence over -reply/-echo.")
61+
execTimeout := fs.Duration("exec-timeout", daemon.DefaultExecTimeout, "with -exec, how long one invocation may run before it's killed and the call fails with a timeout error")
5362
timeout := fs.Duration("timeout", 30*time.Second, "connect timeout, plus how long to wait for one inbound CALL")
5463
direct := fs.Bool("direct", false, "also publish a signed direct-dial DHT advertisement (directdial.AdvertiseDirect), so a caller can resolve and dial this station directly instead of depending on advertise-gossip having propagated a route")
5564
ttl := fs.Duration("ttl", time.Hour, "direct-dial advertisement TTL (only meaningful with -direct)")
@@ -76,6 +85,13 @@ func runServe(args []string) int {
7685
"With -require-ucan-issuer, refuses any CALL that doesn't present a valid\n"+
7786
"UCAN token from that issuer, before the handler ever runs -- composes\n"+
7887
"freely with -direct, since gating and advertising are independent.\n\n"+
88+
"With -exec, the reply is computed per call instead of a fixed -reply:\n"+
89+
"the given shell command runs once per inbound CALL, receiving the\n"+
90+
"call's payload as one JSON document on stdin and answering with\n"+
91+
"whatever JSON it writes to stdout (empty stdout replies null). A\n"+
92+
"non-zero exit, a timeout (-exec-timeout), or invalid JSON on stdout\n"+
93+
"all become a normal ERROR reply to the caller, not a command failure.\n"+
94+
"Cannot be combined with -reply/-echo.\n\n"+
7995
"With -daemon, this registers <procedure> with an already-running\n"+
8096
"\"macula-cli daemon start\" instead of dialing the mesh itself, answers\n"+
8197
"however many calls arrive until -stop unregisters it or the daemon\n"+
@@ -89,6 +105,9 @@ func runServe(args []string) int {
89105
if *certChainFile != "" && !*direct {
90106
return report.Fail(*jsonOut, fmt.Errorf("-cert-chain requires -direct (cert-chain authorization is a direct-dial-only feature)"), nil)
91107
}
108+
if *execCmd != "" && (*echo || *replyJSON != "null") {
109+
return report.Fail(*jsonOut, fmt.Errorf("-exec cannot be combined with -reply or -echo (it computes the reply itself, per call)"), nil)
110+
}
92111
var requiredIssuer []byte
93112
if *requireUcanIssuer != "" {
94113
var hexErr error
@@ -116,6 +135,8 @@ func runServe(args []string) int {
116135
requireUcanIssuer: *requireUcanIssuer,
117136
replyJSON: *replyJSON,
118137
echo: *echo,
138+
execCmd: *execCmd,
139+
execTimeout: *execTimeout,
119140
})
120141
}
121142
if fs.NArg() != 2 {
@@ -133,7 +154,7 @@ func runServe(args []string) int {
133154
if err != nil {
134155
return report.Fail(*jsonOut, err, nil)
135156
}
136-
replyValue, err := wirevalue.FromJSON([]byte(*replyJSON))
157+
replyHandler, err := daemon.BuildReplyHandler(json.RawMessage(*replyJSON), *echo, *execCmd, *execTimeout)
137158
if err != nil {
138159
return report.Fail(*jsonOut, fmt.Errorf("-reply: %w", err), nil)
139160
}
@@ -183,7 +204,8 @@ func runServe(args []string) int {
183204
}
184205
defer func() { _ = session.Unadvertise(frame.NewUnadvertiseSpec(realm, procedure, id.NodeID()), id) }()
185206

186-
var received cbor.Value
207+
var received, repliedValue cbor.Value
208+
var handlerErr error
187209
handlerRan := false
188210
lookup := func(gotRealm []byte, gotProcedure string) (connection.CallHandler, bool) {
189211
if gotProcedure != procedure {
@@ -192,10 +214,10 @@ func runServe(args []string) int {
192214
return func(payload cbor.Value) (cbor.Value, error) {
193215
handlerRan = true
194216
received = payload
195-
if *echo {
196-
return payload, nil
197-
}
198-
return replyValue, nil
217+
reply, err := replyHandler(payload)
218+
repliedValue = reply
219+
handlerErr = err
220+
return reply, err
199221
}, true
200222
}
201223

@@ -227,14 +249,30 @@ func runServe(args []string) int {
227249
return 0
228250
}
229251

230-
replied := replyValue
231-
if *echo {
232-
replied = received
252+
if handlerErr != nil {
253+
// A handler error (only reachable via -exec -- the static
254+
// reply/echo path never errors) is still a successfully-sent
255+
// reply from ServeOneCall's own point of view, exactly like the
256+
// UCAN-refused case above -- a real ERROR frame went out over
257+
// the wire, this command didn't fail. Replied is omitted: there
258+
// is no reply value, only an error.
259+
result := serveResult{
260+
Procedure: procedure,
261+
Payload: wirevalue.ToJSON(received),
262+
HandlerError: handlerErr.Error(),
263+
DurationMs: duration,
264+
}
265+
report.Ok(*jsonOut, result, func() {
266+
fmt.Printf("%s served (%d ms) -- handler error, sent to the caller as an ERROR reply: %s\n", procedure, duration, handlerErr)
267+
fmt.Printf(" received: %v\n", result.Payload)
268+
})
269+
return 0
233270
}
271+
234272
result := serveResult{
235273
Procedure: procedure,
236274
Payload: wirevalue.ToJSON(received),
237-
Replied: wirevalue.ToJSON(replied),
275+
Replied: wirevalue.ToJSON(repliedValue),
238276
DurationMs: duration,
239277
}
240278
report.Ok(*jsonOut, result, func() {
@@ -262,6 +300,8 @@ type serveDaemonArgs struct {
262300
requireUcanIssuer string
263301
replyJSON string
264302
echo bool
303+
execCmd string
304+
execTimeout time.Duration
265305
}
266306

267307
// runServeDaemon registers (or, with -stop, unregisters) a.procedure
@@ -299,6 +339,8 @@ func runServeDaemon(a serveDaemonArgs) int {
299339
RealmHex: a.realmHex,
300340
Reply: replyRaw,
301341
Echo: a.echo,
342+
Exec: a.execCmd,
343+
ExecTimeoutMs: a.execTimeout.Milliseconds(),
302344
Direct: a.direct,
303345
TTLSeconds: int64(a.ttl.Seconds()),
304346
RequireUcanIssuer: a.requireUcanIssuer,

‎guides/HOWTO.md‎

Lines changed: 76 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -239,15 +239,17 @@ $ macula-cli call --json --direct station-de-frankfurt.macula.io macula_cli.howt
239239
## 4. `serve` — advertise and answer
240240

241241
```bash
242-
macula-cli serve [--json] [--identity <path>] [--realm <hex>] [--reply '<json>' | --echo] [--timeout 30s] [--direct] [--ttl 1h] [--cert-chain <pem>] [--require-ucan-issuer <hex>] <host[:port]> <procedure>
243-
macula-cli serve -daemon [--json] [--reply '<json>' | --echo] [--direct] [--ttl 1h] [--cert-chain <pem>] [--require-ucan-issuer <hex>] [-socket-name <name>] <procedure>
242+
macula-cli serve [--json] [--identity <path>] [--realm <hex>] [--reply '<json>' | --echo | --exec '<cmd>' [--exec-timeout 10s]] [--timeout 30s] [--direct] [--ttl 1h] [--cert-chain <pem>] [--require-ucan-issuer <hex>] <host[:port]> <procedure>
243+
macula-cli serve -daemon [--json] [--reply '<json>' | --echo | --exec '<cmd>' [--exec-timeout 10s]] [--direct] [--ttl 1h] [--cert-chain <pem>] [--require-ucan-issuer <hex>] [-socket-name <name>] <procedure>
244244
macula-cli serve -daemon -stop [-socket-name <name>] <procedure>
245245
```
246246
247247
| Flag | Default | Meaning |
248248
|---|---|---|
249249
| `--reply` | `null` | the RESULT payload to send back |
250250
| `--echo` | off | reply with the caller's own payload instead of `--reply` |
251+
| `--exec` | — | compute the RESULT per call instead: run this shell command, the call's payload as JSON on its stdin, its stdout parsed as the reply. Cannot combine with `--reply`/`--echo` (§9's `serve -daemon -exec`) |
252+
| `--exec-timeout` | `10s` | with `--exec`, how long one invocation may run before it's killed and the call fails with a timeout error |
251253
| `--timeout` | `30s` | connect timeout, plus how long to wait for one inbound CALL |
252254
| `--direct` | off | also publish a signed direct-dial DHT advertisement — pair with `call --direct` |
253255
| `--ttl` | `1h` | direct-dial advertisement TTL (only meaningful with `--direct`) |
@@ -666,6 +668,78 @@ $ macula-cli call station-de-frankfurt.macula.io:4433 macula_cli.daemon_smoke_te
666668
error: call failed: unknown_next_peer (code=1) (bolt4=unknown_next_peer, retryable=true)
667669
```
668670
671+
### `serve -daemon -exec`
672+
673+
```bash
674+
macula-cli serve -daemon -exec '<shell command>' [-exec-timeout <duration>] <procedure>
675+
```
676+
677+
`-reply`/`-echo` both answer from something already known at
678+
registration time. `-exec` is the one registration mode that computes a
679+
reply PER CALL: the given shell command runs once per inbound CALL, gets
680+
the call's payload as one JSON document on stdin, and whatever it writes
681+
to stdout becomes the reply (empty stdout replies `null`). Cannot be
682+
combined with `-reply`/`-echo` — rejected at parse time, before ever
683+
registering anything:
684+
685+
```
686+
$ macula-cli serve -daemon -exec 'true' -reply '{"x":1}' station-de-frankfurt.macula.io:4433 macula_cli.should_not_register
687+
error: -exec cannot be combined with -reply or -echo (it computes the reply itself, per call)
688+
```
689+
690+
Real per-call computation, not a cached value — registered once, called
691+
three times with different input, three different genuinely-computed
692+
replies (a two-line Node script reading `{n}` from stdin and writing
693+
`{doubled: n*2, received: n}`):
694+
695+
```
696+
$ macula-cli serve -daemon -socket-name exectest -exec 'node double_handler.mjs' macula_cli.exec_smoke_test
697+
{"registered":true,"procedure":"macula_cli.exec_smoke_test"}
698+
699+
$ macula-cli call --json station-de-frankfurt.macula.io:4433 macula_cli.exec_smoke_test -args '{"n":21}'
700+
{"procedure":"macula_cli.exec_smoke_test","payload":{"doubled":42,"received":21},"duration_ms":53}
701+
702+
$ macula-cli call --json station-de-frankfurt.macula.io:4433 macula_cli.exec_smoke_test -args '{"n":100}'
703+
{"procedure":"macula_cli.exec_smoke_test","payload":{"doubled":200,"received":100},"duration_ms":53}
704+
705+
$ macula-cli call --json station-de-frankfurt.macula.io:4433 macula_cli.exec_smoke_test -args '{"n":-7}'
706+
{"procedure":"macula_cli.exec_smoke_test","payload":{"doubled":-14,"received":-7},"duration_ms":55}
707+
```
708+
709+
**Every procedure a daemon serves shares one `serveSession`** (see the
710+
README's "Three Sessions, not one") — a hung `-exec` would block every
711+
OTHER registered procedure too, not just its own, so `-exec-timeout`
712+
(10s default) kills it. A non-zero exit or invalid JSON on stdout are
713+
treated the same way: all three become a normal ERROR reply to the
714+
caller, never a crash of the daemon or of any OTHER procedure it's
715+
serving — verified live, one working procedure kept answering correctly
716+
throughout while three sibling registrations were deliberately made to
717+
fail each of these three ways:
718+
719+
```
720+
$ macula-cli call --json station-de-frankfurt.macula.io:4433 macula_cli.exec_fail_test
721+
error: call failed: unknown_error (code=15): exec "exit 1": exit status 1
722+
723+
$ macula-cli call --json station-de-frankfurt.macula.io:4433 macula_cli.exec_badjson_test
724+
error: call failed: unknown_error (code=15): exec "echo 'not json'": stdout is not valid JSON: wirevalue: invalid JSON: invalid character 'o' in literal null (expecting 'u')
725+
726+
$ macula-cli call --json -timeout 10s station-de-frankfurt.macula.io:4433 macula_cli.exec_timeout_test
727+
error: call failed: unknown_error (code=15): exec "sleep 5": timed out after 1s
728+
# (registered with -exec-timeout 1s -- correctly killed at 1s, not left running the full 5s sleep)
729+
730+
$ macula-cli call --json -args '{"n":999}' station-de-frankfurt.macula.io:4433 macula_cli.exec_smoke_test
731+
{"procedure":"macula_cli.exec_smoke_test","payload":{"doubled":1998,"received":999},"duration_ms":56}
732+
```
733+
734+
Also works on the one-shot (non-`-daemon`) `serve` — same `-exec`/
735+
`-exec-timeout` flags, same contract, just for exactly one call before
736+
exiting:
737+
738+
```
739+
$ macula-cli serve -exec 'node double_handler.mjs' station-de-frankfurt.macula.io:4433 macula_cli.exec_direct_test --json
740+
{"procedure":"macula_cli.exec_direct_test","payload":{"n":5},"replied":{"doubled":10,"received":5},"duration_ms":5022}
741+
```
742+
669743
### `call -via-daemon`
670744
671745
```bash

0 commit comments

Comments
 (0)