Sync ako/main → upstream: microflow/nanoflow debugger + OpenTelemetry (metrics & traces)#788
Open
ako wants to merge 16 commits into
Open
Sync ako/main → upstream: microflow/nanoflow debugger + OpenTelemetry (metrics & traces)#788ako wants to merge 16 commits into
ako wants to merge 16 commits into
Conversation
Turns the runtime-debugger investigation from the mxcli-sudoku findings/README (and its scripts/mfdebug.sh wrapper) into a supported feature proposal. The Mendix microflow debugger spans two APIs: the M2EE admin plane (:8090, X-M2EE-Authentication) toggles enable/disable/status, and the app endpoint (<app>/debugger/, X-Debugger-Authentication) drives start_session / add_breakpoint / get_paused_microflows / get_object / step_* / continue. A breakpoint's object_id is an activity's model GUID, which nothing in the runtime surfaces. mxcli is uniquely positioned: run --local already owns the admin password + app URL, and sdk/mpr/parser_microflow.go already extracts each activity's $ID — so it can offer breakpoints BY NAME (Module.Flow --activity Name) rather than by GUID. The proposal specifies the `mxcli debug` command surface, a DebuggerClient for the second plane, the name→GUID resolver, warm-loop integration, safety (a breakpoint pauses the browser too; auto-disable on interrupt), five slices, tests, open questions, and a DAP-bridge follow-up. Docs-only; no code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4
First slice of the microflow debugger (docs/11-proposals/PROPOSAL_microflow_debugger.md):
the two-plane wiring only. Breakpoints, paused-microflow inspection, and stepping
are later slices.
- docker.DebuggerClient drives both planes: the M2EE admin plane (reusing
CallM2EE) for enable_debugger {password} / disable_debugger / get_debugger_status,
and the app /debugger/ endpoint (X-Debugger-Authentication: base64(debugPass),
{action, session_token, params} envelope with params always present) for
start_session. The returned session token is cached under
<projectDir>/.mxcli/debug-session.token and reloaded on the next command.
- mxcli debug status/enable/disable, defaulting to a `run --local` runtime
(app :8080, admin :8090, admin pass mxcli-local-dev, debug pass mxdebug),
overridable via flags/env (MXCLI_APP_URL/ADMIN_PASS/DEBUG_PASS). enable also
starts the session; disable clears the cached token. Warns that a breakpoint
pauses the browser too, so the session must be disabled when done.
Tests (httptest stub of both planes): status parsing, enable sends the password,
start_session sends params + no token + base64 auth and caches/ reloads the token,
disable clears the token file, and a 401 yields an actionable error. Build + vet
clean; docker unit tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4
Second slice of the microflow debugger. Resolves a microflow activity to the
model GUID the debugger uses as object_id, so breakpoints are set BY NAME.
Key finding: an activity's model GetID() already equals the debugger's object_id
— types.BlobToUUID emits the Microsoft-GUID / bytes_le form (data[3],data[2],…),
identical to the uuid.UUID(bytes_le=…) conversion the sudoku wrapper did by hand.
So no re-derivation is needed: open the project, GetRawMicroflowByName +
ParseMicroflowBSON, and read each object's GetID().
- docker.DebuggerClient.AddBreakpoint(name, objectID, condition) / RemoveBreakpoint
drive the /debugger/ plane (add_breakpoint {microflow_name, object_id,
condition?}, remove_breakpoint {object_id}) with the session token.
- Resolver (debug_resolve.go): extractActivities reflects each object's Caption +
struct type and pairs it with GetID(); matchActivity picks one by '#<index>' or
a unique caption substring.
- Local breakpoint record (.mxcli/debug-breakpoints.json): the runtime has no
"list breakpoints" call, so mxcli records what it set for 'breaks' (name→GUID
reverse map); cleared on disable.
- Commands: mxcli debug activities <Module.Flow> (discovery), break <Module.Flow>
--activity <#n|caption> [--if <expr>], unbreak, breaks.
Tests: AddBreakpoint/RemoveBreakpoint envelope+token+params (incl. empty-condition
omitted, needs-session error) via the httptest stub; extractActivities,
matchActivity (index/caption/ambiguous/out-of-range), and the local registry
upsert/remove/save-load as pure-logic tests. Build + vet clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4
Third slice of the microflow debugger: drive a paused microflow.
- docker.DebuggerClient: PausedMicroflows (get_paused_microflows), GetObject
(get_object {debug_id, variable_name}), Step (step_over/into/out {debug_id}),
Continue (continue / continue_all).
- CLI: mxcli debug paused (summary + full state), inspect <var> [--flow],
step [over|into|out] [--flow], continue [--all].
- --flow selects the paused microflow by debug_id; when omitted it auto-selects
the only paused flow and otherwise errors asking for --flow, so an action never
targets the wrong flow. extractPausedFlows parses the paused result defensively
(top-level or nested array; debug_id/id/debugId + microflow/name spellings)
since the runtime response shape isn't contract-stable — the full detail is
always printed as raw JSON.
Tests: step over/into/out action mapping + debug_id + unknown-kind error;
continue vs continue_all; get_object params; extractPausedFlows across shapes
and field-name variants. Build + vet clean; docker + cmd unit tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4
Fourth slice of the microflow debugger: wire it into the warm loop. - run --local --debug enables the debugger and starts a session once the runtime is serving, caching the token under <projectDir>/.mxcli/debug-session.token so `mxcli debug break/paused/step/continue -p app.mpr` works from another terminal with no separate `mxcli debug enable`. It reuses the runtime's own admin connection (rt.m2ee) and app URL. - --debug-pass sets the debugger password (default "mxdebug"). - Enabling with no breakpoints does not change runtime behaviour — only a set breakpoint pauses. On shutdown the debugger is disabled best-effort (and the cached token cleared) so a breakpoint can't be left pausing requests. Tests: applyDefaults defaults DebugPass to "mxdebug" under --debug, preserves an explicit value, and leaves it empty without --debug. Build + vet clean. Note: the enable-at-boot path is verified by unit tests + the DebuggerClient stub tests; a full live end-to-end pass (run --local --debug, break by name, hit it from the browser, paused/inspect/step/continue) needs a real runtime + DB and is the acceptance test to run in a dev environment. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4
Final slice: user-facing documentation for mxcli debug. - New skill .claude/skills/mendix/debug-microflows.md (synced to user projects via mxcli init): the loop (run --local --debug in one terminal, break by name + inspect/step/continue in another), the command table, activity/flow selection, and gotchas (a breakpoint pauses the browser too; always disable; same -p everywhere). - New docs-site page docs-site/src/tools/debug-microflows.md + SUMMARY nav entry under Local Dev Loop. - run-local skill + docs-site: --debug / --debug-pass flag rows linking the new page. - CLAUDE.md: microflow debugger added to the Implemented list. check-skill-mdl passes (skills + docs-site). Build clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4
Extends `mxcli debug` to nanoflows and documents nanoflow logging, from the mxcli-sudoku findings on client-side logging/debugging. Nanoflow debugging differs from microflows in two ways the runtime enforces: a nanoflow breakpoint uses the `nanoflow_name` param (the `microflow_name`/ `objectId` keys NPE), and a paused nanoflow appears only in `poll_events`, never in `get_paused_microflows`. Both are now handled: - Resolver auto-detects microflow vs nanoflow (`resolveFlowActivities`: try microflow, else nanoflow via GetRawUnitByName — same ObjectCollection shape), so `break/activities/unbreak Module.Flow` work for either with no new flag. - DebuggerClient.AddBreakpoint takes a `nanoflow` bool → sends `nanoflow_name`; new PollEvents() calls `poll_events`. - `paused` and the debug_id auto-select (`allPausedFlows`) merge get_paused_microflows + poll_events, so a paused nanoflow shows up with its debug_id and variables (printed from the poll_events payload). Docs: write-nanoflows.md documents that nanoflow LOG output goes to the browser console AND the runtime log under a REWRITTEN `Client_Nanoflow` node (a node-name filter for microflows silently drops nanoflow lines). debug-microflows.md + docs-site gain a Nanoflows section; CLAUDE.md updated. Tests: nanoflow_name breakpoint param (and no microflow_name), PollEvents action + token, extractPausedFromEvents (paused_microflow event parsing, no-paused, bad json). Build + vet clean; check-skill-mdl passes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4
…OG DEBUG) The updated mxcli-sudoku findings confirm slice 6's protocol (nanoflow_name + object_id params; paused nanoflows only via poll_events; microflow_name field in the event even for a nanoflow — all already implemented). Two new nuances added to the docs: - A nanoflow's debug_id is SINGLE-USE — it changes after every step, invalidating the old id (a microflow's is stable). mxcli's one-shot commands already handle this by re-reading state each invocation, so `step/inspect/continue` work when they auto-resolve the flow; document to omit --flow for nanoflows (a stale --flow copied from `paused` fails on the second step). - Nanoflow `LOG DEBUG` is dropped server-side (only INFO/WARNING/ERROR reach runtime.log); all four levels still show in the browser console. Docs only — no code change needed; the confirmed params/poll_events shape already match the implementation. (Follow-up noted separately: get_list for inspecting list variables is not yet wired.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4
…list)
Follow-up from the updated nanoflow findings: get_list inspects a LIST variable
of a paused flow (works on both microflows and nanoflows), where get_object is
for a single object.
- DebuggerClient.GetList(debugID, variableName) → post "get_list"
{debug_id, variable_name}.
- `mxcli debug inspect <var> --list` routes to get_list; without --list it stays
get_object. Uses the same debug_id auto-resolution (re-read each invocation, so
a nanoflow's single-use id is handled).
Test: get_list action + params. Docs (skill + docs-site command tables) updated.
Build + vet clean; check-skill-mdl passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4
…passthrough) From the mxcli-sudoku OpenTelemetry findings. The Mendix runtime ships Micrometer registries and the OTel API but starts with none configured; metrics are enabled via update_configuration's Metrics.Registries. - --metrics registers a Prometheus registry at boot; the runtime serves http://127.0.0.1:<admin-port>/prometheus (~70+ metric families). - --runtime-setting Key=Value (repeatable) merges an arbitrary runtime setting into the boot config; Value is parsed as JSON when possible, else a string. Handles a non-prometheus Metrics.Registries (otlp/influx/statsd/jmx) and the OpenTelemetry._RuntimeSpanFilters trace-filter setting. Correctness point from the findings: the admin update_configuration action REPLACES rather than merges and has no read-back, so a separate post-boot call would wipe the DB/BasePath config. mxcli owns the single boot update_configuration (runtimeConfigParams), so these settings are merged into THAT call — the only safe way. The Prometheus URL is printed at boot. Traces (bundled opentelemetry-javaagent) remain a documented manual step — enable via JAVA_TOOL_OPTIONS + OTEL_* env, and apply OpenTelemetry._RuntimeSpanFilters via --runtime-setting (default per-activity tracing is ~10x slower). A first-class --trace flag is a noted follow-up. Tests: buildRuntimeSettings (--metrics registry, JSON-typed values, explicit Metrics.Registries wins over --metrics, string fallback, nil, malformed error) and runtimeConfigParams overlay preserving base keys. Docs: run-local skill + docs-site Metrics/OpenTelemetry section; CLAUDE.md. Build + vet + check-skill-mdl clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4
…an filters Completes the OpenTelemetry findings: metrics (--metrics/--runtime-setting) plus traces (--trace). - --trace attaches the runtime's bundled opentelemetry-javaagent (globbed at <runtimeDir>/agents/opentelemetry-javaagent*.jar) to the runtime JVM via JAVA_TOOL_OPTIONS, and defaults OTEL_SERVICE_NAME (the .mpr name, or --trace-service), OTEL_TRACES_EXPORTER=console, and metrics/logs exporters off — but does NOT override any OTEL_* the user already set, so exporting to an OTLP collector via your own env still works. Console spans ride the #25 runtime-log tee (mx.microflow.name / mx.microflow.depth). - --trace ships default OpenTelemetry._RuntimeSpanFilters (CreateOrChangeVariable/Loop/Gateway/RetrieveFromCache) merged into the boot config, because unfiltered per-activity tracing is ~10x slower; an explicit --runtime-setting filter list wins. - Boot prints where spans go (runtime log, or console when --runtime-log -). Tests: withTraceEnv (agent appended once to existing JAVA_TOOL_OPTIONS, OTEL defaults, user OTEL_* respected), buildRuntimeSettings trace default filters + explicit-wins. Only the pre-existing environmental TestServeIntegration fails. Docs (run-local skill + docs-site OpenTelemetry section incl. OTLP-collector example) and CLAUDE.md updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4
run --local: OpenTelemetry — --metrics (Prometheus), --trace, and --runtime-setting
The nightly matrix (10.24 / 11.6 / 11.12) failed only on 10.24: TestMxCheck_DoctypeScripts/15b-fragment-slots-examples → CE6083 "Design property Card style is not supported by your theme" at every cardWrap container (both engines). `Card style` is an Atlas v3 (11.x) design property; the 10.x Atlas theme doesn't define it. The property sat on the shared `define fragment Card`, instantiated by every page in the file, so a --@Version gate would have to gate the whole file and lose all 10.24 coverage of the content-slot feature 15b actually demonstrates. The design property was incidental (12-styling covers design properties, gated 11.0+), so drop it and keep `class: 'card'` — Atlas card styling that works on every version. Verified: 15b passes on 10.24 AND 11.6.3 (0 errors, both engines). Also swept the doctype suite for other ungated designproperties/building-block usage — 03/12/29/31 gate theirs, 15c was already fixed (#43), and *.test.mdl files aren't run by the doctype test. fix-issue.md records the CE6083 case + the gate-vs-remove rule. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4
Fix nightly on Mendix 10.24: drop 11.x-only design property from 15b
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the 12 commits on ako/main since the last sync (#787). Clean merge against
main;
make build && make testgreen (the only failing test locally is thepre-existing environmental
TestServeIntegration— MxBuild version mismatch).Microflow + nanoflow debugger (
mxcli debug)Headless counterpart to Studio Pro's debugger, driven from the CLI. Sets breakpoints
by name (mxcli owns the admin password + app URL from
run --localand theactivity GUIDs from the
.mpr, so no raw GUIDs).mxcli debug status | enable | disableactivities <Module.Flow>— list activities + object IDsbreak <Module.Flow> --activity <#n|caption> [--if <expr>],unbreak,breakspaused,inspect <var> [--list],step [over|into|out],continue [--all]mxcli run --local --debugenables it at boot (session cached under.mxcli/)nanoflow_namebreakpoint param, paused nanoflowsmerged from
poll_events(they don't appear inget_paused_microflows), and thesingle-use nanoflow
debug_idhandled by re-reading state each command./debugger/); proposal indocs/11-proposals/PROPOSAL_microflow_debugger.md.OpenTelemetry / metrics for
run --local--metrics— register a Prometheus meter registry; served athttp://127.0.0.1:<admin-port>/prometheus.--trace— attach the bundled OpenTelemetry Java agent (console exporter → theruntime log) with default span filters (unfiltered per-activity tracing is ~10×
slower);
--trace-servicesetsOTEL_SERVICE_NAME; user-setOTEL_*(e.g. anOTLP collector) is respected.
--runtime-setting Key=Value(repeatable) — merge any runtime setting into theboot config. All three merge into mxcli's single boot
update_configurationbecause the admin action replaces rather than merges.
Nanoflow logging docs
write-nanoflows.md: nanoflowLOGoutput is rewritten to theClient_Nanoflownode in the runtime log (a microflow node-name filter misses it), and
LOG DEBUGis dropped server-side (browser console only).
Fix
Card styledesign property from the15bfragment-slots example (keptclass: 'card'; CE6083 on the 10.x Atlas theme).Docs: new
debug-microflowsskill + docs-site page;run-localskill/docs gain thedebugger, metrics, and OpenTelemetry sections.