Skip to content

Implement event tracing core - #80

Open
Synte-Peng wants to merge 5 commits into
sysprog21:mainfrom
Synte-Peng:event-tracing
Open

Implement event tracing core#80
Synte-Peng wants to merge 5 commits into
sysprog21:mainfrom
Synte-Peng:event-tracing

Conversation

@Synte-Peng

@Synte-Peng Synte-Peng commented Jul 25, 2026

Copy link
Copy Markdown

Implement a compile-time type-checked kernel event tracing subsystem
backed by an X-macro ring buffer. Event writes are always-inlined with
zero overhead when CONFIG_DEBUG_TRACE=0.

  • X-macro single-source-of-truth — trace_events.h included 5 times to
    generate enums, structs, inline writers, print functions, and dispatch
    table. Adding an event is one line.
  • Per-event always_inline writers with typed C prototypes —
    compile-time type checking on every trace call, zero function-call
    overhead at runtime
  • Per-event write cost: ~17–26 instructions on RV32I at -O2
    TASK_SWITCH : 17 instructions.
    TASK_CREATE : 26 instructions.
  • Fixed-stride ring buffer — each event occupies exactly
    DEBUG_EVENT_STRUCT_SIZE (configurable, default 16 bytes), with a
    union projection so only populated fields are stored
  • CSR-level interrupt gating — TR_SAFE_ENTER/LEAVE via csrrci/csrsi
    (1 instruction per direction, vs 3 for hal_interrupt_set)
  • SAFE events re-enable interrupts around the write (used inside
    CRITICAL_ENTER sections like TASK_CREATE), RAW events skip
    gating (used where interrupts are already disabled, like
    dispatch/TASK_SWITCH)
  • Event slot size configurable via DEBUG_EVENT_STRUCT_SIZE in
    config.h, new fields added via __field in trace_events.h —
    STATIC_ASSERT catches sizing errors at compile time
  • All tracepoints guarded by #if CONFIG_DEBUG_TRACE — when disabled,
    the entire tracing subsystem compiles to nothing
  • All reschedule paths route through _yield() — weak-alias contract
    preserved

Supersedes #79.


Summary by cubic

Add a lightweight, type-checked kernel event tracing system with a fixed-size ring buffer and instrumented task lifecycle/switch events. Tracing is off by default, compiles out when CONFIG_DEBUG_TRACE=0, snapshots the buffer to shorten interrupt-off time in debug_dump_events, and now skips re-entrant dumps to avoid snapshot corruption.

  • New Features

    • X-macro tracing core with typed, always-inlined writers; single-source event registry in include/sys/trace_events.h.
    • Fixed-stride ring buffer with SAFE/RAW interrupt protocols and per-event printers via a dispatch table.
    • Config flags: CONFIG_DEBUG_TRACE (default 0), DEBUG_EVENT_BUFFER_SIZE (256), DEBUG_EVENT_STRUCT_SIZE (16).
    • Public API: DEBUG_TRACE_EVENT(...), plus debug_dump_events, debug_clear_events, debug_trace_count, debug_trace_overwrites, debug_trace_get.
    • Instrumented task.c with TASK_CREATE, TASK_DESTROY, TASK_SWITCH, TASK_DELAY, TASK_SUSPEND, TASK_RESUME, TASK_YIELD. Internal reschedules use _yield() to avoid double-tracing.
  • Migration

    • Enable by setting CONFIG_DEBUG_TRACE=1 in config.h; tune DEBUG_EVENT_BUFFER_SIZE and DEBUG_EVENT_STRUCT_SIZE if needed.
    • Define new events in include/sys/trace_events.h and emit with DEBUG_TRACE_EVENT(EVENT_NAME, ...).
    • Use SAFE for normal paths; use RAW when interrupts are already disabled (e.g., ISR/critical sections).
    • No behavior change to public APIs; mo_task_delay/mo_task_suspend now call _yield() internally to keep one event per action.

Written for commit 04756f0. Summary will update on new commits.

Review in cubic

Add CONFIG_DEBUG_TRACE (default off), DEBUG_EVENT_BUFFER_SIZE (256),
and DEBUG_EVENT_STRUCT_SIZE (16) to config.h.  Wire debug_trace.o
into KERNEL_OBJS in Makefile.
X-macro ring-buffer tracer for lightweight kernel instrumentation.

Architecture:
- trace_events.h: single-source-of-truth event definitions, included
  5 times with varying preprocessor state to generate enums, structs,
  inline writers, print functions, and a dispatch table.
- debug_trace.h: the 5-pass X-macro engine, per-event typed
  always_inline write functions, and public API with compile-out
  stubs for CONFIG_DEBUG_TRACE=0.
- debug_trace.c: fixed-stride ring buffer with dump/clear/count/
  overwrites/get operations.

Two interrupt-protection protocols:
- SAFE: gates the buffer write with csrrci/csrsi to disable/restore
  interrupts. For call sites that do not already hold interrupt
  protection (e.g. NOSCHED sections).
- RAW: skips gating. For call sites already executing inside
  interrupt-disabled sections (e.g. dispatch, CRITICAL_ENTER,
  or ISR context).

Write cost is minimized: class structs use a union projection so only
populated fields are stored; unused fields carry stale data from prior
slots without per-event zeroing. When CONFIG_DEBUG_TRACE=0, all
tracepoints compile to nothing.
Add 7 tracepoints to task.c: TASK_CREATE, TASK_DESTROY,
TASK_SWITCH, TASK_DELAY, TASK_SUSPEND, TASK_RESUME, TASK_YIELD.

Refactor mo_task_delay() and mo_task_suspend() to call _yield()
directly instead of mo_task_yield().  mo_task_yield() now emits
TASK_YIELD, so calling it from within mo_task_delay/suspend would
double-trace (TASK_DELAY + TASK_YIELD, or TASK_SUSPEND + TASK_YIELD).
The internal _yield() path is trace-free so each operation records
exactly one event.

Guard the previous_state capture in mo_task_cancel() with
#if CONFIG_DEBUG_TRACE so the variable is compiled out when
tracing is disabled.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread kernel/debug_trace.c
debug_dump_events() held global interrupt-disable across up to 256
printf() calls.  On a slow UART this delays timer dispatch and other
interrupts for an unbounded amount of time.

Fix by snapshotting the entire ring buffer into a static _tr_snap[]
inside the critical section, then performing all printf output with
interrupts re-enabled.  All _tr_print_EVENT_* functions now
take a buffer pointer instead of hard-coding debug_trace_buffer, and
_tr_print_event() dispatches through it.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 1 file (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread kernel/debug_trace.c
Concurrent debug_dump_events() calls share the static _tr_snap[]
buffer.  Once interrupts are re-enabled for the printf loop the
scheduler can switch to another task which, if it also calls dump,
overwrites the snapshot and corrupts the first task's output.

Add a volatile _tr_dumping flaginside the critical section.
Re-entrant callers skip the dump entirely instead of blocking.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant