Skip to content

PROGRESS.md

Latest commit

 

History

History
43 lines (35 loc) · 12.4 KB

PROGRESS.md

File metadata and controls

43 lines (35 loc) · 12.4 KB
Raw

Progress Log

Development progress log for commit-story-v2. Tracks implementation milestones across PRD work.

Entry format: - (YYYY-MM-DD) Description of feature-level change (PRD #X, milestone)

[Unreleased]

Added

  • (2026-07-11) Promoted gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and commit_story.ai.section_type from recommended to required in the Weaver registry (telemetry/registry/attributes.yaml). These attributes were populated on live spans via a manual code fix on a separate branch (spinybacked-orbweaver PRD #980, milestone M1.5), but recommended-level attributes aren't a forcing function for a spiny-orb instrumentation agent to add them on a fresh run. required documents the durable contract so future instrumentation runs and Datadog dashboard queries can rely on these attributes being present (spinybacked-orbweaver PRD #980, milestone M1.7).
  • (2026-06-19) Added logger.info() calls at six future span sites in src/generators/summary-graph.js (dailySummaryNode, generateDailySummary) and src/generators/journal-graph.js (summaryNode, technicalNode, dialogueNode, generateJournalSections). Each call captures meaningful state — entry count, date, substantialUserMessages, hasFunctional, maxQuotes — so logs will carry trace context once spiny-orb wraps these functions in spans on the next instrument branch, completing the log-trace correlation story for the 28-second black-box gap in Datadog.
  • (2026-06-19) Documented COMMIT_STORY_TRACELOOP=true in .env.example: the env var activates LangChain and MCP auto-instrumentation (LLM call detail spans via Traceloop) when set in the spiny-orb instrument branch run environment. It has never been set in any scoring run — activating it will add model/token/latency spans to Datadog traces.
  • (2026-06-18) Replaced all console.log/console.error/console.warn calls in src/ with pino structured logging. Created a shared src/logger.js that outputs JSON to stdout for the main CLI; the MCP server gets its own pino instance pointed at stderr (stdout is reserved for JSON-RPC). Added pino and @opentelemetry/instrumentation-pino as dependencies. When the OTel SDK is active, the pino bridge will automatically inject trace_id and span_id into log records — this is the logs leg of the observability triangle that makes log-trace correlation in Datadog possible without any application-level span.spanContext() calls.
  • (2026-06-18) Wired the OTLP log pipeline into the OTel SDK bootstrap (examples/instrumentation.js): added a LoggerProvider with SimpleLogRecordProcessor + OTLPLogExporter pointed at http://localhost:4318/v1/logs, registered it globally via logs.setGlobalLoggerProvider, and added PinoInstrumentation to the SDK's instrumentations array. When a spiny-orb instrument branch is active, pino log records emitted within active spans will carry trace_id/span_id fields and flow to Datadog via OTLP — completing the log-trace correlation link without any application code changes.
  • (2026-06-18) Created PRD #77 (Observability Triangle Foundation) to establish structured logging infrastructure on main: pino migration, @opentelemetry/instrumentation-pino bridge, OTLP log exporter in bootstrap, and filelog→OTLP logs pipeline update in otelcol-config.yaml. This makes every spiny-orb eval run automatically inherit log-trace correlation for the Datadog observability triangle demo.
  • (2026-06-16) Added commit_story.context.messages_filtered and commit_story.context.substantial_messages to the Weaver registry context attribute group. These two attributes support the traces-to-logs correlation demo: messages_filtered captures how many messages were dropped as noise during context collection, and substantial_messages tracks how many were substantive enough to gate whether the dialogue and technical_decisions journal sections run. Both are emitted in structured log bodies alongside the LLM generation span to give the log line context color beyond just trace correlation.
  • (2026-03-21) Installed OTel SDK, OTLP exporter, and Traceloop auto-instrumentation packages for local telemetry (PRD #51, milestone 1)
  • (2026-03-21) Created OTel SDK bootstrap with OTLP exporter, resource attributes, LangChain/MCP auto-instrumentation, and graceful shutdown (PRD #51, milestone 2)
  • (2026-03-21) Added Datadog Agent Docker setup/teardown scripts with vals-based secret injection and port/container safety checks (PRD #51, milestone 3)
  • (2026-03-21) Added "files" whitelist to package.json — tarball drops from 181 files (1.5 MB) to 38 files (218 KB), instrumentation.js excluded from distribution (PRD #51, milestone 4)
  • (2026-03-21) Updated git hook to load OTel SDK via NODE_OPTIONS --import flag, with symlink-aware path resolution for npm-linked dev mode (PRD #51, milestone 5)
  • (2026-03-21) End-to-end validation: OTel SDK exports traces successfully to local DD Agent via OTLP HTTP, service name commit-story confirmed in resource attributes (PRD #51, milestone 6)
  • (2026-03-21) Moved @opentelemetry/sdk-node from peerDependencies to devDependencies in eval repo, PR #27 (PRD #51, milestone 7)
  • (2026-03-21) Added spiny-orb.yaml config and semconv/ telemetry schema for spiny-orb agent compatibility (PRD #51, milestone 8)

Fixed

  • (2026-07-08) Fixed the dialogue-extraction prompt conflating "quote supports the summary" with "quote is redundant with the summary." Journal entries were showing "No significant dialogue found" even for sessions with substantial back-and-forth, because the model treated overlap between a quote and the summary's narrative as a reason to discard the quote rather than keep it. Clarified in three places that overlap is a pass condition, not a rejection reason.

  • (2026-07-08) Tightened isFailurePlaceholder to match only the bracketed placeholder shape (e.g. [Daily summary generation failed]) instead of a bare substring check, so real narrative content that happens to mention "generation failed" or "extraction failed" in prose is no longer misclassified as stale and silently overwritten.

  • (2026-07-08) Fixed a data-loss window in journal-manager.js's stale-placeholder regeneration: removing the old block and appending the new entry were two separate writes, so a crash between them could drop the entry entirely. Both are now combined into a single write. Also surfaced non-ENOENT errors from the existing-file read instead of swallowing them silently, and reused the shared ENTRY_SEPARATOR constant instead of a duplicated separator literal.

  • (2026-07-08) Fixed duplicate-detection in journal-manager.js and summary-manager.js treating stale failure-placeholder content (e.g. [Summary generation failed]) as a valid existing entry, so a commit whose journal entry or summary failed to generate once could never be regenerated on a later run. Added a shared isFailurePlaceholder helper (src/utils/failure-placeholder.js) checking for the "generation failed"/"extraction failed" substrings. journal-manager.js now excises a stale block and appends the fresh entry when either the exact-hash or semantic dedup match is a placeholder. All six save*Summary/generateAndSave*Summary functions in summary-manager.js (daily, weekly, monthly) now read existing file content and treat a placeholder match the same as a missing file, regenerating rather than skipping.

  • (2026-07-08) Fixed scripts/install-hook.sh generating a post-commit hook that silently produces placeholder-only journal entries when a commit is made from inside a Claude Code session. Claude Code sets ANTHROPIC_BASE_URL/ANTHROPIC_CUSTOM_HEADERS to route its own Anthropic calls through the Datadog AI Gateway; the hook's child node/npx process inherited both, so commit-story's own Anthropic SDK calls were sent to the gateway URL without valid gateway headers and every LLM-dependent section failed with a 400 while the hook still exited 0. All three invocation branches (vals exec, plain node, and the npx commit-story fallback) now strip both vars via env -u before invoking node. Reinstalled the fixed hook in all 9 repos running the commit-story-v2 hook template.

  • (2026-06-19) Fixed COMMIT_STORY_AUTO_SUMMARIZE env var normalization in src/utils/config.js: values like 'FALSE' or ' false ' (uppercase or with surrounding whitespace) were not treated as false, so auto-summarize ran when users intended to disable it. Added .trim().toLowerCase() before comparison.

  • (2026-06-19) Refactored src/index.js to replace 14 process.exit() calls with return exitCode from main() and handleSummarize(), which are now exported async functions. Auto-execution is gated by a realpathSync(process.argv[1]) === fileURLToPath(import.meta.url) check so tests can import without triggering execution. Span flushing on exit is handled by the shutdownAndExit wrapper in examples/instrumentation.js, which intercepts process.exit() globally when the OTel SDK is active.

  • (2026-06-19) Fixed git.repository.id in examples/instrumentation.js: Datadog was showing the stale repository association mcp-commit-story instead of commit-story-v2 because the Datadog Exporter auto-detects git metadata and Datadog's backend caches a service's repository URL from first registration. Explicitly setting git.repository.id: github.com/wiggitywhitney/commit-story-v2 in the OTel resource attributes overrides whatever is auto-detected.

  • (2026-06-19) Fixed acceptance gate glob in .claude/verify.json: tests/**/acceptance-gate.test.js was not being expanded by bash when the pre-PR hook ran it via bash -c (bash requires shopt -s globstar for ** to be recursive; without it, ** matches only one path component, and the file at tests/acceptance-gate.test.js — zero levels deep — was not found). Replaced with the explicit path tests/acceptance-gate.test.js.

  • (2026-06-19) Fixed four bugs across dialogue extraction, sensitive data filtering, and summary file I/O: (1) cleanDialogueOutput in the journal graph was dropping continuation lines of multi-line assistant responses inside blockquotes — only the first line was kept; (2) redactDiff was applying sensitive-pattern regexes to diff header lines (diff --git, ---, +++, index) causing filenames to be incorrectly redacted; (3) four catch blocks in summary-manager swallowed all errors silently — they now check for ENOENT and rethrow everything else so permission and type errors surface; (4) week-to-month assignment in monthly summary reads used the Sunday boundary instead of the Monday boundary, causing cross-boundary weeks to be counted in the wrong month (fixed to match summary-detector.js behavior).

  • (2026-06-19) Added docs/pino-otel-log-trace-correlation.md: reference guide for the spinybacked-orbweaver team documenting the full pino + OTel SDK integration stack, the IITM ESM hook Node v22+ requirement, the log pipeline path from pino through otelcol-contrib to Datadog, and the traceloop gating finding where IS scoring does not activate @traceloop/instrumentation-langchain or @traceloop/instrumentation-mcp because COMMIT_STORY_TRACELOOP=true is not set during scoring runs.

  • (2026-06-19) Fixed pino log-trace correlation in examples/instrumentation.js: PinoInstrumentation was silently not injecting trace_id/span_id into log records on Node.js v22+ because the import-in-the-middle (IITM) ESM loader hook was never registered. On Node v22+, ESM-imported CJS modules like pino do not route through require-in-the-middle's hooks without an explicit module.register() call. Added createAddHookMessageChannel() + register('import-in-the-middle/hook.mjs', ...) + await waitForAllMessagesAcknowledged() before sdk.start(), and added import-in-the-middle@^3.0.0 as a devDependency. Verified end-to-end: pino logs now appear in Datadog Logs Explorer with trace_id/span_id fields matching the APM root span, completing the log-trace correlation link in the observability triangle.

  • (2026-06-18) Fixed acceptance gate (tests/acceptance-gate.test.js) failing in the pre-PR hook with "env: node: No such file or directory". The vals exec command was missing the -i flag, so it ran in a stripped environment where /opt/homebrew/bin (where node lives) was not on PATH. Also stripped gateway env vars from the command so the LangGraph API call goes directly to the API rather than routing through the enterprise gateway (which rejects subprocess calls).

  • (2026-05-05) Added service.instance.id: randomUUID() to resourceFromAttributes in examples/instrumentation.js so RES-001 passes in IS scoring runs.