Skip to content

perf: persist Claude cost report memo across launches - #3284

Open
eggyrooch-blip wants to merge 1 commit into
steipete:mainfrom
eggyrooch-blip:perf/claude-report-memo-disk
Open

perf: persist Claude cost report memo across launches#3284
eggyrooch-blip wants to merge 1 commit into
steipete:mainfrom
eggyrooch-blip:perf/claude-report-memo-disk

Conversation

@eggyrooch-blip

Copy link
Copy Markdown

Summary

Claude local-cost still uses the monolithic claude-v6.json artifact. The in-memory report memo already skips that decode on a warm refresh, but it dies with the process. On a machine with ~7.7k Claude jsonl files the cache is 44 MB / 116k claudeRows; a 0.56.0 cold start spent ~90s at ~90–115% CPU in _NSJSONReader parseData on com.steipete.codexbar.cost-usage-scan before going idle.

This persists the existing memo (source inventory + report key including cache/pricing stamps + daily report) next to the cache as claude-v6.report-memo.json. After launch, an unchanged transcript inventory returns the saved report and does not decode the JSON cache. Pricing-catalog and cache-artifact stamp changes still miss the memo and reprice from persisted rows, without rewriting transcripts.

Does not migrate Claude onto SQLite (#2760 was Codex-only). Append/delete of a jsonl still invalidates the memo and takes the existing decode + delta-parse path.

Investigated alongside #3247.

Test plan

  • cold process reuses the persisted report memo without decoding the cache — evict in-memory memo, 0 cache decodes / 0 transcript parses, report identical
  • missing persisted memo still decodes the cache without parsing transcripts — delete sidecar, still 1 cache decode / 0 parses
  • pricing-replacement test unchanged (stamp miss still decodes + reprices, does not rewrite cache)
  • spend-dashboard cold read now expects no extra cache decode
  • swift test --filter CostUsageScannerClaudeMemoTests / CostUsageClaudeKimiAliasTests / SpendDashboardFreshnessScannerTests (local toolchain is Swift 6.0.3; package requires 6.2 — relying on CI)

Keep the local-cost daily report and source inventory next to claude-v6.json so an unchanged transcript set skips the full JSON cache decode after process restart. Pricing and cache-artifact stamp changes still miss the memo and reprice from the persisted rows.

Investigated alongside steipete#3247.
@clawsweeper

clawsweeper Bot commented Aug 30, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 30, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-30T09:51:30.642107Z 778b379 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 778b3793c2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

}

static let shared = CostUsageClaudeReportMemo()
static let persistedVersion = 1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Invalidate persisted reports when the app version changes

When a user upgrades to a build that changes bundled Claude pricing, model aliases, or report aggregation while the transcripts, cache artifact, pricing artifact, and requested date range remain unchanged, this constant still accepts the report produced by the previous binary. loadClaudeDaily then returns that report before running the current CostUsagePricing/aggregation code, so costs can remain calculated with old application logic until another key input changes; include an application/report-logic version in the persisted key or invalidate this sidecar across relevant upgrades.

Useful? React with 👍 / 👎.

@clawsweeper clawsweeper Bot added merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Aug 30, 2026
@clawsweeper

clawsweeper Bot commented Aug 30, 2026

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge. Reviewed August 30, 2026, 6:00 AM ET / 10:00 UTC.

ClawSweeper review

What this changes

The PR saves Claude’s computed local cost report beside its JSON cache so an unchanged transcript inventory can be reused after app relaunch.

Merge readiness

Blocked until real behavior proof from a real setup is added - 4 items remain

Keep open: the persistent memo can return a report produced by an older app build after an upgrade, so cost totals or aggregation remain stale until an unrelated source or artifact changes. The PR also needs real after-fix behavior proof beyond its synthetic tests.

Priority: P2
Reviewed head: 778b3793c2bd8e67fcafb753dd2dde2635ef85fb

Review scores

Measure Result What it means
Overall readiness 🦪 silver shellfish (2/6) The patch has focused synthetic coverage but needs an upgrade invalidation repair and real after-fix behavior evidence.
Proof confidence 🦪 silver shellfish (2/6) Needs real behavior proof before merge: The changed production owner is the Claude report memo loaded by the daily scanner, and the added isolated tests exercise in-memory eviction and sidecar reuse; the supplied PR evidence contains no after-fix run on a real Claude corpus or observed cold-start result. Please post redacted real-run output, logs, or a terminal screenshot showing the unchanged-inventory relaunch path and result; after updating the PR body, a fresh review should trigger, or a maintainer can request @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Patch quality 🦐 gold shrimp (3/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Needs proof Needs real behavior proof before merge: The changed production owner is the Claude report memo loaded by the daily scanner, and the added isolated tests exercise in-memory eviction and sidecar reuse; the supplied PR evidence contains no after-fix run on a real Claude corpus or observed cold-start result. Please post redacted real-run output, logs, or a terminal screenshot showing the unchanged-inventory relaunch path and result; after updating the PR body, a fresh review should trigger, or a maintainer can request @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed 6 items Current main lacks disk persistence: The fetched main implementation has only an in-memory report memo; it cannot solve the requested cross-launch reuse behavior.
Introduced early return bypasses current report logic: A matching persisted memo is loaded and returned before the cache is decoded or the daily report is rebuilt.
Persisted compatibility key omits report-logic revision: The new envelope accepts version 1 indefinitely, while the report key contains provider, range, roots, and artifact stamps but no application or report-logic revision.
Findings 1 actionable finding [P2] Version the persisted report against report semantics
Security None None.

How this fits together

CodexBar scans local Claude transcripts into a JSON cache and derives daily cost reports for usage and spend views. This PR adds a sidecar report memo between that cache/transcript inventory and the report consumers.

flowchart LR
A[Claude transcript files] --> B[Cost scan and JSON cache]
B --> C[Report memo validation]
C -->|valid| D[Saved daily report]
C -->|invalid| E[Decode cache and reprice]
D --> F[Usage and spend views]
E --> F
Loading

Before merge

  • Add real behavior proof - Needs real behavior proof before merge: The changed production owner is the Claude report memo loaded by the daily scanner, and the added isolated tests exercise in-memory eviction and sidecar reuse; the supplied PR evidence contains no after-fix run on a real Claude corpus or observed cold-start result. Please post redacted real-run output, logs, or a terminal screenshot showing the unchanged-inventory relaunch path and result; after updating the PR body, a fresh review should trigger, or a maintainer can request @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • Version the persisted report against report semantics (P2) - persistedVersion only guards the envelope format. With unchanged transcripts, JSON cache, and pricing artifact, a later build returns this saved report at the early-return path before current aggregation and alias logic runs, leaving totals stale until another input changes. Include a report-logic revision in the persisted contract and add an upgrade regression test.
  • Resolve merge risk (P1) - An upgrade that changes report aggregation, bundled model aliases, or pricing interpretation can show the previous build’s daily cost report while transcript, JSON-cache, and pricing-file stamps remain unchanged.
  • Complete next step (P2) - A concrete upgrade-compatibility defect and missing real behavior proof block merge; the PR needs contributor updates rather than an automated repair lane.

Findings

  • [P2] Version the persisted report against report semantics — Sources/CodexBarCore/Vendored/CostUsage/CostUsageClaudeCache.swift:68
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Patch surface 176 added, 22 removed across 7 files The change introduces a new persisted cache contract plus focused memo and dashboard coverage.
Production versus tests production +160, tests +34 Most of the patch creates the persisted-report serialization and storage path, making upgrade validation important.

Merge-risk options

Maintainer options:

  1. Version report semantics (recommended)
    Add a report-logic revision to the persisted memo contract and cover a simulated app upgrade so unchanged source artifacts cannot return a prior build’s report.
  2. Pause the persistence optimization
    Defer the sidecar until its upgrade-invalidation contract and real cold-start proof are available.

Technical review

Best possible solution:

Version the sidecar against report semantics, invalidate it when that revision changes, and demonstrate an upgraded cache being rebuilt before merge.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible: save a sidecar, change report semantics between builds while leaving transcript, cache, and pricing stamps unchanged, then reload. The early return skips the current report builder; this review did not execute the scenario.

Is this the best way to solve the issue?

No. Persisting the memo is a reasonable performance approach, but its acceptance key must include an explicit report-semantics revision and upgrade regression coverage.

Full review comments:

  • [P2] Version the persisted report against report semantics — Sources/CodexBarCore/Vendored/CostUsage/CostUsageClaudeCache.swift:68
    persistedVersion only guards the envelope format. With unchanged transcripts, JSON cache, and pricing artifact, a later build returns this saved report at the early-return path before current aggregation and alias logic runs, leaving totals stale until another input changes. Include a report-logic revision in the persisted contract and add an upgrade regression test.
    Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.98

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 39c15c6ad7f6.

Labels

Label changes:

  • add P2: The introduced sidecar can retain inaccurate cost-report output across an upgrade, but the issue is bounded to unchanged local Claude cache inputs.
  • add merge-risk: 🚨 compatibility: The new persisted report has no report-logic compatibility revision, so a release can reuse results produced by an older binary.
  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The changed production owner is the Claude report memo loaded by the daily scanner, and the added isolated tests exercise in-memory eviction and sidecar reuse; the supplied PR evidence contains no after-fix run on a real Claude corpus or observed cold-start result. Please post redacted real-run output, logs, or a terminal screenshot showing the unchanged-inventory relaunch path and result; after updating the PR body, a fresh review should trigger, or a maintainer can request @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

  • P2: The introduced sidecar can retain inaccurate cost-report output across an upgrade, but the issue is bounded to unchanged local Claude cache inputs.
  • merge-risk: 🚨 compatibility: The new persisted report has no report-logic compatibility revision, so a release can reuse results produced by an older binary.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The changed production owner is the Claude report memo loaded by the daily scanner, and the added isolated tests exercise in-memory eviction and sidecar reuse; the supplied PR evidence contains no after-fix run on a real Claude corpus or observed cold-start result. Please post redacted real-run output, logs, or a terminal screenshot showing the unchanged-inventory relaunch path and result; after updating the PR body, a fresh review should trigger, or a maintainer can request @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence

What I checked:

Likely related people:

  • Peter Steinberger: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Add a report-semantics revision and an upgrade regression that proves a stale sidecar is rebuilt.
  • Post redacted real-corpus cold-start evidence showing the saved report is reused only when its inputs and semantics remain valid.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants