feat(comp): skip impact analysis for unchanged compositions - #458
Draft
jcogilvie wants to merge 1 commit into
Draft
feat(comp): skip impact analysis for unchanged compositions#458jcogilvie wants to merge 1 commit into
jcogilvie wants to merge 1 commit into
Conversation
jcogilvie
force-pushed
the
fix/452-exclude-deleting-xrs
branch
from
September 10, 2026 19:41
0fad271 to
e877070
Compare
jcogilvie
force-pushed
the
fix/453-skip-impact-analysis-unchanged-composition
branch
from
September 10, 2026 19:46
e8f8d4e to
0e03d0d
Compare
5 tasks
jcogilvie
force-pushed
the
fix/452-exclude-deleting-xrs
branch
2 times, most recently
from
September 11, 2026 20:04
f8b5b8d to
d507a4b
Compare
5 tasks
jcogilvie
force-pushed
the
fix/453-skip-impact-analysis-unchanged-composition
branch
from
September 11, 2026 20:11
0e03d0d to
e153d1c
Compare
jcogilvie
force-pushed
the
fix/453-skip-impact-analysis-unchanged-composition
branch
from
September 11, 2026 20:22
e153d1c to
a307f0b
Compare
Applying a composition that is identical to its in-cluster version creates no new CompositionRevision, so no XR adopts anything it hasn't already adopted. Any downstream delta computed in that situation is therefore caused by something other than the composition — drift, convergence lag, or a modeling artifact of this tool (we render with our own pulled function images against our own reconstruction of observed state) — and the tool cannot tell those apart. Reporting them under "Impact Analysis" attributes cluster state to a change that does not exist, and it also returned ExitCodeDiffDetected for a composition the user did not change, which in CI reads as "this composition change has impact". So the per-XR work — one function render per affected XR, the dominant cost of comp — is now skipped for an unchanged composition, evaluated per-composition so a mixed input behaves sensibly. The two impact sections are replaced by an explicit note, and CompositionDiff.ImpactAnalysisSkipped (serialized as impactAnalysisSkipped) lets consumers tell "not evaluated" from "no affected XRs found". --analyze-unchanged opts back in. The one workflow the skip would otherwise remove is legitimate: establishing an "is my cluster converged?" baseline before editing a composition, so that any diff afterwards is attributable to the edit. There is no other way to express that today, since xr requires each XR file individually. The gate is deliberately NOT the displayed diff. calculateCompositionDiff now returns a compositionComparison carrying both the diff to show (nil when equal) and whether the composition changed at all; the two come apart only under --ignore-paths. Gating on the displayed diff would mean that masking a path which is load-bearing for rendering (anything under spec.pipeline[].input, say) silently skips the analysis for a composition that genuinely changes the rendered output. Fixes #453 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Jonathan Ogilvie <jonathan.ogilvie@sumologic.com>
jcogilvie
force-pushed
the
fix/453-skip-impact-analysis-unchanged-composition
branch
from
September 11, 2026 20:34
a307f0b to
1e079d1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of your changes
Note
Stacked on #457 (issue #452). Base is
fix/452-exclude-deleting-xrs, so the diff shown here is #453's change alone. Retarget tomainonce #457 merges.Applying a composition identical to its in-cluster version creates no new
CompositionRevision, so no XR adopts anything it hasn't already adopted. Any downstream delta computed in that situation is therefore caused by something other than the composition, and the tool cannot tell the causes apart:EnvironmentConfigor other required resource changed and the XRs haven't re-reconciled yet.Reporting these under
=== Impact Analysis ===attributes cluster state to a change that doesn't exist. It also returnedExitCodeDiffDetected(3) for a composition the user didn't change —CompositionDiff.HasChanges()returns true on anyXRStatusChangedimpact even whenCompositionDiff == nil— which in CI reads as "this composition change has impact" and would gate a pipeline on drift the user didn't cause.So the per-XR work (one function render per affected XR — the dominant cost of
comp) is now skipped for an unchanged composition, evaluated per-composition so a mixed input behaves sensibly. The two impact sections are replaced by an explicit note, andCompositionDiff.ImpactAnalysisSkipped(serialized asimpactAnalysisSkipped) lets structured-output consumers distinguish "not evaluated" from "no affected XRs found".Full reasoning, including the nested-composition case and the arguments considered on the other side, is written up in this comment on #453.
--analyze-unchangedopts back inOne workflow the skip would otherwise remove is legitimate: establishing an "is my cluster converged?" baseline before editing a composition, so any diff afterwards is attributable to the edit. There's no other way to express that today, since
xrrequires each XR file individually. The flag is cheap —ProcessorConfigalready carriesIncludeManual,EventualState, andIgnorePathsin exactly this shape.The gate is deliberately NOT the displayed diff
calculateCompositionDiffnow returns acompositionComparisoncarrying both the diff to show (nil when equal) and whether the composition changed at all. The two come apart only under--ignore-paths, which is applied to the composition comparison too. Gating on the displayed diff would mean that masking a path which is load-bearing for rendering — anything underspec.pipeline[].input, say — silently skips the analysis for a composition that genuinely changes the rendered output. The second comparison is local (no API calls) and only runs when--ignore-pathsis in play and the masked diff came back equal.Testing
TestDefaultCompDiffProcessor_calculateCompositionDiffis a new table pinning the display-diff /changedsplit, including the case where--ignore-pathsmasks the only difference — that must still count as changed.TestDefaultCompDiffProcessor_DiffCompositiongainsUnchangedCompositionSkipsImpactAnalysis(skip note present, both section headers absent) andUnchangedCompositionWithAnalyzeUnchanged(sections return).UnchangedCompositionSkipsImpactAnalysisandUnchangedCompositionAnalyzeUnchangedEvaluatesXRsuse the same fixtures deliberately — fixtures that DO produce a downstream delta when evaluated. So the pair asserts the contrast directly: skipped ⇒ no report and exit0;--analyze-unchanged⇒ the downstream modification and exit3. That delta (an annotation-key discrepancy in the fixtures) is itself a clean example of a bucket-3 artifact, which is why the opt-in is explicit.Two existing tests changed, both deliberately
CompositionDiffIgnorePathsfeedscomposition-no-changes.yamland asserts on the impact sections to validate XR-level ignore paths. It now passes--analyze-unchanged, so it keeps testing what it was written to test rather than being rewritten.MultipleCompositionDiffImpactturns out to feed a second composition (xnopresources-v2.diff.example.org) that is byte-identical to its in-cluster version —updated-composition-2.yamlandoriginal-composition-2.yamlare the same file. It has therefore always been a mixed changed/unchanged case rather than the "two changed compositions" its name suggests. Rather than change the fixture, I updated its expectation and reworded itsreasonto say what it actually covers, so it now serves as the per-composition-skip case. Flagging it explicitly in case you'd rather I make the fixture genuinely differ instead.TestDefaultCompDiffProcessor_DiffComposition_StderrErrorOutputalso needed its input composition to differ from the cluster's (a label), since otherwise the XR failure it tests became unreachable.Fixes #453
I have:
earthly -P +reviewableto ensure this PR is ready for review.README.md; architecture indesign/design-doc-cli-diff.mdand its diagrams).Need help with this checklist? See the cheat sheet.
🤖 Generated with Claude Code