Improve and validate the edit-timeline-studio Agent Skill

[jackroc]

Goal

Improve the public edit-timeline-studio Agent Skill through reproducible, real-world evaluation and focused fixes.

The Skill already supports Codex, Claude Code, Copilot, and Gemini CLI, and can route work through either the versioned command runner or the browser editor. This issue is for improving how reliably an Agent discovers, interprets, executes, verifies, and hands off those workflows—not for adding unrelated editor features.

Relevant files:

• skills/edit-timeline-studio/SKILL.md
• skills/edit-timeline-studio/agents/openai.yaml
• skills/edit-timeline-studio/references/
• skills/edit-timeline-studio/scripts/validate_edit_plan.mjs
• skills/edit-timeline-studio/docs/

Issue #37 remains the capability matrix. This issue focuses on Skill quality and behavior.

Areas to investigate

Contributors may focus on one area:

• Triggering and discovery: false negatives, false positives, unclear user-intent coverage, or stale UI metadata.
• Context efficiency: duplicated guidance, unnecessary context loading, unclear reference routing, or missing progressive disclosure.
• Execution routing: choosing correctly between the hosted editor, local repository, command runner, and browser compatibility path.
• Planning safety: preserving the user's request, resolving assets explicitly, using dry runs, and avoiding invented IDs or unsupported operations.
• Browser workflow reliability: handling signed-in state, dynamic ports, responsive layouts, uploads, long-running local AI work, and recoverable UI failures.
• Verification and handoff: checking timeline structure, captions, audio, overlays, exports, editable project archives, and truthful limitations.
• Cross-Agent portability: differences observed in Codex, Claude Code, Copilot, Gemini CLI, skills.sh, or GitHub Skills CLI installation.
• Validation tooling: deterministic checks, fixtures, smoke tests, or evaluation scripts that make regressions easier to detect.

Contribution workflow

1. Start with a concrete user request that should trigger this Skill.
2. Run it from a clean Agent context against the current main branch.
3. Record the request, platform, execution path, observed behavior, expected behavior, and evidence.
4. Identify whether the failure belongs to Skill guidance, the command runner, browser control, the editor product, or the environment.
5. Propose the smallest Skill or validation change that addresses the demonstrated failure.
6. Rerun the same scenario and at least one adjacent scenario to check for regressions.

Please do not rewrite the whole Skill based only on preference. One reproducible failure mode or measurable improvement per pull request is preferred.

Suggested evaluation scenarios

• Open the hosted editor for an underspecified “let me edit this” request without inventing edits.
• Modify an existing .timeline project through project.diff and project.run while preserving media identity.
• Route an unsupported command-runner operation to the browser editor without claiming deterministic execution.
• Assemble visuals, captions, voiceover, music, or overlays and preserve an editable project.
• Verify a final export's dimensions, duration, decoded frames, visible composition, and real audio track.
• Recover clearly from a missing asset, unsupported capability, unavailable browser model, or cancelled operation.
• Install and invoke the Skill through one supported Agent platform using the documented instructions.

Contributors may propose other realistic scenarios.

Deliverables

A focused pull request should include:

• the reproducible before/after scenario;
• the smallest necessary change to SKILL.md, metadata, references, scripts, or docs;
• updated or new deterministic validation where applicable;
• evidence that the affected scenario now works;
• an adjacent regression check;
• a note identifying any product limitation discovered but intentionally left out of scope.

If SKILL.md metadata changes, keep agents/openai.yaml aligned. If a script changes, run it against both valid and invalid fixtures.

Acceptance criteria

• The pull request demonstrates a real Skill failure, ambiguity, inefficiency, or portability problem before changing guidance.
• The improvement is concise and preserves progressive disclosure; detailed material belongs in the relevant reference instead of bloating SKILL.md.
• Capability claims remain accurate and distinguish command-runner execution from browser UI automation.
• The Skill never invents assets, IDs, successful exports, deterministic guarantees, or unsupported capabilities.
• Destructive edits remain reversible and the editable timeline stays the source of truth.
• Existing plan validation, command-runner behavior, and browser compatibility guidance are preserved or improved.
• Relevant lint, type checks, tests, build, Skill validation, and scenario-specific checks pass.
• Public documentation is updated only when user-facing installation or behavior actually changes.

Contributor profile

This is suitable for contributors interested in Agent Skills, prompt and workflow design, video automation, browser control, evaluation, CLI tooling, or cross-Agent compatibility.

Please comment with the platform and scenario you want to investigate before starting so the scope can be confirmed and duplicate work avoided.