Repository navigation
Add teacher-to-student overnight training guide - #2
Merged
Merged
Conversation
ManagerBasedRLEnv.step resets terminated environments before returning, so play.py read pose and velocity error from the freshly reset episode for exactly the environments that had just finished. Terminal-error evidence was therefore not trustworthy. TerminalMetricsMixin intercepts _reset_idx while the scene still holds the terminal physics state. The hook is inert unless an evaluator installs it, so task physics, observation/action dimensions, rewards, and checkpoint compatibility are unchanged; a two-iteration resume from the epoch-700 checkpoint still trains through the new entry point. InsertionTerminalMetrics records one row per completed episode: success, categorized termination reason, curriculum stage, control steps, cycle time, axial/lateral/orientation error, blade linear and angular velocity, and tool-to-handle position and orientation error. A physics blow-up or mount failure in the same control step as a geometric success is categorized as the instability, never as a success. aggregate_evaluation.py pools per-run rows into count, mean, p50, p95, max, success rate, and Wilson 95% intervals, then applies the promotion gate. Held-out certification of the epoch-700 Level-0 policy, three curriculum stages by seeds 1060/2060/3060, 128 environments, nine runs: 9,086/9,086 successes, Wilson 95% lower bound 0.9996, zero instability and zero non-finite terminations. Every stage is 100%. Terminal axial error reaches 11.96 mm of the 12 mm limit and orientation 0.0484 of the 0.0524 rad limit, so Level-0 margin is thin on exactly the two axes rail contact will stress. Simulation evidence only. The blade is held by a PhysX fixed joint standing in for an already-secured grasp; this is not learned or physically validated grasping and not Sim2Real validation. Ruff clean, 47/47 non-Sim tests pass, Isaac smoke passed for the corrected evaluator and for checkpoint-resume training. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fine-tuned the epoch-700 Level-0 checkpoint for 500 more PPO epochs at 512 environments, seed 61, robustness level 1: physical wide side-rail collision plus doubled reset joint noise. Observation and action dimensions are unchanged, so resuming the checkpoint is valid. Held-out certification of epoch 1200 on seeds 1061/2061/3061 across all three curriculum stages, 128 environments, nine runs: 9,014/9,014 successes, Wilson 95% lower bound 0.9996, zero instability and zero non-finite terminations. Every stage is 100%. Contact did not merely preserve success, it improved terminal precision: stage-0 axial error fell from 4.15 mm mean at Level 0 to 1.65 mm at Level 1. Terminal angular velocity roughly doubled to 0.025 rad/s, the expected signature of real rail contact, and stays well inside the 0.080 rad/s limit. Simulation evidence only. The blade is still held by a PhysX fixed joint standing in for an already-secured grasp; blade mass is fixed and rail stiction is absent at this level. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Level 2 randomizes blade mass over 5-15 kg, and the project rule is at least 80% success in every mass bucket, not just on average. The existing per-parameter bucket counter lived in play.py and could not be pooled across runs, so a heavy-blade weakness could hide in an aggregate. InsertionTerminalMetrics now records blade mass as an extra column when the task actually randomizes it. Because that changes the row width, reports align runs by column name instead of position: align_rows widens or narrows a run to the pooled field list and fills genuinely absent columns with NaN, and the finiteness check now covers physical and categorical columns only so missing optional data is never miscounted as a physics instability. bucket_success_rates splits any continuous randomized parameter into low/mid/ high bands with per-band Wilson intervals, and the aggregator fails the gate when the worst band falls below 80%. Verified result-preserving: regenerating the committed Level-0 and Level-1 reports through the new path reproduces every numeric block unchanged. Ruff clean, 50/50 non-Sim tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CLAUDE.md loads on every agent session, so its 13.8 KB of result tables, limitation lists, prior art, and a now-completed two-shot mandate were paid for on every task regardless of relevance. It is now a 4.2 KB routing file: mission, operating rules, the evaluation contract, and a table saying which document to open for a given kind of work. Every fact moved rather than disappeared. docs/status.md holds the stack, promoted checkpoints, measured tables, robustness profile states, static validation, and known failures. docs/roadmap.md holds ordered next work and the research basis. docs/claim_vs_evidence.md is new: a one-page separation of what was measured from what is not established, written for a skeptical reader, with the orbital-servicing motivation stated up front. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fine-tuned the Level-1 epoch-1200 checkpoint for 600 more PPO epochs at 512 environments, seed 62, robustness level 2: tight 1.5 mm side-rail clearance plus blade mass randomized over 5-15 kg with recomputed inertia. Held-out certification of epoch 1800 on seeds 1062/2062/3062 across all three curriculum stages: 9,021/9,021 successes, Wilson 95% lower bound 0.9996, zero instability and zero non-finite terminations. Every stage is 100%, and so is every mass band across an observed 5.00-14.97 kg, which is the gate that matters for a randomized payload. Full-distance median cycle time improved to 7.20 s from 7.90 s at Level 0 despite tighter clearance and a threefold mass range. Recorded honestly: terminal orientation error now reaches 0.0512 rad of the 0.0524 rad limit, 97.8% of budget, at both stage 1 and stage 2. Orientation is the axis with the least headroom and the predicted first failure mode when Level-3 rail stiction is enabled, which is where the roadmap now points. Also adds an 'array' inspection view that frames the whole parallel environment grid, and corrects the Level-1 per-stage episode counts to the values in the committed report. Three levels, 27,121 held-out simulated episodes, 100%. Simulation evidence only; the blade is still held by a fixed joint standing in for a grasp. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three defects made a multi-environment recording useless. The video folder was derived from the checkpoint path, so a second run silently overwrote the first. The array camera guessed the cloner's grid layout and framed empty space. And Fabric-cloned prims do not all reach the RTX renderer, so most workcells were missing from the frame entirely. --video_dir now selects the output folder, the array view reads the actual scene.env_origins bounding box and places the camera from it, and recording runs with more than one environment disable Fabric cloning. Recording is presentation only and uses few environments, so the cloning speedup is worth trading for correct visuals. Physics, observations, and actions are unchanged. Verified by extracting frames: the array clip now shows all nine UR10e workcells holding and inserting their blades under the promoted Level-2 policy, 16/16 and 9/9 episodes successful in the recorded runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Why
The repository now explains not only how to launch training, but why there are two PPO sessions, what is transferred between them, what can safely run overnight, and what evidence is required before calling a policy ready.
Validation
git diff --checkpassed