Goal
Move the PEFT support required by Megatron-Bridge into Megatron-LM so Bridge PEFT recipes can use megatron.training as the canonical training loop.
Scope
- Move performant LoRA, canonical LoRA, DoRA linear adapters, and the
LoRALinear wrapper into Megatron Core.
- Apply adapters and freeze the base model before model wrapping in the Megatron-LM training loop.
- Add adapter-only checkpoint save and load support to
megatron.training.checkpointing.
- Migrate Megatron-Bridge PEFT recipes to the Megatron-LM training loop.
Completion criteria
- Existing Megatron-Bridge PEFT APIs continue to work through the migration.
- Existing PEFT recipes retain their current behavior when run through the Megatron-LM training loop.
- Representative performant LoRA, canonical LoRA, and DoRA recipes preserve convergence and performance within agreed tolerances.
- Adapter-only checkpointing, adapter merging, and adapter export continue to work.
Goal
Move the PEFT support required by Megatron-Bridge into Megatron-LM so Bridge PEFT recipes can use
megatron.trainingas the canonical training loop.Scope
LoRALinearwrapper into Megatron Core.megatron.training.checkpointing.Completion criteria