All pull requests

Pull requests

• Fix cosine LR climbing back up after lr_decay_iters

  #1438·MohammadHijjawi97 opened Sep 28, 2026·

  ·

• Warn when top-k MoE routing is configured for training (no load-balancing loss)

  #1437·rohan-patnaik opened Sep 24, 2026·

  ·

• Add head dim support

  #1436·aflah02 opened Sep 20, 2026Contributor·

  ·
  2

• Fix gradient clipping configuration and remove clip_grad

  #1433·aflah02 opened Sep 18, 2026Contributor·

  ·
  2

• Fix unused μP attention and output temperature arguments

  #1430·aflah02 opened Sep 15, 2026Contributor·

  ·
  1

• Remove unused sparse_attention argument

  #1429·aflah02 opened Sep 15, 2026Contributor·

  ·
  1

• Fix FP16/BF16 configuration handling and document mixed-precision options

  #1428·aflah02 opened Sep 15, 2026Contributor·

  ·
  1

• Remove unused derived ZeRO arguments

  #1427·aflah02 opened Sep 15, 2026Contributor·

  ·

• Use unused weight_server_port arg

  #1426·aflah02 opened Sep 15, 2026Contributor·

  ·

• Improving QK Layernorm

  #1424·aflah02 opened Sep 9, 2026Contributor·

  ·
  5

  

• fix(neox_args): mamba/rwkv hidden_dropout assertions never fire (1-tuple always truthy)

  #1422·Anai-Guo opened Sep 7, 2026·

  ·

• fix(mpu): ColumnParallelLinear.mup_reinitialize_weights raises NameError on keep_master_weight_for_test

  #1421·Anai-Guo opened Sep 7, 2026·

  ·

• CI: install dependencies before preparing test data

  #1419·LOGO127 opened Sep 4, 2026·

  ·
  1

  1

• Add Inter Doc Attn Masking

  #1416·aflah02 opened Sep 2, 2026Contributor·

  ·
  4

  

• Abstraction for Intel GPUs

  #1414·jahatef opened Aug 11, 2026Collaborator·

  ·
  1

• Add HF Supported Inference Time Sampling Args

  #1411·aflah02 opened Jul 20, 2026Contributor·

  ·

• Add support for extra_eval_iters

  #1410·aflah02 opened Jul 20, 2026Contributor·

  ·

• OLMo2 Dense Model Training Configs

  #1408·aflah02 opened Jul 17, 2026Contributor·

  ·
  4

  

• Qwen 2.5 Dense Model Training Configs

  #1406·aflah02 opened Jul 17, 2026Contributor·

  ·

• feat: add optional torchembed fused Triton RoPE kernel (use_torchembed_rope)

  #1401·py-ai-dev opened Jul 6, 2026·

  ·

• Qwen 3 Dense Model Training Configs

  #1399·aflah02 opened Jun 27, 2026Contributor·

  ·
  2

• Qwen 2 Dense Model Training Configs

  #1397·aflah02 opened Jun 26, 2026Contributor·

  ·

• Qwen 1.5 Dense Model Training Configs

  #1396·aflah02 opened Jun 26, 2026Contributor·

  ·

• Validate norm type when converting checkpoints to HF

  #1390·discobot opened Jun 13, 2026·

  ·
  1

  1

• Torch Run Launcher

  #1388·aflah02 opened Jun 2, 2026Contributor·

  ·
  2