Training

Approaches and techniques for training and adapting models.

Topics

• Pretraining corpora and scaling laws
• Supervised fine-tuning (SFT)
• Preference optimization (DPO/RLHF)
• LoRA/QLoRA and PEFT
• Distillation and continued pretraining