The Curse of Depth in Large Language Models
Paper
•
2502.05795
•
Published
•
40
Training optimization
-
-
Transformers without Normalization
Paper • 2503.10622 • Published • 170 -
Parallel Scaling Law for Language Models
Paper • 2505.10475 • Published • 83 -
Learning to Skip the Middle Layers of Transformers
Paper • 2506.21103 • Published • 18 -
FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
Paper • 2506.20920 • Published • 75 -
All is Not Lost: LLM Recovery without Checkpoints
Paper • 2506.15461 • Published • 37 -
TiKMiX: Take Data Influence into Dynamic Mixture for Language Model Pre-training
Paper • 2508.17677 • Published • 14