Minimal yet high performant code for pretraining llms. Attempts to implement some SOTA features. Implements training through: Deepspeed, Megatron-LM, and FSDP. WIP
Stars
6
Forks
0
Watchers
6
Open Issues
0
Overall repository health assessment
No package.json found
This might not be a Node.js project
64
commits