Smaller than nano! :)
Stars
1
Forks
0
Watchers
Open Issues
Overall repository health assessment
No package.json found
This might not be a Node.js project
User
31
commits
feat: remove bias, change ReLU to SwiGLU
d621ef9
feat: add wandb in train.py
7765190
feat: add temperature and top_k arguments in generate
ed1e28f
feat: add argparser for sample.py
3072b3d
update README.md
9da63f3
feat: modify log info in train.py
553a45b
feat: replace pos_embedding with RoPE in MHA
48e9a7d
feat: add job id in filename of checkpoint
fb3c849
feat: add tqdm for train.py
1705a4e
feat: add info log for sample.py
a33766a
fix: delete redundant <endoftext>
e197a5f
feat: modify train.py to load data with np.memmap
2e529f7
feat: add prepare.py
aeb4e57
feat: add weight tying
b1483cf
feat: LayerNorm -> RMSNorm, and add RoPE
b023f7a