Practice in understanding and coding LLMS from scratch, following https://karpathy.ai/zero-to-hero.html
Stars
2
Forks
0
Watchers
2
Open Issues
0
Overall repository health assessment
No package.json found
This might not be a Node.js project
19
commits
Added word level transformer. Started expirementing wth openAI tiktoken tokenizer
9f8a479View on GitHubtransformer.ipynb was getting messy due to experimentation, so made transfromer_playground.ipynb and cleaned up the original. Added LLMTV, which is supposed ot be a music generating project and will me moved to its own repo if started seriously
e624fcdView on GitHubAdded code to load and generate from model_finalvideo_shakespeare, which is the model in transformer.ipnyb trained for 40 minuts on free T4 from google colab
3ce1751View on GitHubFinished implementation of attention is all you need in trasnformer.ipynb, trained on T4 on google colab
c332f38View on GitHubAdded transformer.ipynb as a cleaned up version of the transofmer architecture, leaving gpt as a testing/ learning playground
c7197aaView on GitHubfurther work on implementing transformer architecture for simple makemore model
d7ea95aView on GitHubAdded bigram model to gpt.ipynb for comparison, added bigram.py as a script
85d68bcView on GitHubfinished wavenet-like network implementation, did some tests
aff7fd8View on GitHubAdded manual backpropogation through multilayer perceptron as an exercise
79fae1bView on GitHubvarious changes, added some tests, added makemore_mlp_pt2.ipynb
094b16dView on GitHubfurther work on optimized_mlp, saved a trained model model.pt
ce195cbView on GitHub