This repository contains the complete source code, explanations, and visualizations for the "Building LLMs from Scratch" series. Whether you're a beginner curious about how ChatGPT works or an experienced developer wanting to understand transformer architecture deeply, this series will guide you through every component step by step.
Stars
5
Forks
0
Watchers
5
Open Issues
0
Overall repository health assessment
No package.json found
This might not be a Node.js project
23
commits
Update README.md to reflect the completion of Part 9: Multi-Head Attention, and adjust the status of upcoming sections in the educational series roadmap for clarity and consistency.
26ce3f4View on GitHubUpdate links for Part 8: Causal Attention across README.md, Jupyter notebook, and Python module to ensure consistency and accuracy in documentation. The new links direct to the correct Medium article.
e8a418bView on GitHubAdd Part 8: Causal Attention (Masked Self-Attention) implementation, including updates to README.md, new medium article, Jupyter notebook, and Python module. This part addresses the future leakage problem in autoregressive models by implementing causal masking, ensuring proper attention behavior during text generation.
a69ada9View on GitHubUpdate README.md to reflect the completion of Parts 6 (The Attention Mechanism) and 7 (Self-Attention with Trainable Weights), and adjust the status of upcoming sections, ensuring clarity in the educational series roadmap.
d05a238View on GitHubUpdate Part 7: Self-Attention with Trainable Weights to include the correct article link across README, notebook, and Python module, ensuring consistency in documentation.
e74dae9View on GitHubUpdate README.md and related files to reflect the correct link for Part 6 (The Attention Mechanism). Adjust notebook and Python module to include the updated article link, ensuring consistency across documentation.
b125357View on GitHubUpdate README.md, medium article, and Python module to reflect the correct link for Part 5 (Complete Data Preprocessing Pipeline). Adjust notebook to include the updated article link and add a blank markdown cell for future content.
819d748View on GitHubUpdate README.md to include the completion of Part 5 (Complete Data Preprocessing Pipeline) and adjust the roadmap status for upcoming parts, ensuring clarity in the series progression.
eccb59dView on GitHubUpdate README.md to reflect the completion of Part 4 (Token Embeddings & Positional Encoding) and adjust the status of the Self-Attention Mechanism. Update links in the notebook and Python module to point to the correct Medium article for Part 4.
e373fabView on GitHubRefactor vector embedding demonstration notebook to improve clarity and organization, enhancing the user experience for word vector operations and similarity calculations.
2447876View on GitHubAdd Jupyter notebook for vector embedding demonstration, showcasing word vector loading, similarity calculations, and vector operations using Gensim.
9e7cb50View on GitHubAdd torch to requirements.txt for enhanced model support
655bd36View on GitHubUpdate README.md to reflect the completion status of Part 3 (Data Pipeline) and adjust the roadmap for upcoming parts, including changes to status indicators and links.
5198a5dView on GitHub