该系列的目的是让读者可以在基础的pytorch上，不依赖任何其他现成的外部库，从零开始理解并实现一个大语言模型的所有组成部分，以及训练微调代码，因此读者仅需python，pytorch和最基础深度学习背景知识即可。

This is an ongoing project where I'm building a small LLaMA-like LLM from scratch.

This repository presents a comprehensive, step-by-step guide to building a small-scale Large Language Model (LLM) from scratch using Python and Jupyter Notebooks.

Building a small LLM from scratch

Building a small LLM model from scratch using PyTorch - A hands-on transformer implementation project

This project demonstrates building and training a small language model (LLM) from scratch using PyTorch.

Building a small-scaled LLM from scratch! Including: Tokenizer, RoPE, SwiGLU, RMSNorm, Transformer, Pre-Train, Post-Training...

Learn LLM fundamentals by building a small (~90M parameter) Transformer from scratch that translates Dutch to English.

Building an LLM from scratch specifically a small GPT-2 124M parameter model that is pre-trained on books from Project Gutenberg.

Building a small LLM from scratch"LLM from Scratch" Implementation Built a language model from the ground up following a tutorial. Implements tokenization, transformer architecture (attention, feedforward layers), and text generation. Includes training scripts, pretrained weights, and inference demos. Perfect for learning how LLMs work internal.

tinyLLM is a small LLM built from scratch on TinyStories dataset from HF using pytorch, GPT-2 and transformers architecture. Its meant for beginners to learn and understands the fundamentals of AI/ML by building a LLM from ground-up on low-level consumer hardware.

a very work in progress learning by building a small LLM from scratch and attempting to make a networking app in which you can create your own chatbot to interact with other user-generated chatbots,

A beginner-friendly toolkit for mastering Large Language Models from scratch. This repo demystifies core concepts, outlines prerequisites , curates key papers, and guides hands-on practice—running open-source LLMs locally, fine-tuning small models with LoRA/QLoRA, and building apps. It bridges theory and practice for aspiring LLM practitioners.