🚀🚀 「大模型」2小时完全从0训练64M的小参数GPT！🌏 Train a 64M-parameter GPT from scratch in just 2h!

《Build a Large Language Model (From Scratch)》是一本深入探讨大语言模型原理与实现的电子书，适合希望深入了解 GPT 等大模型架构、训练过程及应用开发的学习者。为了让更多中文读者能够接触到这本极具价值的教材，我决定将其翻译成中文，并通过 GitHub 进行开源共享。

🔥 LeetCode for PyTorch — practice implementing softmax, attention, GPT-2 and more from scratch with instant auto-grading. Jupyter-based, self-hosted or try online.

Tiny GPT implemented from scratch in pure Go. Trained on Jules Verne books. Explained.

gpt-2 from scratch in mlx

Discover the power of GPT-3.5 turbo in presentation creation! 💻🤩 This web application can effortlessly generate captivating PowerPoint presentations. Say goodbye to the struggle of starting from scratch and let AI revolutionize your content creation process. 🎨

Building a GPT-like LLM from scratch with PyTorch.

A PyTorch implementation of the GPT-OSS-20B architecture. All components are coded from scratch: RoPE with YaRN, RMSNorm, SwiGLU with clamping and residual connection, Mixture-of-Experts (MoE), Self-Attention with learned sinks, banded attention, GQA, and KV-cache.

implement GPT-OSS 20B & 120B C++ inference from scratch on AMD GPUs

Educational implementation of a small GPT model from scratch in a single Jupyter Notebook

Autograd to GPT-2 completely from scratch

This repository contains the collection of explorative notebooks pure in python and in the language that we, humans can read. Have tried to compile all lectures from the Andrej Karpathy's 💎 playlist on Neural Networks - which we will end up with building GPT.

Implementation of GPT from scratch. Design to be lightweight and easy to modify.

Learn the building blocks of how to build gpt-oss from scratch

BabyGPT: Build Your Own GPT Large Language Model from Scratch Pre-Training Generative Transformer Models: Building GPT from Scratch with a Step-by-Step Guide to Generative AI in PyTorch and Python

From-scratch implementation of OpenAI's GPT-OSS model in Python. No Torch, No GPUs.

A truly open version of gpt-oss which shows the entire pre-training from scratch

A GPT-2 style transformer language model implemented from scratch in Rust for educational purposes.

Implementation of a GPT-4o like Multimodal from Scratch using Python

Andrej Karpathy's Let's build GPT: from scratch video & notebook implemented in Rust + candle

A codebase implementing a simple GPT-like model from scratch based on the Attention is All You Need paper.

This repository guides you through the process of building a GPT-style Large Language Model (LLM) from scratch using PyTorch. The structure and approach are inspired by the book Build a Large Language Model (From Scratch) by Sebastian Raschka.

Deep learning library implemented from scratch in numpy. Mixtral, Mamba, LLaMA, GPT, ResNet, and other experiments.

Combining ViT and GPT-2 for image captioning. Trained on MS-COCO. The model was implemented mostly from scratch.

An existential western RPG about semi-conscious GPT agents. Written in a 3d engine from scratch targeting Wasm/Webgl and Desktop

Build GPT from scratch using Nx and Axon

Deep learning library (training & inference) and GPT-2 in C++ from scratch.

Train a 29M parameter GPT from Scratch

This repository contains a step-by-step implementation of a ChatGPT-like model from scratch using PyTorch. It covers everything from tokenization and transformer architecture to training and fine-tuning on conversational data. The course is designed for those who want to deeply understand how large language models work and build their own! 🚀

PyTorch Implementation of GPT-2