Train a 1B LLM with 1T tokens from scratch by personal

A straightforward method for training your LLM, from downloading data to generating text.

使用deepspeed从头开始训练一个LLM,经过pretrain和sft阶段,验证llm学习知识、理解语言、回答问题的能力

A companion to CMU professor Zico Kolter's Intro to Modern AI. Learn the basics of machine learning, then train your own LLM from scratch.

Multi-model video-to-text by combining embeddings from Flan-T5 + CLIP + Whisper + SceneGraph. The 'backbone LLM' is pre-trained from scratch on YouTube (YT-1B dataset).

No description available

Implement and train a Tiny LLM from scratch!

Trained a 114 million Parameter LLM from Scratch.

Personal project to learn how to build and pre-train a modern LLM from scratch.

Train LLMs such as GPT and LLama from scratch.

A simple but somehow complete(though not enough) project to train a LLM from scratch

Train LLM from scratch with SOTA techniques like RoPE, GQA and KV caching.

A single script to train an LLM from scratch for less than $100 and run inference through a ChatGPT-like Web Interface or a CLI.

Train your own LLM from scratch

🦐 A minimal pretraining trainer for LLMs — from scratch.

AI LLM Project that barks trained from scratch

Generative Pretrained Model (GPT) in JAX. A step by step guide to train LLMs on large datasets from scratch

Train a Hindi LLM from Scratch in pure Pytorch.

A project to pre-train a small LLM from scratch

LLM Forge: Experimental playground for building and training practical LLMs with limited compute resources. Pre-train from scratch, fine-tune, and generate text using modular GPT-style models

This repository contains a small-scale Transformer-based LLM built from scratch using PyTorch and Hugging Face. The model is trained on Wikipedia data and deployed via FastAPI.

Lightweight LLM inspired by Qwen3, built from scratch in PyTorch. Full training pipeline with transformer components including RMSNorm, Rotary Position Embeddings (RoPE), Grouped-Query Attention (GQA), and SwiGLU layers. Trained with hybrid Muon + AdamW optimizer, causal masking, efficient batching, and evaluation tools.

No description available

A simplistic script for training your own hybrid llm (using autoregressive model for drafting and diffusion model for refining).

Code & Train Anthropic Claude style LLM from scratch.

~58M parameter transformer LLM trained from scratch for Git/GitHub developer assistance.

LLM built from scratch predicting the next word trained on self defined weights.

AI web application, using custom data-set to train and build a LLM from scratch to provide assistance for fraud detection and protfolio management

This is the official UI for UrduGPT — a custom-built English → Urdu translator powered by a Transformer-based LLM trained from scratch using PyTorch.

This project implements an LLM from scratch, utilizing the transformer mechanism similar to that of GPT-2. The model is trained on a customer service dataset.