🍀 Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.⭐⭐⭐

A PyTorch implementation of external attention.

No description available

Goal: Benchmark attention/MLP layers on CPU, GPU, and Apple M-series. Deliverable: Latency, FLOPs utilization, memory bottlenecks. Grading: Benchmark correctness (30%), Comparison depth (40%), Report (30%). Reference: pytorch/examplesLinks to an external site.

No description available

No description available

No description available

Developed a custom 211,406 parameter autoregressive language model from scratch using PyTorch, implementing core components like multi-head self-attention, positional embeddings, and layer normalization. Built a character-level tokenizer, custom training loop, and data pipeline using NumPy and PyTorch with zero external NLP libraries.

🧸 A fully custom GPT-style language model built from scratch using PyTorch and trained on Winnie-the-Pooh! Explored the core mechanics of self-attention, autoregressive text generation, and modular model training, all without relying on any external libraries.

MiniGPT is a lightweight, self-contained implementation of a GPT-style transformer built entirely in PyTorch, without relying on external deep learning libraries like transformers. This project serves as an educational tool for understanding the inner workings of transformers, self-attention, and language modeling.