Implemented a character-level GPT from scratch in PyTorch, architecting and comparing a novel implementation of Rotary Positional Embeddings (RoPE) against a vanilla transformer. Improved sequence generation accuracy from less than 10\% to over 30\% by leveraging RoPE and a full pre-training/fine-tuning pipeline.