Back to search
Implemented a character-level GPT from scratch in PyTorch, architecting and comparing a novel implementation of Rotary Positional Embeddings (RoPE) against a vanilla transformer. Improved sequence generation accuracy from less than 10\% to over 30\% by leveraging RoPE and a full pre-training/fine-tuning pipeline.
Stars
0
Forks
0
Watchers
0
Open Issues
0
Overall repository health assessment
No package.json found
This might not be a Node.js project
1
commits
Initial commit: Next-Gen Language Model Development From Scratch - GPT with Rotary Positional Embeddings
39b4dceView on GitHub