This project provides a hands-on tutorial for understanding and implementing the Proximal Policy Optimization (PPO) algorithm to fine-tune Large Language Models (LLMs) using Reinforcement Learning (RL). It is inspired by the logic found in the TinyZero repository but significantly simplified for pedagogical purposes.