A PyTorch implementation of an embedding model with RoPE (Rotary Position Embedding), temperature scaling, and Masked Language Modeling (MLM) pre-training support. This is a research and learning project to test out various ideas about embedding models and how a good Machine Learning project should look like.