Efficient Triton Kernels for LLM Training

JAX Implementation of Liger Kernels

This repository contains code, experiments, and a report for a study on the Liger Kernel and its application in memory-efficient training of Large Language Models (LLMs). The report analyzes the Liger Kernel's techniques, including fused kernel operations and chunking strategies, and demonstrates its impact on memory usage and throughput.

A proof of concept wrapper to wrap your pytorch modules with liger kernels.

Redistributed Liger Kernel for RenderFormer

Benchmarking the performance of Liger Kernels Library Using instruction following and reasoning tasks

eagle model & trainer with liger kernel

No description available

No description available

No description available

Student project for demonstration of liger kernel python module

No description available

No description available

Fix gradient accumulation support in 360-LLaMA-Factory for Liger Kernel - enables 80%+ memory savings

High-performance Triton kernel library for LLM training with 12 fused operators (AttnRes, RMSNorm, RoPE, CrossEntropy, GRPO, JSD, FusedLinear, etc.) — up to 24x faster than PyTorch with 78% memory savings, outperforming Liger-Kernel on RTX 5090

A deep dive into fine-tuning large language models from scratch. Inspired by LLM.c (built from scratch) and Liger-Kernel/Unsloth (kernel fusion), this project explores techniques such as CPU offloading, gradient checkpointing, and tensor lifetime management, leveraging Triton, C++, PyTorch, and more.

This repository provides a simplebenchmarking suite for measuring the latency of a single forward pass through a HuggingFace Transformer model on CUDA GPUs. It supports both the standard PyTorch implementation and an optional Liger kernel acceleration for causal‑language models.