Finetune Falcon, LLaMA, MPT, and RedPajama on consumer hardware using PEFT LoRA

Patch for MPT-7B which allows using and training a LoRA

A repo for finetuning MPT using LoRA. It is currently configured to work with the Alpaca dataset from Stanford but can easily be adapted to use another.

Testing MPT 7B finetuning using LORA

No description available

Simple Example to train MPT 30B (Single GPU and DDP) model using LORA and Int8 training