This project discusses the complete pipeline, of creating an end-to-end system of finetuning the LLMs, with PEFT STTF and DPO. This project explanation is also available in the medium article which can be accessed through this link https://medium.com/dev-genius/how-to-harness-peft-sftt-and-dpo-to-fine-tune-llms-394e9cd0b150.