The peft module contains state-of-the-art Parameter-Efficient Fine-Tuning
(PEFT) methods.

Fine-tuning large pretrained models is often prohibitively costly due to their
scale. Parameter-Efficient Fine-Tuning (PEFT) methods enable efficient
adaptation of large pretrained models to various downstream applications by only
fine-tuning a small number of (extra) model parameters instead of all the
model's parameters. This significantly decreases the computational and storage
costs. Recent state-of-the-art PEFT techniques achieve performance comparable to
fully fine-tuned models.

PEFT is integrated with Transformers for easy model training and inference,
Diffusers for conveniently managing different adapters, and Accelerate for
distributed training and inference for really big models.