LoRA

The process of further training a pretrained AI model on a task-specific or domain-specific dataset to improve its performance, behavior, or specialization for particular use cases.

A library for parameter-efficient fine-tuning of pretrained models by updating a small subset of parameters instead of the full model.

The process of reducing the precision of a model's weights and activations to lower memory footprint and speed up inference.

A model optimization technique in which a smaller AI model, called the student model, is trained to mimic the behavior or outputs of a larger, more capable teacher model, reducing model size and inference cost while preserving much of the original performance.