CUAD Question Answering with LoRA Fine-Tuning
Fine-tuned DistilBERT with LoRA on CUAD, updating only 1–2% of parameters for contract clause QA.
In this project, DistilBERT is fine-tuned on the CUAD dataset (41 clause types) using LoRA (Low-Rank Adaptation) so that only 1–2% of parameters are trained.
Highlights:
- Parameter-efficient fine-tuning for legal QA
- Support for multiple clause types in contracts
- Training, evaluation, and reproducibility scripts
This setup is ideal for experimenting with legal-domain QA while keeping compute requirements manageable.