|
|
--- |
|
|
license: mit |
|
|
datasets: |
|
|
- EdinburghNLP/xsum |
|
|
metrics: |
|
|
- accuracy |
|
|
base_model: |
|
|
- mistralai/Mistral-7B-Instruct-v0.3 |
|
|
--- |
|
|
|
|
|
Lightweight LoRA adapter trained on the EdinburghNLP/XSum dataset for abstractive news summarization. |
|
|
|
|
|
The adapter fine-tunes Mistral-7B-Instruct-v0.3 using ~50k training and 2k validation samples prepared via a summarization prompt. |
|
|
|
|
|
Training used the following parameters: |
|
|
r=8, |
|
|
lora_alpha=16, |
|
|
lora_dropout=0.1, |
|
|
target_modules=["q_proj", "k_proj", "v_proj", "o_proj"] |
|
|
|
|
|
Adapters can be merged into the base model for inference using PeftModel.merge_and_unload(). |
