dmitry315/llm-course-hw3-dora

Model Card for Model ID

LLM trained with DoRA for VK NLP course.

Model Details

Model Description

The model is LLM OuteAI/Lite-Oute-1-300M-Instruct.

LLM trained with DoRA for k, v matricies.

Model Sources

• Pretrained Model: OuteAI/Lite-Oute-1-300M-Instruct
• Train Data: cardiffnlp/tweet_eval

Getting start

from safetensors.torch import load_file
from huggingface_hub import hf_hub_download
device = torch.device("cuda")

model = AutoModelForCausalLM.from_pretrained(f"dmitry315/llm-course-hw3-dora", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(f"dmitry315/llm-course-hw3-dora")
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "left"

apply_peft_to_module(model, LinearWithDoRA, r=8, alpha=16, target_submodules=["k_proj", "v_proj"])
model.to(device)

path = hf_hub_download(f"dmitry315/llm-course-hw3-dora", "model.safetensors")
state_dict = load_file(path)

model.load_state_dict(state_dict, strict=False)