sareena commited on
Commit
9d71bd3
·
verified ·
1 Parent(s): dcc0684

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -1
README.md CHANGED
@@ -63,7 +63,8 @@ fine-tuning, making it ideal for working with limited GPU resources. LoRA is esp
63
  adaptation when the dataset is moderately sized and instruction formatting is consistent as in the case of this dataset of stepGame.
64
  In previous experiments with spatial reasoning fine-tuning, LoRA performed better than prompt tuning. While prompt tuning resulted in close to 0% accuracy on both the StepGame and
65
  MMLU evaluations, LoRA preserved partial task performance (18% accuracy) and retained some general knowledge ability (46% accuracy on
66
- MMLU geography vs. 52% before training).
 
67
 
68
  # Evaluation
69
 
@@ -71,6 +72,19 @@ MMLU geography vs. 52% before training).
71
 
72
  # Usage and Intended Uses
73
 
 
 
 
 
 
 
 
 
 
 
 
 
 
74
  # Prompt Format
75
 
76
  # Expected Output Format
 
63
  adaptation when the dataset is moderately sized and instruction formatting is consistent as in the case of this dataset of stepGame.
64
  In previous experiments with spatial reasoning fine-tuning, LoRA performed better than prompt tuning. While prompt tuning resulted in close to 0% accuracy on both the StepGame and
65
  MMLU evaluations, LoRA preserved partial task performance (18% accuracy) and retained some general knowledge ability (46% accuracy on
66
+ MMLU geography vs. 52% before training). I used a learning rate of 2e-4, batch size of 8, and trained for 2 epochs.
67
+ This setup preserved general reasoning ability while improving spatial accuracy.
68
 
69
  # Evaluation
70
 
 
72
 
73
  # Usage and Intended Uses
74
 
75
+ ```python
76
+ from transformers import AutoTokenizer, AutoModelForCausalLM
77
+
78
+ model = AutoModelForCausalLM.from_pretrained("sareena/spatial_lora_mistral")
79
+ tokenizer = AutoTokenizer.from_pretrained("sareena/spatial_lora_mistral")
80
+
81
+ inputs = tokenizer("Q: The couch is to the left of the table. The lamp is on the couch. Where is the lamp?", return_tensors="pt")
82
+ outputs = model.generate(**inputs, max_new_tokens=50)
83
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
84
+
85
+ ```
86
+
87
+
88
  # Prompt Format
89
 
90
  # Expected Output Format