Update README.md
Browse files
README.md
CHANGED
|
@@ -18,10 +18,12 @@ base_model: trollek/danube2-1.8b-SlimOrcaDedup
|
|
| 18 |
dtype: bfloat16
|
| 19 |
```
|
| 20 |
|
|
|
|
|
|
|
| 21 |
### LLama-Factory config
|
| 22 |
```yaml
|
| 23 |
### model
|
| 24 |
-
model_name_or_path:
|
| 25 |
|
| 26 |
### method
|
| 27 |
stage: sft
|
|
@@ -68,5 +70,3 @@ per_device_eval_batch_size: 1
|
|
| 68 |
eval_strategy: steps
|
| 69 |
eval_steps: 100
|
| 70 |
```
|
| 71 |
-
|
| 72 |
-
The eval went up after 1 epoch so I stopped and merged that.
|
|
|
|
| 18 |
dtype: bfloat16
|
| 19 |
```
|
| 20 |
|
| 21 |
+
The thought tokens were added to the merged model before training a LoRA.
|
| 22 |
+
|
| 23 |
### LLama-Factory config
|
| 24 |
```yaml
|
| 25 |
### model
|
| 26 |
+
model_name_or_path: merged-base-chatml
|
| 27 |
|
| 28 |
### method
|
| 29 |
stage: sft
|
|
|
|
| 70 |
eval_strategy: steps
|
| 71 |
eval_steps: 100
|
| 72 |
```
|
|
|
|
|
|