Update 'Recommended Inference Parameters' section
Browse files
README.md
CHANGED
|
@@ -67,10 +67,10 @@ widget:
|
|
| 67 |
<|im_start|>assistant
|
| 68 |
inference:
|
| 69 |
parameters:
|
| 70 |
-
add_special_tokens: true
|
| 71 |
max_new_tokens: 250
|
| 72 |
penalty_alpha: 0.5
|
| 73 |
-
top_k:
|
|
|
|
| 74 |
---
|
| 75 |
|
| 76 |
# A Llama Chat Model of 101M Parameters
|
|
@@ -90,8 +90,6 @@ inference:
|
|
| 90 |
|
| 91 |
## Recommended Prompt Format
|
| 92 |
|
| 93 |
-
The recommended prompt format is as follows:
|
| 94 |
-
|
| 95 |
```
|
| 96 |
<|im_start|>system
|
| 97 |
{system_message}<|im_end|>
|
|
@@ -102,10 +100,8 @@ The recommended prompt format is as follows:
|
|
| 102 |
|
| 103 |
## Recommended Inference Parameters
|
| 104 |
|
| 105 |
-
To get the best results, add special tokens and use [contrastive search](https://huggingface.co/docs/transformers/main/en/generation_strategies#contrastive-search):
|
| 106 |
-
|
| 107 |
```yml
|
| 108 |
-
add_special_tokens: true
|
| 109 |
penalty_alpha: 0.5
|
| 110 |
-
top_k:
|
|
|
|
| 111 |
```
|
|
|
|
| 67 |
<|im_start|>assistant
|
| 68 |
inference:
|
| 69 |
parameters:
|
|
|
|
| 70 |
max_new_tokens: 250
|
| 71 |
penalty_alpha: 0.5
|
| 72 |
+
top_k: 4
|
| 73 |
+
repetition_penalty: 1.105
|
| 74 |
---
|
| 75 |
|
| 76 |
# A Llama Chat Model of 101M Parameters
|
|
|
|
| 90 |
|
| 91 |
## Recommended Prompt Format
|
| 92 |
|
|
|
|
|
|
|
| 93 |
```
|
| 94 |
<|im_start|>system
|
| 95 |
{system_message}<|im_end|>
|
|
|
|
| 100 |
|
| 101 |
## Recommended Inference Parameters
|
| 102 |
|
|
|
|
|
|
|
| 103 |
```yml
|
|
|
|
| 104 |
penalty_alpha: 0.5
|
| 105 |
+
top_k: 4
|
| 106 |
+
repetition_penalty: 1.105
|
| 107 |
```
|