Add instructions on how to run the model with transformers (#31)
Browse files- Update README.md (f2b0fc04dc4a811841bab00bcc5986d4a83004b2)
Co-authored-by: Younes Belkada <[email protected]>
README.md
CHANGED
|
@@ -104,6 +104,27 @@ num1, num2):
|
|
| 104 |
# return the sum
|
| 105 |
```
|
| 106 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 107 |
## Limitations
|
| 108 |
|
| 109 |
The Codestral-22B-v0.1 does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to
|
|
|
|
| 104 |
# return the sum
|
| 105 |
```
|
| 106 |
|
| 107 |
+
## Usage with transformers library
|
| 108 |
+
|
| 109 |
+
This model is also compatible with `transformers` library, first run `pip install -U transformers` then use the snippet below to quickly get started:
|
| 110 |
+
|
| 111 |
+
```python
|
| 112 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 113 |
+
|
| 114 |
+
model_id = "mistralai/Codestral-22B-v0.1"
|
| 115 |
+
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
| 116 |
+
|
| 117 |
+
model = AutoModelForCausalLM.from_pretrained(model_id)
|
| 118 |
+
|
| 119 |
+
text = "Hello my name is"
|
| 120 |
+
inputs = tokenizer(text, return_tensors="pt")
|
| 121 |
+
|
| 122 |
+
outputs = model.generate(**inputs, max_new_tokens=20)
|
| 123 |
+
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
| 124 |
+
```
|
| 125 |
+
|
| 126 |
+
By default, transformers will load the model in full precision. Therefore you might be interested to further reduce down the memory requirements to run the model through the optimizations we offer in HF ecosystem.
|
| 127 |
+
|
| 128 |
## Limitations
|
| 129 |
|
| 130 |
The Codestral-22B-v0.1 does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to
|