Add ZipNN text
Browse files
README.md
CHANGED
|
@@ -46,6 +46,20 @@ zipnn_hf()
|
|
| 46 |
tokenizer = AutoTokenizer.from_pretrained("royleibov/granite-7b-instruct-ZipNN-Compressed")
|
| 47 |
model = AutoModelForCausalLM.from_pretrained("royleibov/granite-7b-instruct-ZipNN-Compressed")
|
| 48 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
|
| 50 |
# Model Card for Granite-7b-lab [Paper](https://arxiv.org/abs/2403.01081)
|
| 51 |
|
|
|
|
| 46 |
tokenizer = AutoTokenizer.from_pretrained("royleibov/granite-7b-instruct-ZipNN-Compressed")
|
| 47 |
model = AutoModelForCausalLM.from_pretrained("royleibov/granite-7b-instruct-ZipNN-Compressed")
|
| 48 |
```
|
| 49 |
+
### ZipNN
|
| 50 |
+
ZipNN also allows you to seemlessly save local disk space in your cache after the model is downloaded.
|
| 51 |
+
|
| 52 |
+
To compress the cached model, simply run:
|
| 53 |
+
```bash
|
| 54 |
+
python zipnn_compress_path.py safetensors --model royleibov/granite-7b-instruct-ZipNN-Compressed --hf_cache
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
The model will be decompressed automatically and safely as long as `zipnn_hf()` is added at the top of the file like in the [example above](#use-this-model).
|
| 58 |
+
|
| 59 |
+
To decompress manualy, simply run:
|
| 60 |
+
```bash
|
| 61 |
+
python zipnn_decompress_path.py --model royleibov/granite-7b-instruct-ZipNN-Compressed --hf_cache
|
| 62 |
+
```
|
| 63 |
|
| 64 |
# Model Card for Granite-7b-lab [Paper](https://arxiv.org/abs/2403.01081)
|
| 65 |
|