qwp4w3hyb
/

c4ai-command-r-v01-iMat-GGUF

function calling

importance matrix

Model card Files Files and versions

qwp4w3hyb commited on May 11, 2024

Commit

6332cea

·

verified ·

1 Parent(s): c459948

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -31,7 +31,7 @@ license: cc-by-nc-4.0
 - quants done with an importance matrix for improved quantization loss
 - 0, K & IQ quants in basically all variants from Q8 down to IQ1_S
 - Quantized with [llama.cpp](https://github.com/ggerganov/llama.cpp) commit [04976db7a819fcf8bfefbfc09a3344210b79dd27](https://github.com/ggerganov/llama.cpp/commit/04976db7a819fcf8bfefbfc09a3344210b79dd27) (master from 2024-05-07)
-- Imatrtix generated with [this](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384) dataset.
   ```
   ./imatrix -c 512 -m $model_name-f16.gguf -f $llama_cpp_path/groups_merged.txt -o $out_path/imat-f16-gmerged.dat
   ```

 - quants done with an importance matrix for improved quantization loss
 - 0, K & IQ quants in basically all variants from Q8 down to IQ1_S
 - Quantized with [llama.cpp](https://github.com/ggerganov/llama.cpp) commit [04976db7a819fcf8bfefbfc09a3344210b79dd27](https://github.com/ggerganov/llama.cpp/commit/04976db7a819fcf8bfefbfc09a3344210b79dd27) (master from 2024-05-07)
+- Imatrix generated with [this](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384) dataset.
   ```
   ./imatrix -c 512 -m $model_name-f16.gguf -f $llama_cpp_path/groups_merged.txt -o $out_path/imat-f16-gmerged.dat
   ```