coco101010 commited on
Commit
5a74d76
·
verified ·
1 Parent(s): 02d215b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +28 -1
README.md CHANGED
@@ -2,4 +2,31 @@
2
  license: apache-2.0
3
  base_model:
4
  - Qwen/Qwen3-32B
5
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  license: apache-2.0
3
  base_model:
4
  - Qwen/Qwen3-32B
5
+ ---
6
+
7
+ This model is created with the following code:
8
+
9
+ ```Python
10
+ from datasets import load_dataset
11
+ from gptqmodel import GPTQModel, QuantizeConfig
12
+ from huggingface_hub import constants
13
+
14
+ model_id = "Qwen/Qwen3-32B"
15
+ # Save the quantized model in the HF cache directory
16
+ cache_dir = constants.HF_HUB_CACHE
17
+ quant_path = os.path.join(cache_dir, "models--quantized--" + model_id.replace("/", "--"))
18
+ os.makedirs(quant_path, exist_ok=True)
19
+
20
+ # Load calibration data (1024 samples from C4)
21
+ calibration_dataset = load_dataset(
22
+ "allenai/c4",
23
+ data_files="en/c4-train.00001-of-01024.json.gz",
24
+ split="train"
25
+ ).select(range(1024))["text"]
26
+
27
+ # Configure and run quantization
28
+ quant_config = QuantizeConfig(bits=4, group_size=128)
29
+ model = GPTQModel.load(model_id, quant_config)
30
+ model.quantize(calibration_dataset, batch_size=2)
31
+ model.save(quant_path)
32
+ ```