|
|
--- |
|
|
tags: |
|
|
- mlx |
|
|
- apple-silicon |
|
|
- text-generation |
|
|
- gemma3 |
|
|
library_name: mlx-lm |
|
|
pipeline_tag: text-generation |
|
|
base_model: Daizee/Gemma3-Callous-Calla-4B |
|
|
--- |
|
|
|
|
|
# Gemma3-Callous-Calla-4B — **MLX** builds (Apple Silicon) |
|
|
|
|
|
This repo hosts **MLX-converted** variants of **Daizee/Gemma3-Callous-Calla-4B** for fast, local inference on Apple Silicon (M-series). |
|
|
Tokenizer/config are included at the repo root. MLX weight folders live under `mlx/`. |
|
|
|
|
|
> **Note on vocab padding:** For MLX compatibility, the tokenizer/embeddings were padded to the next multiple of 64 tokens. |
|
|
> In this build: **262,208 tokens** (added 64 placeholder tokens named `<pad_ex_*>`). |
|
|
|
|
|
## Variants |
|
|
|
|
|
| Path | Bits | Group Size | Notes | |
|
|
|--------------|------|------------|------------------------------------| |
|
|
| `mlx/g128/` | int4 | 128 | Smallest & fastest | |
|
|
| `mlx/g64/` | int4 | 64 | Slightly larger, often steadier | |
|
|
| `mlx/int8/` | 8 | — | Closest to fp16 quality (slower) | |
|
|
|
|
|
## Quickstart (MLX-LM) |
|
|
|
|
|
### Run from Hugging Face (no cloning needed) |
|
|
```bash |
|
|
python -m mlx_lm.generate \ |
|
|
--model hf://Daizee/Gemma3-Callous-Calla-4B-mlx/mlx/g64 \ |
|
|
--prompt "Summarize the Bill of Rights for 7th graders in 4 bullet points." \ |
|
|
--max-tokens 180 --temp 0.3 --top-p 0.92 |
|
|
|
