deepcogito (Deep Cogito)

posted an update about 16 hours ago

Post

578

You can now run Kimi K2 Thinking locally with our Dynamic 1-bit GGUFs: unsloth/Kimi-K2-Thinking-GGUF

We shrank the 1T model to 245GB (-62%) & retained ~85% of accuracy on Aider Polyglot. Run on >247GB RAM for fast inference.

We also collaborated with the Moonshot AI Kimi team on a system prompt fix! 🥰

Guide + fix details: https://docs.unsloth.ai/models/kimi-k2-thinking-how-to-run-locally

danielhanchen

posted an update 3 months ago

Post

6160

Run DeepSeek-V3.1 locally on 170GB RAM with Dynamic 1-bit GGUFs!🐋
GGUFs: unsloth/DeepSeek-V3.1-GGUF

The 715GB model gets reduced to 170GB (-80% size) by smartly quantizing layers.

The 1-bit GGUF passes all our code tests & we fixed the chat template for llama.cpp supported backends.

Guide: https://docs.unsloth.ai/basics/deepseek-v3.1

danielhanchen

posted an update 3 months ago

Post

5281

Run OpenAI's new gpt-oss models locally with Unsloth GGUFs! 🔥🦥
20b GGUF: unsloth/gpt-oss-20b-GGUF
120b GGUF: unsloth/gpt-oss-120b-GGUF

Model will run on 14GB RAM for 20b and 66GB for 120b.

2 replies

·

drishanarora

in deepcogito/cogito-v2-preview-llama-70B 3 months ago

Was this trained on top of Llama 3.1 70B or Llama 3.3 70B?

2

#4 opened 3 months ago by

ddh0

Tool Calling possible?

2

#2 opened 3 months ago by

RobertGro

drishanarora

updated 5 models 3 months ago

baseten-admin

in deepcogito/cogito-v2-preview-llama-109B-MoE 3 months ago

Update tokenizer_config.json

#1 opened 3 months ago by

baseten-admin

in deepcogito/cogito-v2-preview-llama-70B 3 months ago

Update tokenizer_config.json

#1 opened 3 months ago by

baseten-admin

danielhanchen

posted an update 4 months ago

Post

3462

It's Qwen3 week! 💜 We uploaded Dynamic 2-bit GGUFs for:

Qwen3-Coder: unsloth/Qwen3-Coder-480B-A35B-Instruct-GGUF
Qwen3-2507: unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF

So you can run them both locally!
Guides are in model cards.

1 reply

·

danielhanchen

posted an update 4 months ago

Post

3698

Made some 245GB (80% size reduction) 1.8bit quants for Kimi K2!

unsloth/Kimi-K2-Instruct-GGUF

danielhanchen

posted an update 4 months ago

Post

3824

We fixed more issues! Use --jinja for all!
* Fixed Nanonets OCR-s unsloth/Nanonets-OCR-s-GGUF
* Fixed THUDM GLM-4 unsloth/GLM-4-32B-0414-GGUF
* DeepSeek Chimera v2 is uploading! unsloth/DeepSeek-TNG-R1T2-Chimera-GGUF

danielhanchen

posted an update 4 months ago

Post

3062

Gemma 3n finetuning is now 1.5x faster and uses 50% less VRAM in Unsloth!

Click "Use this model" and click "Google Colab"!

unsloth/gemma-3n-E4B-it

unsloth/gemma-3n-E2B-it

https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Gemma3N_(4B)-Conversational.ipynb

2 replies

·

danielhanchen

posted an update 5 months ago

Post

1228

We updated lots of our GGUFs and uploaded many new ones!
* unsloth/dots.llm1.inst-GGUF
* unsloth/Jan-nano-GGUF
* unsloth/Nanonets-OCR-s-GGUF
* Updated and fixed Q8_0 upload for unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF
* Added Q2_K_XL for unsloth/DeepSeek-R1-0528-GGUF
* Updated and fixed Vision support for unsloth/Llama-4-Scout-17B-16E-Instruct-GGUF

danielhanchen

posted an update 5 months ago

Post

2446

Mistral releases Magistral, their new reasoning models! 🔥
GGUFs to run: unsloth/Magistral-Small-2506-GGUF

Magistral-Small-2506 excels at mathematics and coding.

You can run the 24B model locally with just 32GB RAM by using our Dynamic GGUFs.

danielhanchen

posted an update 5 months ago

Post

3787

New DeepSeek-R1-0528 1.65-bit Dynamic GGUF!

Run the model locally even easier! Will fit on a 192GB Macbook and run at 7 tokens/s.

DeepSeek-R1-0528 GGUFs: unsloth/DeepSeek-R1-0528-GGUF
Qwen3-8B DeepSeek-R1-0528 GGUFs: unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF

And read our Guide: https://docs.unsloth.ai/basics/deepseek-r1-0528

danielhanchen

posted an update 6 months ago

Post

2287

💜 Qwen3 128K Context Length: We've released Dynamic 2.0 GGUFs + 4-bit safetensors!
Fixed: Now works on any inference engine and fixed issues with the chat template.
Qwen3 GGUFs:
30B-A3B: unsloth/Qwen3-30B-A3B-GGUF
235-A22B: unsloth/Qwen3-235B-A22B-GGUF
32B: unsloth/Qwen3-32B-GGUF

Read our guide on running Qwen3 here: https://docs.unsloth.ai/basics/qwen3-how-to-run-and-finetune

128K Context Length:
30B-A3B: unsloth/Qwen3-30B-A3B-128K-GGUF
235-A22B: unsloth/Qwen3-235B-A22B-128K-GGUF
32B: unsloth/Qwen3-32B-128K-GGUF

All Qwen3 uploads: unsloth/qwen3-680edabfb790c8c34a242f95

AI & ML interests

Team members 19

deepcogito's activity

Was this trained on top of Llama 3.1 70B or Llama 3.3 70B?

Tool Calling possible?

Update tokenizer_config.json

Update tokenizer_config.json