Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
AdinaY 
posted an update Sep 26
Post
2052
Ring-mini-linear-2.0 🔥a hybrid attention MoE model released by Ant group

inclusionAI/Ring-mini-linear-2.0

✨ Hybrid linear + standard attention
✨ 16.4B total, only 1.6B activated
✨ 512k context window via YaRN
✨ Faster than same-size MoE
engrtipusultan

Hopefully you will get some love from llama.cpp and unsloth. That will make it consumer hardware friendly to try.

houxiaowei

amz

In this post