Hugging Face's logo

aquif-ai
/
aquif-3.5-Max-42B-A3B

Text Generation
Transformers
Safetensors
qwen3_moe
language
aquif
text-generation-inference
reasoning
math
coding
frontier
aquif-3.5
Mixture of Experts
conversational
Model card Files
xet
Community
8

Seems... Familiar ?

#6
by DavidAU - opened
Discussion
DavidAU
•
edited 3 days ago

42B?
67 Layers?
Odd reasoning does not work.

Interesting design choices.

YOYO-AI
This comment has been hidden
gopi87

gguf is not showing the reasoning not sure how to run it anyone has any idea for fixing ?

YOYO-AI
•
edited 3 days ago

@gopi87 The original source is exactly my merged model: Qwen3-30B-A3B-YOYO-V3. The chat template is identical. Simply change  set thinking = false  to  set thinking = true  in the chat template and it will work.

vico44

The chat template tokenizer begins with: {%- if not thinking is defined %}{% set thinking = false %}

So for example if using llama.cpp you might add this argument ?
--chat-template-kwargs "{"thinking": true}"

Or something like that (i have not test it ^^)

YOYO-AI
•
edited 3 days ago

image

@vico44 I generally modify it directly in the Jinja template of LM Studio.

gopi87

Failed to parse Jinja template: Parser Error: Expected closing statement token. OpenSquareBracket !== CloseStatement.

i am getting this error in lm studio @YOYO-AI can you share me the template ?

gopi87
•
edited 3 days ago

{%- set thinking = true %}
{%- if tools %}
<|im_start|>system
{%- if messages[0].role == 'system' %}
{{ messages[0].content }}

{%- endif %}

Tools

You may call one or more functions to assist with the user query.

You are provided with function signatures within XML tags:

{%- for tool in tools %}
{{ tool | tojson }}
{%- endfor %}

For each function call, return a json object with function name and arguments within XML tags:

{"name": "", "arguments": }
<|im_end|>
{%- else %}
{%- if messages[0].role == 'system' %}
<|im_start|>system
{{ messages[0].content }}<|im_end|>
{%- endif %}
{%- endif %}
{%- for message in messages %}
{%- if message.role == 'user' or (message.role == 'system' and not loop.first) %}
<|im_start|>{{ message.role }}
{{ message.content }}<|im_end|>
{%- elif message.role == 'assistant' %}
<|im_start|>assistant
{%- if message.reasoning_content %}

{{ message.reasoning_content }}

{%- endif %}
{{ message.content }}
{%- if message.tool_calls %}
{%- for tool_call in message.tool_calls %}

{"name": "{{ tool_call.function.name }}", "arguments": {{ tool_call.function.arguments | tojson }}}

{%- endfor %}
{%- endif %}
<|im_end|>
{%- elif message.role == 'tool' %}
<|im_start|>user

{{ message.content }}
<|im_end|>
{%- endif %}
{%- endfor %}
<|im_start|>assistant

aquiffoo
 aquif AI org

I do like the number 67

gopi87

I do like the number 67

can you fix the chat template since the llama.cpp doest recognizing the reasoning

aquiffoo
 aquif AI org

sure, I'll do that after school, in about 7 hours. you can use the unsloth gguf which includes chat template fixes while I'm on it. I'll keep you updated in this thread.

gopi87

sure, I'll do that after school, in about 7 hours. you can use the unsloth gguf which includes chat template fixes while I'm on it. I'll keep you updated in this thread.

school ? really ?

YOYO-AI

{%- set thinking = true %}
{%- if tools %}
<|im_start|>system
{%- if messages[0].role == 'system' %}
{{ messages[0].content }}

{%- endif %}

Tools

You may call one or more functions to assist with the user query.

You are provided with function signatures within XML tags:

{%- for tool in tools %}
{{ tool | tojson }}
{%- endfor %}

For each function call, return a json object with function name and arguments within XML tags:

{"name": "", "arguments": }
<|im_end|>
{%- else %}
{%- if messages[0].role == 'system' %}
<|im_start|>system
{{ messages[0].content }}<|im_end|>
{%- endif %}
{%- endif %}
{%- for message in messages %}
{%- if message.role == 'user' or (message.role == 'system' and not loop.first) %}
<|im_start|>{{ message.role }}
{{ message.content }}<|im_end|>
{%- elif message.role == 'assistant' %}
<|im_start|>assistant
{%- if message.reasoning_content %}

{{ message.reasoning_content }}

{%- endif %}
{{ message.content }}
{%- if message.tool_calls %}
{%- for tool_call in message.tool_calls %}

{"name": "{{ tool_call.function.name }}", "arguments": {{ tool_call.function.arguments | tojson }}}

{%- endfor %}
{%- endif %}
<|im_end|>
{%- elif message.role == 'tool' %}
<|im_start|>user

{{ message.content }}
<|im_end|>
{%- endif %}
{%- endfor %}
<|im_start|>assistant

@gopi87 Your approach is completely correct, which aligns with the original design logic of YOYO-V3. After testing various ways to switch thinking modes, I found that only direct configuration via the chat template yields the most stable results — the thinking switch must be set to "true" to ensure the "think" tag is successfully added.

It’s important to note that if you use the chat template for Qwen3-30B-A3B-Thinking-2507, LM-Studio may automatically remove the "think" tag from the template, allowing the model to generate the tag independently within the main text. However, this model is a hybrid variant developed based on Qwen3-30B-YOYO-V3, and all such hybrid models share a key trait: the "think" tag must be manually added for stable thinking, as the model cannot generate it autonomously in the main text.

YOYO-AI
•
edited 2 days ago

@gopi87 These models cannot generate the "think" tag within the main text, so the tag must and can only be added in the chat template. To enable thinking, use the following template:

{%- if not thinking is defined %}{% set thinking = true %}{% endif %}
{%- if tools %}
    {{- '<|im_start|>system\n' }}
    {%- if messages[0].role == 'system' %}
        {{- messages[0].content + '\n\n' }}
    {%- endif %}
    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
    {%- for tool in tools %}
        {{- "\n" }}
        {{- tool | tojson }}
    {%- endfor %}
    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
    {%- if messages[0].role == 'system' %}
        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
    {%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
    {%- set index = (messages|length - 1) - loop.index0 %}
    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
        {%- set ns.multi_step_tool = false %}
        {%- set ns.last_query_index = index %}
    {%- endif %}
{%- endfor %}
{%- for message in messages %}
    {%- if message.content is string %}
        {%- set content = message.content %}
    {%- else %}
        {%- set content = '' %}
    {%- endif %}
    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
    {%- elif message.role == "assistant" %}
        {%- set reasoning_content = '' %}
        {%- if message.reasoning_content is string %}
            {%- set reasoning_content = message.reasoning_content %}
        {%- else %}
            {%- if '</think>' in content %}
                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
            {%- endif %}
        {%- endif %}
        {%- if loop.index0 > ns.last_query_index %}
            {%- if loop.last or (not loop.last and reasoning_content) %}
                {%- if thinking %}
                    {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
                {%- else %}
                    {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
                {%- endif %}
            {%- else %}
                {{- '<|im_start|>' + message.role + '\n' + content }}
            {%- endif %}
        {%- else %}
            {{- '<|im_start|>' + message.role + '\n' + content }}
        {%- endif %}
        {%- if message.tool_calls %}
            {%- for tool_call in message.tool_calls %}
                {%- if (loop.first and content) or (not loop.first) %}
                    {{- '\n' }}
                {%- endif %}
                {%- if tool_call.function %}
                    {%- set tool_call = tool_call.function %}
                {%- endif %}
                {{- '<tool_call>\n{"name": "' }}
                {{- tool_call.name }}
                {{- '", "arguments": ' }}
                {%- if tool_call.arguments is string %}
                    {{- tool_call.arguments }}
                {%- else %}
                    {{- tool_call.arguments | tojson }}
                {%- endif %}
                {{- '}\n</tool_call>' }}
            {%- endfor %}
        {%- endif %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "tool" %}
        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
            {{- '<|im_start|>user' }}
        {%- endif %}
        {{- '\n<tool_response>\n' }}
        {{- content }}
        {{- '\n</tool_response>' }}
        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
    {%- if thinking %}
        {{- '<|im_start|>assistant\n<think>\n' }}
    {%- else %}
        {{- '<|im_start|>assistant\n' }}
    {%- endif %}
{%- endif %}
gopi87

@gopi87 These models cannot generate the "think" tag within the main text, so the tag must and can only be added in the chat template. To enable thinking, use the following template:

{%- if not thinking is defined %}{% set thinking = ture %}{% endif %}
{%- if tools %}
    {{- '<|im_start|>system\n' }}
    {%- if messages[0].role == 'system' %}
        {{- messages[0].content + '\n\n' }}
    {%- endif %}
    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
    {%- for tool in tools %}
        {{- "\n" }}
        {{- tool | tojson }}
    {%- endfor %}
    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
    {%- if messages[0].role == 'system' %}
        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
    {%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
    {%- set index = (messages|length - 1) - loop.index0 %}
    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
        {%- set ns.multi_step_tool = false %}
        {%- set ns.last_query_index = index %}
    {%- endif %}
{%- endfor %}
{%- for message in messages %}
    {%- if message.content is string %}
        {%- set content = message.content %}
    {%- else %}
        {%- set content = '' %}
    {%- endif %}
    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
    {%- elif message.role == "assistant" %}
        {%- set reasoning_content = '' %}
        {%- if message.reasoning_content is string %}
            {%- set reasoning_content = message.reasoning_content %}
        {%- else %}
            {%- if '</think>' in content %}
                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
            {%- endif %}
        {%- endif %}
        {%- if loop.index0 > ns.last_query_index %}
            {%- if loop.last or (not loop.last and reasoning_content) %}
                {%- if thinking %}
                    {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
                {%- else %}
                    {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
                {%- endif %}
            {%- else %}
                {{- '<|im_start|>' + message.role + '\n' + content }}
            {%- endif %}
        {%- else %}
            {{- '<|im_start|>' + message.role + '\n' + content }}
        {%- endif %}
        {%- if message.tool_calls %}
            {%- for tool_call in message.tool_calls %}
                {%- if (loop.first and content) or (not loop.first) %}
                    {{- '\n' }}
                {%- endif %}
                {%- if tool_call.function %}
                    {%- set tool_call = tool_call.function %}
                {%- endif %}
                {{- '<tool_call>\n{"name": "' }}
                {{- tool_call.name }}
                {{- '", "arguments": ' }}
                {%- if tool_call.arguments is string %}
                    {{- tool_call.arguments }}
                {%- else %}
                    {{- tool_call.arguments | tojson }}
                {%- endif %}
                {{- '}\n</tool_call>' }}
            {%- endfor %}
        {%- endif %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "tool" %}
        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
            {{- '<|im_start|>user' }}
        {%- endif %}
        {{- '\n<tool_response>\n' }}
        {{- content }}
        {{- '\n</tool_response>' }}
        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
    {%- if thinking %}
        {{- '<|im_start|>assistant\n<think>\n' }}
    {%- else %}
        {{- '<|im_start|>assistant\n' }}
    {%- endif %}
{%- endif %}

is this for lm studio or for llama cpp ? i tested in lm studio and its not working did i missing something ?

Elsephire
•
edited 2 days ago

First line

{%- if not thinking is defined %}{% set thinking = ture %}{% endif %}

change "ture " in "true"

YOYO-AI

First line

{%- if not thinking is defined %}{% set thinking = ture %}{% endif %}

change "ture " in "true"

@Elsephire I made a mistake without noticing it. Thanks for correcting me!

gopi87

thanks guys i have been test it any have if you guys have free time feel free to check this too

https://huggingface.co/gopi87/Qwen3-Next-80B-A3B-Instruct-GGUF

aquiffoo
 aquif AI org

I am so sorry for the time I took. I'm now back from school, and I'm happy you've solved your problem regarding the chat template.

aquiffoo changed discussion status to closed

Sign up or log in to comment