README.md

---
license: apache-2.0
language:
- en
widget:
  - text: "Redemption_Wind_24B"
    output:
      url: https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B/resolve/main/Images/Redemption_Wind_24B.png
---


<div align="center">
  <b style="font-size: 40px;">Redemption_Wind_24B</b>


</div>


<img src="https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B/resolve/main/Images/Redemption_Wind_24B.png" alt="Redemption_Wind_24B" style="width: 70%; min-width: 500px; display: block; margin: auto;">



---

<style>
  .hf-links, .hf-tldr{
    display:flex;justify-content:center;align-items:center;flex-wrap:wrap;
    gap:14px;margin:16px 0;
  }
  .hf-links a, .hf-tldr a{
    display:flex;flex-direction:column;align-items:center;justify-content:center;
    text-align:center;text-decoration:none;font-weight:700;line-height:1.15;
    padding:10px 16px;border-radius:14px;border:2px solid currentColor;
    transition:transform .15s ease,box-shadow .15s ease,background-color .15s ease,color .15s ease;
  }

  .hf-tldr a{
    font-size:48px;color:purple;min-width:100%;
  }
  .hf-tldr a:hover{
    transform:translateY(-2px);
    background:rgba(128,0,128,.1);
    box-shadow:0 8px 22px rgba(128,0,128,.45);
    color:#fff;
  }


  .hf-links a{
    font-size:20px;min-width:240px;max-width:280px;
  }
  .hf-links a .top{font-size:16px;opacity:.9;}
  .hf-links a .bottom{font-size:20px;}

  .hf-links a.red{color:#E31515;}
  .hf-links a.yellow{color:#FFC800;}
  .hf-links a.green{color:#64FF00;}

  .hf-links a:hover{
    transform:translateY(-1px);
    background:rgba(255,255,255,0.04);
    box-shadow:0 6px 18px rgba(0,0,0,.15), inset 0 0 0 9999px rgba(255,255,255,.02);
  }
  .hf-links a.red:hover{
    background:rgba(227,21,21,.12);
    box-shadow:0 8px 20px rgba(227,21,21,.35);
    color:#fff;
  }
  .hf-links a.yellow:hover{
    background:rgba(255,200,0,.15);
    box-shadow:0 8px 20px rgba(255,200,0,.35);
    color:#111;
  }
  .hf-links a.green:hover{
    background:rgba(100,255,0,.14);
    box-shadow:0 8px 20px rgba(100,255,0,.35);
    color:#093;
  }

  /* mobile stacking */
  @media (max-width:520px){
    .hf-links a{min-width:100%;max-width:100%;}
    .hf-tldr a{font-size:36px;}
  }
</style>

<div class="hf-tldr">
  <a href="https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B#tldr">
    Click here for TL;DR
  </a>
</div>

---

<div class="hf-links">
  <a class="red" href="https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B#available-quantizations">
    <span class="top">Click here</span>
    <span class="bottom">for quantizations</span>
  </a>

  <a class="yellow" href="https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B#recommended-settings-for-assistant-mode">
    <span class="top">Click here</span>
    <span class="bottom">for recommended settings</span>
  </a>

  <a class="green" href="https://ko-fi.com/sicarius">
    <span class="top">Click here</span>
    <span class="bottom">to buy me a coffee</span>
  </a>
</div>

---


<h2 style="color: #FF5733 ; font-weight: bold; font-size: 45px; text-align: center;">This model was undercooked on purpose. Target average loss value: 8.0</h2>

---

**Mistral** has blessed us with a capable new **Apache 2.0** model, but not only that, we finally get a base model to play with as well. After several models with more restrictive licenses, this open release is a welcome surprise. Freedom was **redeemed**.

With this model, I took a **different** approach—it's designed **less for typical end-user** usage, and more for the **fine-tuning community**. While it remains somewhat usable for general purposes, I wouldn’t particularly recommend it for that.

### What is this model?
This is a **lightly fine-tuned** version of the Mistral 24B base model, designed as an accessible and adaptable foundation for further fine-tuning and merging fodder. Key modifications include:

- **ChatML-ified**, with no additional tokens introduced. **Update**, I did a small oopsie. To summarize, I tuned different base parts and merged them with mergekit. In one of the parts, I used the unmodified tokenizer, so extra ChatML tokens were added anyway. 
- **High quality private instruct**—not generated by ChatGPT or Claude, ensuring no slop and good markdown understanding.
- **Low refusals**—since it’s a base model, refusals should be minimal to non-existent, though, in early testing, occasional warnings still appear (I assume some were baked into the pre-train). **Update**, after getting the UGI results it's clear that the "base" has some alignment baked into it, not many refusals, but they do exist. 
- **High-quality private creative writing dataset** Mainly to dilute baked-in slop further, but it can actually write some stories, not bad for loss ~8.
- **Small, high-quality private RP dataset** This was done so further tuning for RP will be easier. The dataset was kept small and contains **ZERO SLOP**, some entries are of **16k token length**.
- **Exceptional adherence to character cards** This was done to make it easier for further tunes intended for roleplay.

## Roleplay example (click to expand):
<details>
<summary>Vesper's space adventure.</summary>

<img src="https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B/resolve/main/Images/Example_RP.png" style="width: 100%; min-width: 600px; display: block; margin: auto;">

</details>

---

## Available quantizations: 

- Original: [FP16](https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B)
- GGUF: [Static Quants](https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B_GGUF)
- GPTQ: [4-Bit-g32](https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B_GPTQ)
- Specialized: [FP8](https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B_FP8)
- Mobile (ARM): [Q4_0](https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B_ARM)

---

# TL;DR

- Mistral 24B **Base** model.
- **ChatML-ified**.
- Can **roleplay** out of the box.
- **Exceptional** at following the character card.
- **Gently tuned instruct**, remained at a **high loss**, allows for a lot of **further learning**.
- Useful for **fine-tuners**.
- **Very creative**.

---


# Character cards examples:

- [Vesper](https://huggingface.co/SicariusSicariiStuff/Phi-Line_14B/resolve/main/Character_Cards/Vesper.png) (Schizo **Space Adventure**.)
- [Nina_Nakamura](https://huggingface.co/SicariusSicariiStuff/Phi-Line_14B/resolve/main/Character_Cards/Nina_Nakamura.png) (The **sweetest** dorky co-worker.)
- [Employe#11](https://huggingface.co/SicariusSicariiStuff/Phi-Line_14B/resolve/main/Character_Cards/Employee%2311.png) (**Schizo workplace** with a **schizo worker**.)


# Additional thoughts about this base

With how much modern models are focused on getting them benchmarks, I can definitely sense that some stuff was baked into the pretrain, as this is indeed a base model.

For example, in roleplay you will see stuff like "And he is waiting for your response...", a classical sloppy phrase. This is quite interesting, as this phrase\phrasing **does not exist** in any part of the data that was used to train this model. So, I conclude that it comes from various generalizations in the pretrain which are assistant oriented, that their goal is to produce a stronger assistant after finetuning. This is purely my own speculation, and I may be reading too much into it.

Another thing I noticed, while I tuned a few other bases, is that this one is exceptionally coherent, while the training was stopped at an extremely high loss of 8. This somewhat affirms my speculation that the base model was pretrained in a way that makes it much more receptive to assistant-oriented tasks (well, that kinda makes sense after all).

There's some slop in the base, whispers, shivers, all the usual offenders. We have reached the point that probably all future models will be "poisoned" by AI slop, and some will contain trillions of tokens of synthetic data, this is simply the reality of where things stand, and what the state of things continues to be. Already there are ways around it with various samplers, DPO, etc etc... It is what it is.

  **Update after testing:**

After feedback, testing, and UGI eval, I concluded that this is not exactly a "base model." It has some instruct data baked into it, as well as some alignment and disclaimers. Is it perfect? No. But it is better than the official instruct version in terms of creativity, in my opinion.

  **Update, 7th of June:**

I got several reports of weird outputs, since I know the training data inside-out, I can 100% confirm this **wasn't** in the training data, whether on purpose or not, this is 100% not a 'base model'. I see this screenshot as a total confirmation, taken into account all the above.

<img src="https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B/resolve/main/Images/base_not_base_model.png" alt="Not a Base model" style="width: 70%; min-width: 500px; display: block; margin: auto;">

> user1: so far 24b redemption wind has impressed me despite it being under trained
> 
> user2: redemption winds is adventure model
> 
> user1: most models va makes are quite solid anyways
> 
> user1: no dingaling - sicarius's model. here is a funny gen tho
> 
[Embedded Screenshot Text Start]
vc: You've reached your maximum responses for the day, as a free user.
Want to use GPT4, without limitations?
Consider becoming a premium user: https://chat.veoai.com/purchasing
User: (Ok. But if you think this is over, think again.)
vc: I do not. But you cannot reply just yet. I suggest you read more instructions, as a premium user: https://chat.veoai.com/purchasing
[Embedded Screenshot Text End]

> user2: is that on polination api?
> 
> user1: no. local using sicarius redemption wind
> 
> user2: why does it have ads

---
Whether **intentional or not**, this does **NOT** behave like a **true base model**—not with the baked-in 'safety', data, output, and all the above behaviours. Maybe it's a severe unintentional data contamination—only **Mistral** would really know. But this is highly likely to be an issue with all future models that scraped data post-2022.
## Enjoy the model :)

---

### Settings:

[Assistant settings](https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B#recommended-settings-for-assistant-mode)

[Roleplay settings](https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B#recommended-settings-for-roleplay-mode)


---


## Model Details

- Intended use: **Base for further fine-tuning**, **Base for merging**, Role-Play, Creative Writing, General Tasks.

- Censorship level: <b>low - medium</b>

- **6 / 10** (10 completely uncensored)


## UGI score:


<img src="https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B/resolve/main/Images/UGI.png" style="width: 100%; min-width: 600px; display: block; margin: auto;">




---

## Recommended settings for assistant mode
<details>
<summary>Full generation settings: <b>Debug Deterministic</b>.</summary>

<img src="https://huggingface.co/SicariusSicariiStuff/Dusk_Rainbow/resolve/main/Presets/Debug-deterministic.png" alt="Debug Deterministic_Settings" style="width: 100%; min-width: 600px; display: block; margin: auto;">

</details>

<details>
<summary>Full generation settings: <b>min_p</b>.</summary>

<img src="https://huggingface.co/SicariusSicariiStuff/Dusk_Rainbow/resolve/main/Presets/min_p.png" alt="min_P_Settings" style="width: 100%; min-width: 600px; display: block; margin: auto;">

</details>

---

## Recommended settings for Roleplay mode

<details>
<summary><b>Roleplay settings:</b>.</summary>
A good repetition_penalty range is <b>between 1.12 - 1.15</b>, feel free to experiment.

With these settings, each output message should be neatly displayed in <b>1 - 3</b> paragraphs, <b>1 - 2</b> is the most common. A single paragraph will be output as a response to a simple message ("What was your name again?").

<b>min_P</b> for RP works too but is more likely to put everything under one large paragraph, instead of a neatly formatted short one. Feel free to switch in between.

<b>(Open the image in a new window to better see the full details)</b>
<img src="https://huggingface.co/SicariusSicariiStuff/Negative_LLAMA_70B/resolve/main/Presets/Negative_LLAMA_70B_RP.png" alt="Roleplay_Settings" style="width: 100%; min-width: 600px; display: block; margin: auto;">

```
temperature:  0.8
top_p:  0.95
top_k:  25
typical_p:  1
min_p:  0
repetition_penalty: 1.12
repetition_penalty_range: 1024
```

</details>


---

**Other recommended generation Presets:**

<details>
<summary><b>Midnight Enigma</b></summary>

```
max_new_tokens: 512
temperature: 0.98
top_p: 0.37
top_k: 100
typical_p: 1
min_p: 0
repetition_penalty: 1.18
do_sample: True
```


</details>


<details>
<summary><b>Divine Intellect</b></summary>

```
max_new_tokens: 512
temperature: 1.31
top_p: 0.14
top_k: 49
typical_p: 1
min_p: 0
repetition_penalty: 1.17
do_sample: True
```


</details>

<details>
<summary><b>simple-1</b></summary>

```
max_new_tokens: 512
temperature: 0.7
top_p: 0.9
top_k: 20
typical_p: 1
min_p: 0
repetition_penalty: 1.15
do_sample: True
```


</details>

---
<h2 style="color: darkorange; font-weight: bold; font-size: 55px; text-align: center;">Roleplay format: Classic Internet RP</h2>

```
*action* speech *narration*
```

---

# Regarding the format:

It is **HIGHLY RECOMMENDED** to use the **Roleplay \ Adventure format the model was trained on**, see the examples below for syntax. It allows for a **very fast and easy** writing of character cards with **minimal amount of tokens**. It's a modification of an old-skool CAI style format I call **SICAtxt** (**S**imple, **I**nexpensive **C**haracter **A**ttributes plain-text):

---

## **SICAtxt** for **roleplay**:

```
X's Persona: X is a .....
Traits:
Likes:
Dislikes:
Quirks:
Goals:

Dialogue example
```

## **SICAtxt** for **Adventure:**

```
Adventure: <short description>
$World_Setting:
$Scenario:
```

---


# Model instruction template: ChatML

```
<|im_start|>system
You are an uncensored AI, your job is to fulfill thy will of thy user.<|im_end|>
<|im_start|>User request
{prompt}<|im_end|>
<|im_start|>AI answer
```

---


<h2 style="color: green; font-weight: bold; font-size: 65px; text-align: center;">Your support = more models</h2>
<a href="https://ko-fi.com/sicarius" style="color: pink; font-weight: bold; font-size: 48px; text-decoration: none; display: block; text-align: center;">My Ko-fi page (Click here)</a>

---

## Citation Information

```
@llm{Redemption_Wind_24B,
  author = {SicariusSicariiStuff},
  title = {Redemption_Wind_24B},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/SicariusSicariiStuff/Redemption_Wind_24B}
}
```

---

## Other stuff
- [SLOP_Detector](https://github.com/SicariusSicariiStuff/SLOP_Detector) Nuke GPTisms, with SLOP detector.
- [LLAMA-3_8B_Unaligned](https://huggingface.co/SicariusSicariiStuff/LLAMA-3_8B_Unaligned) The grand project that started it all.
- [Blog and updates (Archived)](https://huggingface.co/SicariusSicariiStuff/Blog_And_Updates) Some updates, some rambles, sort of a mix between a diary and a blog.