Nohobby commited on
Commit
5dcce4c
·
verified ·
1 Parent(s): c081e05

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +107 -13
README.md CHANGED
@@ -1,29 +1,123 @@
1
  ---
2
- base_model:
3
- - unsloth/Mistral-Small-Instruct-2409
 
4
  library_name: transformers
5
  tags:
6
  - mergekit
7
  - merge
8
-
 
 
 
 
 
 
 
 
 
 
9
  ---
10
- # Schisandra0.3
 
 
 
 
 
 
 
 
 
 
11
 
12
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
 
14
  ## Merge Details
15
- ### Merge Method
16
 
17
- This model was merged using the della_linear merge method using SchisandraVA3 as a base.
18
 
19
- ### Models Merged
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
 
21
- The following models were included in the merge:
22
- * [unsloth/Mistral-Small-Instruct-2409](https://huggingface.co/unsloth/Mistral-Small-Instruct-2409)
23
 
24
- ### Configuration
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
 
26
- The following YAML configuration was used to produce this model:
27
 
28
  ```yaml
29
  dtype: bfloat16
@@ -61,4 +155,4 @@ models:
61
  - filter: down_proj
62
  value: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
63
  - value: 1
64
- ```
 
1
  ---
2
+ language:
3
+ - en
4
+ license: other
5
  library_name: transformers
6
  tags:
7
  - mergekit
8
  - merge
9
+ base_model:
10
+ - unsloth/Mistral-Small-Instruct-2409
11
+ - TheDrummer/Cydonia-22B-v1.3
12
+ - Gryphe/Pantheon-RP-Pure-1.6.2-22b-Small
13
+ - anthracite-org/magnum-v4-22b
14
+ - ArliAI/Mistral-Small-22B-ArliAI-RPMax-v1.1
15
+ - spow12/ChatWaifu_v2.0_22B
16
+ - Envoid/Mistral-Small-NovusKyver
17
+ - InferenceIllusionist/SorcererLM-22B
18
+ - ToastyPigeon/MS-Meadowlark-Alt-22B
19
+ - invisietch/MiS-Firefly-v0.2-22B
20
  ---
21
+ ***
22
+ ## Schisandra
23
+
24
+ Many thanks to the authors of the models used!
25
+
26
+ [RPMax v1.1](https://huggingface.co/ArliAI/Mistral-Small-22B-ArliAI-RPMax-v1.1) | [Pantheon-RP](https://huggingface.co/Gryphe/Pantheon-RP-Pure-1.6.2-22b-Small) | [Cydonia-v1.3](https://huggingface.co/TheDrummer/UnslopSmall-22B-v1) | [Magnum V4](https://huggingface.co/anthracite-org/magnum-v4-22b) | [ChatWaifu v2.0](https://huggingface.co/spow12/ChatWaifu_v2.0_22B) | [SorcererLM](https://huggingface.co/InferenceIllusionist/SorcererLM-22B) | [NovusKyver](https://huggingface.co/Envoid/Mistral-Small-NovusKyver) | [Meadowlark](https://huggingface.co/ToastyPigeon/MS-Meadowlark-Alt-22B) | [Firefly](https://huggingface.co/invisietch/MiS-Firefly-v0.2-22B)
27
+ ***
28
+
29
+ ### Overview
30
+
31
+ Main uses: RP
32
 
33
+ Prompt format: Mistral-V3
34
+
35
+ At the moment, I'm not entirely sure it's an improvement on v0.2. It may have lost some of the previous version's tracking, but the writing seems a little more vivid and the swipes are more distinct.
36
+
37
+ ***
38
+
39
+ ### Quants
40
+
41
+ GGUF: [5_K_L](https://huggingface.co/Nohobby/MS-Schisandra-22B-v0.3-Q5_K_L/resolve/main/MS-Schisandra-22B-v0.3_Q5_K_L.gguf?download=true)
42
+
43
+ ***
44
+
45
+ ### Settings
46
+
47
+ My SillyTavern preset: https://huggingface.co/Nohobby/MS-Schisandra-22B-v0.3/resolve/main/ST-formatting-Schisandra0.3.json
48
+
49
+ ***
50
 
51
  ## Merge Details
52
+ ### Merging steps
53
 
54
+ ## Karasik-v0.3
55
 
56
+ ```yaml
57
+ models:
58
+ - model: Mistral-Small-22B-ArliAI-RPMax-v1.1
59
+ parameters:
60
+ weight: [0.2, 0.3, 0.2, 0.3, 0.2]
61
+ density: [0.45, 0.55, 0.45, 0.55, 0.45]
62
+ - model: Mistral-Small-NovusKyver
63
+ parameters:
64
+ weight: [0.01768, -0.01675, 0.01285, -0.01696, 0.01421]
65
+ density: [0.6, 0.4, 0.5, 0.4, 0.6]
66
+ - model: MiS-Firefly-v0.2-22B
67
+ parameters:
68
+ weight: [0.208, 0.139, 0.139, 0.139, 0.208]
69
+ density: [0.7]
70
+ - model: magnum-v4-22b
71
+ parameters:
72
+ weight: [0.33]
73
+ density: [0.45, 0.55, 0.45, 0.55, 0.45]
74
+ merge_method: della_linear
75
+ base_model: Mistral-Small-22B-ArliAI-RPMax-v1.1
76
+ parameters:
77
+ epsilon: 0.05
78
+ lambda: 1.05
79
+ int8_mask: true
80
+ rescale: true
81
+ normalize: false
82
+ dtype: bfloat16
83
+ tokenizer_source: base
84
+ ```
85
 
86
+ ## SchisandraVA3
87
+ (Config taken from [here](https://huggingface.co/HiroseKoichi/Llama-3-8B-Stroganoff-4.0))
88
 
89
+ ```yaml
90
+ merge_method: della_linear
91
+ dtype: bfloat16
92
+ parameters:
93
+ normalize: true
94
+ int8_mask: true
95
+ tokenizer_source: base
96
+ base_model: Cydonia-22B-v1.3
97
+ models:
98
+ - model: Karasik03
99
+ parameters:
100
+ density: 0.55
101
+ weight: 1
102
+ - model: Pantheon-RP-Pure-1.6.2-22b-Small
103
+ parameters:
104
+ density: 0.55
105
+ weight: 1
106
+ - model: ChatWaifu_v2.0_22B
107
+ parameters:
108
+ density: 0.55
109
+ weight: 1
110
+ - model: MS-Meadowlark-Alt-22B
111
+ parameters:
112
+ density: 0.55
113
+ weight: 1
114
+ - model: SorcererLM-22B
115
+ parameters:
116
+ density: 0.55
117
+ weight: 1
118
+ ```
119
 
120
+ ## Schisandra-v0.3
121
 
122
  ```yaml
123
  dtype: bfloat16
 
155
  - filter: down_proj
156
  value: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
157
  - value: 1
158
+ ```