Hugging Face's logo

bigscience-data
/
sgpt-bloom-1b7-nli

Sentence Similarity
sentence-transformers
PyTorch
Safetensors
bloom
feature-extraction
mteb
Eval Results
Model card Files
xet
Community
2
Muennighoff commited on
Commit
63126d4
·
1 Parent(s): 24a4ede

Add MTEB

Browse files
Files changed (1) hide show
  1. README.md +124 -0
README.md CHANGED
@@ -4,6 +4,130 @@ tags:
4
  - sentence-transformers
5
  - feature-extraction
6
  - sentence-similarity
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  ---
8
 
9
  # sgpt-nli-bloom-1b3
 
4
  - sentence-transformers
5
  - feature-extraction
6
  - sentence-similarity
7
+ - mteb
8
+ model-index:
9
+ - name: sgpt-bloom-1b7-nli
10
+ results:
11
+ - task:
12
+ type: Classification
13
+ dataset:
14
+ type: mteb/amazon_reviews_multi
15
+ name: MTEB AmazonReviewsClassification (fr)
16
+ config: fr
17
+ split: test
18
+ revision: c379a6705fec24a2493fa68e011692605f44e119
19
+ metrics:
20
+ - type: accuracy
21
+ value: 39.286
22
+ - type: f1
23
+ value: 38.87078070073539
24
+ - task:
25
+ type: Classification
26
+ dataset:
27
+ type: mteb/amazon_reviews_multi
28
+ name: MTEB AmazonReviewsClassification (zh)
29
+ config: zh
30
+ split: test
31
+ revision: c379a6705fec24a2493fa68e011692605f44e119
32
+ metrics:
33
+ - type: accuracy
34
+ value: 37.634
35
+ - type: f1
36
+ value: 36.86046604093418
37
+ - task:
38
+ type: Classification
39
+ dataset:
40
+ type: mteb/mtop_domain
41
+ name: MTEB MTOPDomainClassification (fr)
42
+ config: fr
43
+ split: test
44
+ revision: a7e2a951126a26fc8c6a69f835f33a346ba259e3
45
+ metrics:
46
+ - type: accuracy
47
+ value: 83.79893517068588
48
+ - type: f1
49
+ value: 83.72326662566203
50
+ - task:
51
+ type: Classification
52
+ dataset:
53
+ type: mteb/mtop_intent
54
+ name: MTEB MTOPIntentClassification (fr)
55
+ config: fr
56
+ split: test
57
+ revision: 6299947a7777084cc2d4b64235bf7190381ce755
58
+ metrics:
59
+ - type: accuracy
60
+ value: 63.36047604134043
61
+ - type: f1
62
+ value: 44.261707019308126
63
+ - task:
64
+ type: Classification
65
+ dataset:
66
+ type: mteb/amazon_massive_intent
67
+ name: MTEB MassiveIntentClassification (fr)
68
+ config: fr
69
+ split: test
70
+ revision: 072a486a144adf7f4479a4a0dddb2152e161e1ea
71
+ metrics:
72
+ - type: accuracy
73
+ value: 64.57632817753867
74
+ - type: f1
75
+ value: 62.60453982786661
76
+ - task:
77
+ type: Classification
78
+ dataset:
79
+ type: mteb/amazon_massive_scenario
80
+ name: MTEB MassiveScenarioClassification (fr)
81
+ config: fr
82
+ split: test
83
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
84
+ metrics:
85
+ - type: accuracy
86
+ value: 69.59986550100874
87
+ - type: f1
88
+ value: 69.71803697939914
89
+ - task:
90
+ type: STS
91
+ dataset:
92
+ type: mteb/sts22-crosslingual-sts
93
+ name: MTEB STS22 (zh)
94
+ config: zh
95
+ split: test
96
+ revision: 2de6ce8c1921b71a755b262c6b57fef195dd7906
97
+ metrics:
98
+ - type: cos_sim_pearson
99
+ value: 59.71781185663265
100
+ - type: cos_sim_spearman
101
+ value: 58.538648447630514
102
+ - type: euclidean_pearson
103
+ value: 53.53848180206165
104
+ - type: euclidean_spearman
105
+ value: 56.33730262964236
106
+ - type: manhattan_pearson
107
+ value: 54.62109820575505
108
+ - type: manhattan_spearman
109
+ value: 57.223846291318914
110
+ - task:
111
+ type: STS
112
+ dataset:
113
+ type: mteb/sts22-crosslingual-sts
114
+ name: MTEB STS22 (fr)
115
+ config: fr
116
+ split: test
117
+ revision: 2de6ce8c1921b71a755b262c6b57fef195dd7906
118
+ metrics:
119
+ - type: cos_sim_pearson
120
+ value: 73.44021434651606
121
+ - type: cos_sim_spearman
122
+ value: 73.13412769502769
123
+ - type: euclidean_pearson
124
+ value: 68.16368597409867
125
+ - type: euclidean_spearman
126
+ value: 72.44964781564485
127
+ - type: manhattan_pearson
128
+ value: 69.42307032478939
129
+ - type: manhattan_spearman
130
+ value: 73.3523195012387
131
  ---
132
 
133
  # sgpt-nli-bloom-1b3