codersan/FaLaBSE_Mizan3

SentenceTransformer based on codersan/FaLabse

This is a sentence-transformers model finetuned from codersan/FaLabse. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: codersan/FaLabse
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 768, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
  (3): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("codersan/FaLaBSE_Mizan3")
# Run inference
sentences = [
    'اگر این کار مداومت می\u200cیافت، سنگر قادر به مقاومت نمی\u200cبود.',
    'If this were continued, the barricade was no longer tenable.',
    'Well, for this moment she had a protector.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 1,021,596 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 3 tokens mean: 18.63 tokens max: 81 tokens	min: 3 tokens mean: 16.37 tokens max: 85 tokens

• Samples:

  anchor	positive
  دختران برای اطاعت امر پدر از جا برخاستند.	They arose to obey.
  همه چیز را بم وقع خواهی دانست.	You'll know it all in time
  او هر لحظه گرفتار یک‌ وضع است، زارزار گریه می‌کند. می‌گوید به ما توهین کرده‌اند، حیثیتمان را لکه‌دار نمودند.	She is in hysterics up there, and moans and says that we have been 'shamed and disgraced.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• load_best_model_at_end: True
• push_to_hub: True
• hub_model_id: codersan/FaLaBSE_Mizan3
• eval_on_start: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: True
• resume_from_checkpoint: None
• hub_model_id: codersan/FaLaBSE_Mizan3
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: True
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss
0	0	-
0.0016	100	0.067
0.0031	200	0.0623
0.0047	300	0.0887
0.0063	400	0.0615
0.0078	500	0.0647
0.0094	600	0.0599
0.0110	700	0.0504
0.0125	800	0.0548
0.0141	900	0.0491
0.0157	1000	0.0418
0.0172	1100	0.0484
0.0188	1200	0.0499
0.0204	1300	0.0604
0.0219	1400	0.0492
0.0235	1500	0.0619
0.0251	1600	0.0505
0.0266	1700	0.0552
0.0282	1800	0.0384
0.0298	1900	0.0456
0.0313	2000	0.0459
0.0329	2100	0.0385
0.0345	2200	0.0493
0.0360	2300	0.0483
0.0376	2400	0.0398
0.0392	2500	0.0392
0.0407	2600	0.0652
0.0423	2700	0.0323
0.0439	2800	0.0521
0.0454	2900	0.0424
0.0470	3000	0.0334
0.0486	3100	0.0457
0.0501	3200	0.0349
0.0517	3300	0.032
0.0532	3400	0.0334
0.0548	3500	0.0442
0.0564	3600	0.0414
0.0579	3700	0.0555
0.0595	3800	0.0378
0.0611	3900	0.0583
0.0626	4000	0.0452
0.0642	4100	0.0406
0.0658	4200	0.0461
0.0673	4300	0.0433
0.0689	4400	0.0371
0.0705	4500	0.0392
0.0720	4600	0.0434
0.0736	4700	0.046
0.0752	4800	0.0477
0.0767	4900	0.06
0.0783	5000	0.0415
0.0799	5100	0.0434
0.0814	5200	0.0495
0.0830	5300	0.043
0.0846	5400	0.0515
0.0861	5500	0.0457
0.0877	5600	0.0444
0.0893	5700	0.0454
0.0908	5800	0.0575
0.0924	5900	0.0394
0.0940	6000	0.0554
0.0955	6100	0.0362
0.0971	6200	0.0381
0.0987	6300	0.0446
0.1002	6400	0.0426
0.1018	6500	0.0606
0.1034	6600	0.052
0.1049	6700	0.0414
0.1065	6800	0.059
0.1081	6900	0.0485
0.1096	7000	0.0488
0.1112	7100	0.0312
0.1128	7200	0.0662
0.1143	7300	0.0495
0.1159	7400	0.0529
0.1175	7500	0.0598
0.1190	7600	0.0465
0.1206	7700	0.0577
0.1222	7800	0.0571
0.1237	7900	0.055
0.1253	8000	0.0679
0.1269	8100	0.0525
0.1284	8200	0.0427
0.1300	8300	0.0534
0.1316	8400	0.0593
0.1331	8500	0.0483
0.1347	8600	0.0437
0.1363	8700	0.0505
0.1378	8800	0.0462
0.1394	8900	0.0411
0.1410	9000	0.0665
0.1425	9100	0.0338
0.1441	9200	0.0512
0.1457	9300	0.0683
0.1472	9400	0.077
0.1488	9500	0.0469
0.1504	9600	0.0408
0.1519	9700	0.0374
0.1535	9800	0.0422
0.1551	9900	0.0429
0.1566	10000	0.0466
0.1582	10100	0.0504
0.1597	10200	0.0401
0.1613	10300	0.0384
0.1629	10400	0.0554
0.1644	10500	0.0515
0.1660	10600	0.0529
0.1676	10700	0.0413
0.1691	10800	0.0386
0.1707	10900	0.046
0.1723	11000	0.059
0.1738	11100	0.0716
0.1754	11200	0.0496
0.1770	11300	0.0541
0.1785	11400	0.0499
0.1801	11500	0.0486
0.1817	11600	0.0446
0.1832	11700	0.0508
0.1848	11800	0.0451
0.1864	11900	0.0379
0.1879	12000	0.0515
0.1895	12100	0.0422
0.1911	12200	0.0335
0.1926	12300	0.0408
0.1942	12400	0.0584
0.1958	12500	0.0524
0.1973	12600	0.0562
0.1989	12700	0.0399
0.2005	12800	0.0465
0.2020	12900	0.0334
0.2036	13000	0.0471
0.2052	13100	0.0462
0.2067	13200	0.0419
0.2083	13300	0.0398
0.2099	13400	0.0581
0.2114	13500	0.037
0.2130	13600	0.0328
0.2146	13700	0.0466
0.2161	13800	0.0343
0.2177	13900	0.0491
0.2193	14000	0.0485
0.2208	14100	0.0417
0.2224	14200	0.0603
0.2240	14300	0.0392
0.2255	14400	0.0605
0.2271	14500	0.0369
0.2287	14600	0.0368
0.2302	14700	0.0418
0.2318	14800	0.0471
0.2334	14900	0.0521
0.2349	15000	0.0455
0.2365	15100	0.0371
0.2381	15200	0.044
0.2396	15300	0.0388
0.2412	15400	0.0445
0.2428	15500	0.0608
0.2443	15600	0.0552
0.2459	15700	0.0416
0.2475	15800	0.0395
0.2490	15900	0.0285
0.2506	16000	0.0513
0.2522	16100	0.0524
0.2537	16200	0.0379
0.2553	16300	0.0321
0.2569	16400	0.0405
0.2584	16500	0.0448
0.2600	16600	0.0362
0.2616	16700	0.0516
0.2631	16800	0.068
0.2647	16900	0.057
0.2662	17000	0.046
0.2678	17100	0.0349
0.2694	17200	0.0502
0.2709	17300	0.0508
0.2725	17400	0.0575
0.2741	17500	0.0369
0.2756	17600	0.0385
0.2772	17700	0.0441
0.2788	17800	0.0455
0.2803	17900	0.0404
0.2819	18000	0.057
0.2835	18100	0.0328
0.2850	18200	0.0577
0.2866	18300	0.0355
0.2882	18400	0.0629
0.2897	18500	0.0703
0.2913	18600	0.0464
0.2929	18700	0.0624
0.2944	18800	0.027
0.2960	18900	0.0541
0.2976	19000	0.0361
0.2991	19100	0.0494
0.3007	19200	0.0652
0.3023	19300	0.0574
0.3038	19400	0.0451
0.3054	19500	0.0443
0.3070	19600	0.0571
0.3085	19700	0.0553
0.3101	19800	0.042
0.3117	19900	0.0583
0.3132	20000	0.0394
0.3148	20100	0.0378
0.3164	20200	0.042
0.3179	20300	0.0391
0.3195	20400	0.0452
0.3211	20500	0.0421
0.3226	20600	0.0348
0.3242	20700	0.0332
0.3258	20800	0.0612
0.3273	20900	0.0492
0.3289	21000	0.0319
0.3305	21100	0.0432
0.3320	21200	0.0339
0.3336	21300	0.0441
0.3352	21400	0.0652
0.3367	21500	0.044
0.3383	21600	0.0459
0.3399	21700	0.0571
0.3414	21800	0.0357
0.3430	21900	0.0378
0.3446	22000	0.0364
0.3461	22100	0.0381
0.3477	22200	0.0214
0.3493	22300	0.0424
0.3508	22400	0.054
0.3524	22500	0.0272
0.3540	22600	0.0478
0.3555	22700	0.0438
0.3571	22800	0.0519
0.3587	22900	0.0666
0.3602	23000	0.06
0.3618	23100	0.0462
0.3634	23200	0.0447
0.3649	23300	0.0426
0.3665	23400	0.0518
0.3681	23500	0.0443
0.3696	23600	0.0585
0.3712	23700	0.0356
0.3727	23800	0.0511
0.3743	23900	0.044
0.3759	24000	0.0342
0.3774	24100	0.0272
0.3790	24200	0.0602
0.3806	24300	0.0438
0.3821	24400	0.027
0.3837	24500	0.0485
0.3853	24600	0.0404
0.3868	24700	0.0562
0.3884	24800	0.0334
0.3900	24900	0.0354
0.3915	25000	0.0558
0.3931	25100	0.0366
0.3947	25200	0.0387
0.3962	25300	0.0302
0.3978	25400	0.0554
0.3994	25500	0.0317
0.4009	25600	0.0418
0.4025	25700	0.0534
0.4041	25800	0.0331
0.4056	25900	0.0489
0.4072	26000	0.0428
0.4088	26100	0.0273
0.4103	26200	0.0346
0.4119	26300	0.041
0.4135	26400	0.027
0.4150	26500	0.025
0.4166	26600	0.0522
0.4182	26700	0.0319
0.4197	26800	0.0461
0.4213	26900	0.031
0.4229	27000	0.0318
0.4244	27100	0.0412
0.4260	27200	0.0388
0.4276	27300	0.0336
0.4291	27400	0.0345
0.4307	27500	0.0403
0.4323	27600	0.0567
0.4338	27700	0.0378
0.4354	27800	0.0438
0.4370	27900	0.0419
0.4385	28000	0.0533
0.4401	28100	0.0569
0.4417	28200	0.0458
0.4432	28300	0.0293
0.4448	28400	0.0501
0.4464	28500	0.0616
0.4479	28600	0.0526
0.4495	28700	0.0332
0.4511	28800	0.0367
0.4526	28900	0.0334
0.4542	29000	0.0332
0.4558	29100	0.0296
0.4573	29200	0.0504
0.4589	29300	0.0444
0.4605	29400	0.0404
0.4620	29500	0.0396
0.4636	29600	0.041
0.4652	29700	0.0289
0.4667	29800	0.0354
0.4683	29900	0.0377
0.4699	30000	0.0381
0.4714	30100	0.0456
0.4730	30200	0.0474
0.4745	30300	0.0299
0.4761	30400	0.0433
0.4777	30500	0.0509
0.4792	30600	0.0397
0.4808	30700	0.0449
0.4824	30800	0.0338
0.4839	30900	0.0597
0.4855	31000	0.0369
0.4871	31100	0.0404
0.4886	31200	0.0483
0.4902	31300	0.0501
0.4918	31400	0.0502
0.4933	31500	0.046
0.4949	31600	0.0461
0.4965	31700	0.0365
0.4980	31800	0.0394
0.4996	31900	0.0348
0.5012	32000	0.0309
0.5027	32100	0.0312
0.5043	32200	0.0318
0.5059	32300	0.0349
0.5074	32400	0.0351
0.5090	32500	0.0392
0.5106	32600	0.0478
0.5121	32700	0.0374
0.5137	32800	0.0396
0.5153	32900	0.0346
0.5168	33000	0.0357
0.5184	33100	0.0428
0.5200	33200	0.0348
0.5215	33300	0.048
0.5231	33400	0.0491
0.5247	33500	0.0681
0.5262	33600	0.0348
0.5278	33700	0.034
0.5294	33800	0.0388
0.5309	33900	0.0426
0.5325	34000	0.0355
0.5341	34100	0.0322
0.5356	34200	0.047
0.5372	34300	0.0416
0.5388	34400	0.0267
0.5403	34500	0.0415
0.5419	34600	0.0377
0.5435	34700	0.0426
0.5450	34800	0.0399
0.5466	34900	0.0456
0.5482	35000	0.0458
0.5497	35100	0.0337
0.5513	35200	0.0448
0.5529	35300	0.0437
0.5544	35400	0.0468
0.5560	35500	0.0347
0.5576	35600	0.0406
0.5591	35700	0.037
0.5607	35800	0.0304
0.5623	35900	0.0318
0.5638	36000	0.0405
0.5654	36100	0.0426
0.5670	36200	0.0338
0.5685	36300	0.0254
0.5701	36400	0.0399
0.5717	36500	0.0493
0.5732	36600	0.0435
0.5748	36700	0.0357
0.5764	36800	0.042
0.5779	36900	0.0336
0.5795	37000	0.0492
0.5810	37100	0.0263
0.5826	37200	0.0277
0.5842	37300	0.0414
0.5857	37400	0.0418
0.5873	37500	0.0494
0.5889	37600	0.0355
0.5904	37700	0.0306
0.5920	37800	0.045
0.5936	37900	0.0304
0.5951	38000	0.0305
0.5967	38100	0.0485
0.5983	38200	0.0513
0.5998	38300	0.0362
0.6014	38400	0.0255
0.6030	38500	0.038
0.6045	38600	0.0586
0.6061	38700	0.0221
0.6077	38800	0.0199
0.6092	38900	0.0466
0.6108	39000	0.0251
0.6124	39100	0.0288
0.6139	39200	0.0341
0.6155	39300	0.0335
0.6171	39400	0.037
0.6186	39500	0.0295
0.6202	39600	0.0334
0.6218	39700	0.0518
0.6233	39800	0.0367
0.6249	39900	0.044
0.6265	40000	0.0431
0.6280	40100	0.0496
0.6296	40200	0.0425
0.6312	40300	0.0449
0.6327	40400	0.0337
0.6343	40500	0.0273
0.6359	40600	0.0359
0.6374	40700	0.0283
0.6390	40800	0.0317
0.6406	40900	0.0297
0.6421	41000	0.0302
0.6437	41100	0.0309
0.6453	41200	0.0327
0.6468	41300	0.013
0.6484	41400	0.041
0.6500	41500	0.0254
0.6515	41600	0.0208
0.6531	41700	0.0338
0.6547	41800	0.0371
0.6562	41900	0.0253
0.6578	42000	0.0314
0.6594	42100	0.0452
0.6609	42200	0.0371
0.6625	42300	0.0367
0.6641	42400	0.052
0.6656	42500	0.0267
0.6672	42600	0.0353
0.6688	42700	0.0385
0.6703	42800	0.0361
0.6719	42900	0.0342
0.6735	43000	0.0311
0.6750	43100	0.0587
0.6766	43200	0.0393
0.6782	43300	0.0325
0.6797	43400	0.0356
0.6813	43500	0.0428
0.6829	43600	0.026
0.6844	43700	0.0292
0.6860	43800	0.0499
0.6875	43900	0.0192
0.6891	44000	0.0378
0.6907	44100	0.0323
0.6922	44200	0.0303
0.6938	44300	0.0153
0.6954	44400	0.023
0.6969	44500	0.0261
0.6985	44600	0.0311
0.7001	44700	0.029
0.7016	44800	0.0247
0.7032	44900	0.0353
0.7048	45000	0.041
0.7063	45100	0.0238
0.7079	45200	0.0415
0.7095	45300	0.0394
0.7110	45400	0.0342
0.7126	45500	0.0468
0.7142	45600	0.0409
0.7157	45700	0.0345
0.7173	45800	0.0405
0.7189	45900	0.0454
0.7204	46000	0.0277
0.7220	46100	0.05
0.7236	46200	0.0329
0.7251	46300	0.0281
0.7267	46400	0.0232
0.7283	46500	0.0325
0.7298	46600	0.0332
0.7314	46700	0.0504
0.7330	46800	0.0338
0.7345	46900	0.0355
0.7361	47000	0.0441
0.7377	47100	0.0386
0.7392	47200	0.0467
0.7408	47300	0.0371
0.7424	47400	0.0407
0.7439	47500	0.0313
0.7455	47600	0.0397
0.7471	47700	0.0279
0.7486	47800	0.04
0.7502	47900	0.0318
0.7518	48000	0.0337
0.7533	48100	0.0483
0.7549	48200	0.0356
0.7565	48300	0.0368
0.7580	48400	0.0322
0.7596	48500	0.0228
0.7612	48600	0.0275
0.7627	48700	0.0361
0.7643	48800	0.0257
0.7659	48900	0.0277
0.7674	49000	0.0448
0.7690	49100	0.0369
0.7706	49200	0.0224
0.7721	49300	0.049
0.7737	49400	0.036
0.7753	49500	0.0373
0.7768	49600	0.0225
0.7784	49700	0.0355
0.7800	49800	0.0296
0.7815	49900	0.0228
0.7831	50000	0.0506
0.7847	50100	0.0411
0.7862	50200	0.0289
0.7878	50300	0.0331
0.7894	50400	0.0278
0.7909	50500	0.0246
0.7925	50600	0.0329
0.7940	50700	0.0295
0.7956	50800	0.0551
0.7972	50900	0.0291
0.7987	51000	0.0358
0.8003	51100	0.0387
0.8019	51200	0.0303
0.8034	51300	0.0319
0.8050	51400	0.0282
0.8066	51500	0.0247
0.8081	51600	0.0298
0.8097	51700	0.0336
0.8113	51800	0.0408
0.8128	51900	0.04
0.8144	52000	0.0425
0.8160	52100	0.0326
0.8175	52200	0.0308
0.8191	52300	0.0307
0.8207	52400	0.0226
0.8222	52500	0.0325
0.8238	52600	0.0449
0.8254	52700	0.0437
0.8269	52800	0.031
0.8285	52900	0.0228
0.8301	53000	0.0411
0.8316	53100	0.0226
0.8332	53200	0.029
0.8348	53300	0.033
0.8363	53400	0.0325
0.8379	53500	0.0382
0.8395	53600	0.035
0.8410	53700	0.027
0.8426	53800	0.0221
0.8442	53900	0.0394
0.8457	54000	0.03
0.8473	54100	0.0406
0.8489	54200	0.0257
0.8504	54300	0.0322
0.8520	54400	0.0312
0.8536	54500	0.0338
0.8551	54600	0.0522
0.8567	54700	0.023
0.8583	54800	0.0353
0.8598	54900	0.0381
0.8614	55000	0.0318
0.8630	55100	0.0402
0.8645	55200	0.0314
0.8661	55300	0.0358
0.8677	55400	0.0201
0.8692	55500	0.0348
0.8708	55600	0.0256
0.8724	55700	0.0289
0.8739	55800	0.0541
0.8755	55900	0.0337
0.8771	56000	0.0239
0.8786	56100	0.0255
0.8802	56200	0.0204
0.8818	56300	0.0321
0.8833	56400	0.0241
0.8849	56500	0.022
0.8865	56600	0.0325
0.8880	56700	0.03
0.8896	56800	0.0365
0.8912	56900	0.0341
0.8927	57000	0.028
0.8943	57100	0.0271
0.8958	57200	0.0256
0.8974	57300	0.0305
0.8990	57400	0.0292
0.9005	57500	0.0496
0.9021	57600	0.0236
0.9037	57700	0.0245
0.9052	57800	0.0388
0.9068	57900	0.0296
0.9084	58000	0.046
0.9099	58100	0.0301
0.9115	58200	0.0441
0.9131	58300	0.0244
0.9146	58400	0.0355
0.9162	58500	0.0343
0.9178	58600	0.0377
0.9193	58700	0.0256
0.9209	58800	0.0378
0.9225	58900	0.03
0.9240	59000	0.0296
0.9256	59100	0.0214
0.9272	59200	0.0257
0.9287	59300	0.0366
0.9303	59400	0.0435
0.9319	59500	0.0399
0.9334	59600	0.0302
0.9350	59700	0.033
0.9366	59800	0.0391
0.9381	59900	0.0228
0.9397	60000	0.0354
0.9413	60100	0.0276
0.9428	60200	0.0332
0.9444	60300	0.0219
0.9460	60400	0.0249
0.9475	60500	0.0327
0.9491	60600	0.0296
0.9507	60700	0.0357
0.9522	60800	0.0268
0.9538	60900	0.0371
0.9554	61000	0.0376
0.9569	61100	0.0308
0.9585	61200	0.0342
0.9601	61300	0.0222
0.9616	61400	0.0279
0.9632	61500	0.0351
0.9648	61600	0.0405
0.9663	61700	0.0315
0.9679	61800	0.0354
0.9695	61900	0.0203
0.9710	62000	0.0295
0.9726	62100	0.0338
0.9742	62200	0.0341
0.9757	62300	0.0315
0.9773	62400	0.0221
0.9789	62500	0.0288
0.9804	62600	0.0249
0.9820	62700	0.0332
0.9836	62800	0.0289
0.9851	62900	0.0416
0.9867	63000	0.0363
0.9883	63100	0.0297
0.9898	63200	0.0253
0.9914	63300	0.0268
0.9930	63400	0.0318
0.9945	63500	0.0409
0.9961	63600	0.0248
0.9977	63700	0.0262
0.9992	63800	0.0294

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.3.1
• Transformers: 4.47.0
• PyTorch: 2.5.1+cu121
• Accelerate: 1.2.1
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}