Model save

README.md

@@ -16,12 +16,12 @@ should probably proofread and complete it, then remove this comment. -->
 
 # roberta_en_med_merged_classes
 
-This model is a fine-tuned version of [FacebookAI/roberta-base](https://huggingface.co/FacebookAI/roberta-base) on the None dataset.
+This model is a fine-tuned version of [FacebookAI/roberta-base](https://huggingface.co/FacebookAI/roberta-base) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.4635
-- Accuracy: 0.8405
-- F1 Macro: 0.8001
-- F1 Weighted: 0.8401
+- Loss: 0.4434
+- Accuracy: 0.8467
+- F1 Macro: 0.7876
+- F1 Weighted: 0.8463
 
 ## Model description
 
@@ -41,11 +41,11 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 2e-05
-- train_batch_size: 32
-- eval_batch_size: 64
+- train_batch_size: 64
+- eval_batch_size: 32
 - seed: 42
-- gradient_accumulation_steps: 2
-- total_train_batch_size: 64
+- gradient_accumulation_steps: 8
+- total_train_batch_size: 512
 - optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_ratio: 0.1
@@ -56,23 +56,11 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch  | Step | Validation Loss | Accuracy | F1 Macro | F1 Weighted |
 |:-------------:|:------:|:----:|:---------------:|:--------:|:--------:|:-----------:|
-| 0.7259        | 0.4681 | 400  | 0.6577          | 0.7766   | 0.6330   | 0.7683      |
-| 0.5302        | 0.9362 | 800  | 0.5115          | 0.8099   | 0.7577   | 0.8083      |
-| 0.4494        | 1.4037 | 1200 | 0.4656          | 0.8280   | 0.7818   | 0.8278      |
-| 0.4417        | 1.8719 | 1600 | 0.4583          | 0.8334   | 0.7872   | 0.8326      |
-| 0.3551        | 2.3394 | 2000 | 0.4665          | 0.8239   | 0.7727   | 0.8238      |
-| 0.3486        | 2.8075 | 2400 | 0.4337          | 0.8390   | 0.7933   | 0.8372      |
-| 0.3028        | 3.2750 | 2800 | 0.4387          | 0.8391   | 0.7961   | 0.8392      |
-| 0.2975        | 3.7431 | 3200 | 0.4378          | 0.8407   | 0.7986   | 0.8400      |
-| 0.2576        | 4.2106 | 3600 | 0.4655          | 0.8359   | 0.7975   | 0.8358      |
-| 0.2602        | 4.6788 | 4000 | 0.4580          | 0.8374   | 0.7960   | 0.8377      |
-| 0.2196        | 5.1463 | 4400 | 0.4635          | 0.8405   | 0.8001   | 0.8401      |
-| 0.2153        | 5.6144 | 4800 | 0.4738          | 0.8388   | 0.7961   | 0.8385      |
+| 0.3997        | 3.3621 | 400  | 0.4434          | 0.8467   | 0.7876   | 0.8463      |
 
 
 ### Framework versions
 
 - Transformers 4.56.1
-- Pytorch 2.8.0+cu126
-- Datasets 4.1.0
+- Pytorch 2.6.0+cu124
 - Tokenizers 0.22.0

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:38ea42df563f2db6ae44315147df7062fcd1c686e5dd0d9ce284887b156bb3b8
+oid sha256:644d74679213435046b4e3cfa83c44ba57f229a391da75f2b08e1da95e4d2993
 size 498640508