medgemma-4b-it-sft-lora-diff

This model is a fine-tuned version of unsloth/medgemma-4b-it-unsloth-bnb-4bit on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.1144

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.03
• num_epochs: 7
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.4505	0.1333	2	6.9109
0.413	0.2667	4	5.8505
0.3524	0.4	6	5.0933
0.3067	0.5333	8	4.4616
0.268	0.6667	10	3.9022
0.2333	0.8	12	3.3732
0.2006	0.9333	14	2.8276
0.1669	1.0667	16	2.3019
0.1349	1.2	18	1.8597
0.1078	1.3333	20	1.5445
0.0907	1.4667	22	1.3199
0.0786	1.6	24	1.1433
0.0679	1.7333	26	0.9538
0.0559	1.8667	28	0.8040
0.0488	2.0	30	0.7037
0.0431	2.1333	32	0.6440
0.0394	2.2667	34	0.6086
0.0372	2.4	36	0.5814
0.036	2.5333	38	0.5506
0.0335	2.6667	40	0.5160
0.0315	2.8	42	0.4795
0.0293	2.9333	44	0.4408
0.0267	3.0667	46	0.3978
0.0241	3.2	48	0.3642
0.0226	3.3333	50	0.3685
0.0234	3.4667	52	0.3769
0.0232	3.6	54	0.3629
0.0223	3.7333	56	0.3403
0.0209	3.8667	58	0.3231
0.0199	4.0	60	0.3149
0.0194	4.1333	62	0.3050
0.0187	4.2667	64	0.2897
0.0177	4.4	66	0.2734
0.0167	4.5333	68	0.2596
0.0158	4.6667	70	0.2453
0.015	4.8	72	0.2285
0.0139	4.9333	74	0.2105
0.0127	5.0667	76	0.1926
0.0116	5.2	78	0.1769
0.0107	5.3333	80	0.1634
0.0099	5.4667	82	0.1505
0.0091	5.6	84	0.1393
0.0083	5.7333	86	0.1307
0.008	5.8667	88	0.1248
0.0077	6.0	90	0.1208
0.0074	6.1333	92	0.1186
0.0073	6.2667	94	0.1171
0.0071	6.4	96	0.1160
0.0071	6.5333	98	0.1154
0.007	6.6667	100	0.1147
0.0071	6.8	102	0.1146
0.007	6.9333	104	0.1144

Framework versions

• PEFT 0.16.0
• Transformers 4.57.1
• Pytorch 2.6.0+cu124
• Datasets 4.3.0
• Tokenizers 0.22.1