0.5B-value-iteration_1

This model is a fine-tuned version of AngelRaychev/0.5B-value-iteration_0 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3933

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 1024
• eval_batch_size: 1024
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
5.9606	0.8130	100	5.9124
4.1873	1.6260	200	4.2078
2.6983	2.4390	300	2.6579
1.0651	3.2520	400	1.0912
0.5126	4.0650	500	0.5242
0.4485	4.8780	600	0.4762
0.4188	5.6911	700	0.4344
0.4063	6.5041	800	0.4181
0.4025	7.3171	900	0.4204
0.3972	8.1301	1000	0.4092
0.3981	8.9431	1100	0.4093
0.3995	9.7561	1200	0.4161
0.3945	10.5691	1300	0.4101
0.3933	11.3821	1400	0.4063
0.3941	12.1951	1500	0.4039
0.3901	13.0081	1600	0.4029
0.3908	13.8211	1700	0.4024
0.3878	14.6341	1800	0.4007
0.3859	15.4472	1900	0.4011
0.3882	16.2602	2000	0.4004
0.3851	17.0732	2100	0.3990
0.3834	17.8862	2200	0.3991
0.3842	18.6992	2300	0.3979
0.3838	19.5122	2400	0.3971
0.3846	20.3252	2500	0.3971
0.381	21.1382	2600	0.3978
0.3837	21.9512	2700	0.3975
0.3805	22.7642	2800	0.3957
0.3811	23.5772	2900	0.3973
0.3814	24.3902	3000	0.3953
0.3821	25.2033	3100	0.3957
0.3813	26.0163	3200	0.3951
0.3794	26.8293	3300	0.3953
0.3824	27.6423	3400	0.3945
0.3779	28.4553	3500	0.3944
0.3796	29.2683	3600	0.3953
0.3793	30.0813	3700	0.3948
0.3809	30.8943	3800	0.3949
0.3796	31.7073	3900	0.3946
0.3785	32.5203	4000	0.3939
0.3791	33.3333	4100	0.3940
0.3791	34.1463	4200	0.3942
0.3785	34.9593	4300	0.3937
0.3784	35.7724	4400	0.3939
0.3789	36.5854	4500	0.3941
0.3775	37.3984	4600	0.3940
0.3784	38.2114	4700	0.3939
0.3795	39.0244	4800	0.3940
0.3768	39.8374	4900	0.3938
0.3789	40.6504	5000	0.3938
0.378	41.4634	5100	0.3939
0.3794	42.2764	5200	0.3938
0.3792	43.0894	5300	0.3941
0.3786	43.9024	5400	0.3936
0.3785	44.7154	5500	0.3938
0.3793	45.5285	5600	0.3933
0.3782	46.3415	5700	0.3936
0.3789	47.1545	5800	0.3956
0.3765	47.9675	5900	0.3936
0.3781	48.7805	6000	0.3946
0.3805	49.5935	6100	0.3947

Framework versions

• Transformers 4.51.2
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1