Hugging Face's logo

sky-2002
/
SmolLM2-360M-GRPO-v1

Transformers
Safetensors
Generated from Trainer
trl
grpo
Model card Files
xet
Community
sky-2002 commited on
Commit
1d71deb
·
verified ·
1 Parent(s): 7f9d9e4

Upload folder using huggingface_hub

Browse files
Files changed (40) hide show
  1. README.md +10 -1
  2. checkpoint-100/README.md +202 -0
  3. checkpoint-100/adapter_config.json +37 -0
  4. checkpoint-100/adapter_model.safetensors +3 -0
  5. checkpoint-100/merges.txt +0 -0
  6. checkpoint-100/optimizer.pt +3 -0
  7. checkpoint-100/rng_state.pth +3 -0
  8. checkpoint-100/scheduler.pt +3 -0
  9. checkpoint-100/special_tokens_map.json +28 -0
  10. checkpoint-100/tokenizer.json +0 -0
  11. checkpoint-100/tokenizer_config.json +156 -0
  12. checkpoint-100/trainer_state.json +1233 -0
  13. checkpoint-100/training_args.bin +3 -0
  14. checkpoint-100/vocab.json +0 -0
  15. checkpoint-200/README.md +202 -0
  16. checkpoint-200/adapter_config.json +37 -0
  17. checkpoint-200/adapter_model.safetensors +3 -0
  18. checkpoint-200/merges.txt +0 -0
  19. checkpoint-200/optimizer.pt +3 -0
  20. checkpoint-200/rng_state.pth +3 -0
  21. checkpoint-200/scheduler.pt +3 -0
  22. checkpoint-200/special_tokens_map.json +28 -0
  23. checkpoint-200/tokenizer.json +0 -0
  24. checkpoint-200/tokenizer_config.json +156 -0
  25. checkpoint-200/trainer_state.json +2433 -0
  26. checkpoint-200/training_args.bin +3 -0
  27. checkpoint-200/vocab.json +0 -0
  28. checkpoint-294/README.md +202 -0
  29. checkpoint-294/adapter_config.json +37 -0
  30. checkpoint-294/adapter_model.safetensors +3 -0
  31. checkpoint-294/merges.txt +0 -0
  32. checkpoint-294/optimizer.pt +3 -0
  33. checkpoint-294/rng_state.pth +3 -0
  34. checkpoint-294/scheduler.pt +3 -0
  35. checkpoint-294/special_tokens_map.json +28 -0
  36. checkpoint-294/tokenizer.json +0 -0
  37. checkpoint-294/tokenizer_config.json +156 -0
  38. checkpoint-294/trainer_state.json +0 -0
  39. checkpoint-294/training_args.bin +3 -0
  40. checkpoint-294/vocab.json +0 -0
README.md CHANGED
@@ -12,7 +12,16 @@ licence: license
12
  # Model Card for SmolLM2-360M-GRPO-v1
13
 
14
  This model is a fine-tuned version of [HuggingFaceTB/SmolLM2-360M](https://huggingface.co/HuggingFaceTB/SmolLM2-360M).
15
- It has been trained using [TRL](https://github.com/huggingface/trl).
 
 
 
 
 
 
 
 
 
16
 
17
  ## Quick start
18
 
 
12
  # Model Card for SmolLM2-360M-GRPO-v1
13
 
14
  This model is a fine-tuned version of [HuggingFaceTB/SmolLM2-360M](https://huggingface.co/HuggingFaceTB/SmolLM2-360M).
15
+ It has been trained using [TRL](https://github.com/huggingface/trl) and using the `lamini/taylor_swift` dataset.
16
+
17
+ ## Evals
18
+ Referring this [blog post](https://datawizz.ai/blog/grpo-fine-tuning-qwen-0-5b-vs-openai-o1-preview), used a similar evaluation method:
19
+
20
+ | Model | Average ROUGE-L |
21
+ |-------|-----------------|
22
+ | Qwen-0.5B | 0.3313 |
23
+ | SmolLM2-360M-GRPO-v0 | 0.1644 |
24
+ | SmolLM2-360M-GRPO-v1 | 0.1672 |
25
 
26
  ## Quick start
27
 
checkpoint-100/README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: HuggingFaceTB/SmolLM2-360M
3
+ library_name: peft
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.14.0
checkpoint-100/adapter_config.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "HuggingFaceTB/SmolLM2-360M",
5
+ "bias": "none",
6
+ "eva_config": null,
7
+ "exclude_modules": null,
8
+ "fan_in_fan_out": false,
9
+ "inference_mode": true,
10
+ "init_lora_weights": true,
11
+ "layer_replication": null,
12
+ "layers_pattern": null,
13
+ "layers_to_transform": null,
14
+ "loftq_config": {},
15
+ "lora_alpha": 64,
16
+ "lora_bias": false,
17
+ "lora_dropout": 0.05,
18
+ "megatron_config": null,
19
+ "megatron_core": "megatron.core",
20
+ "modules_to_save": null,
21
+ "peft_type": "LORA",
22
+ "r": 32,
23
+ "rank_pattern": {},
24
+ "revision": null,
25
+ "target_modules": [
26
+ "k_proj",
27
+ "up_proj",
28
+ "v_proj",
29
+ "gate_proj",
30
+ "q_proj",
31
+ "down_proj",
32
+ "o_proj"
33
+ ],
34
+ "task_type": "CAUSAL_LM",
35
+ "use_dora": false,
36
+ "use_rslora": false
37
+ }
checkpoint-100/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ab6ba06359a4e33a6bb9d0489ac289f115e229adca4ecd0ce56d3be06610b366
3
+ size 69527352
checkpoint-100/merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-100/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cab295c214b888cda14e25fe86824c274e9a7c8e3dd675a86b7d824667b53c5d
3
+ size 139313234
checkpoint-100/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:281fc5b575006c17429df6952a63f0dd26d4c1f124d999e9e3d9113e6e8c1045
3
+ size 14244
checkpoint-100/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:691263e55c7bb34b6e090bd89cf079f362100d7bd758be475bd3e7091c1a4ed6
3
+ size 1064
checkpoint-100/special_tokens_map.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ {
4
+ "content": "<|im_start|>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ {
11
+ "content": "<|im_end|>",
12
+ "lstrip": false,
13
+ "normalized": false,
14
+ "rstrip": false,
15
+ "single_word": false
16
+ }
17
+ ],
18
+ "bos_token": "<|im_start|>",
19
+ "eos_token": "<|im_end|>",
20
+ "pad_token": "<|im_end|>",
21
+ "unk_token": {
22
+ "content": "<|endoftext|>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": false,
26
+ "single_word": false
27
+ }
28
+ }
checkpoint-100/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-100/tokenizer_config.json ADDED
@@ -0,0 +1,156 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "0": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": false,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "1": {
13
+ "content": "<|im_start|>",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "2": {
21
+ "content": "<|im_end|>",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "3": {
29
+ "content": "<repo_name>",
30
+ "lstrip": false,
31
+ "normalized": false,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "4": {
37
+ "content": "<reponame>",
38
+ "lstrip": false,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ },
44
+ "5": {
45
+ "content": "<file_sep>",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false,
50
+ "special": true
51
+ },
52
+ "6": {
53
+ "content": "<filename>",
54
+ "lstrip": false,
55
+ "normalized": false,
56
+ "rstrip": false,
57
+ "single_word": false,
58
+ "special": true
59
+ },
60
+ "7": {
61
+ "content": "<gh_stars>",
62
+ "lstrip": false,
63
+ "normalized": false,
64
+ "rstrip": false,
65
+ "single_word": false,
66
+ "special": true
67
+ },
68
+ "8": {
69
+ "content": "<issue_start>",
70
+ "lstrip": false,
71
+ "normalized": false,
72
+ "rstrip": false,
73
+ "single_word": false,
74
+ "special": true
75
+ },
76
+ "9": {
77
+ "content": "<issue_comment>",
78
+ "lstrip": false,
79
+ "normalized": false,
80
+ "rstrip": false,
81
+ "single_word": false,
82
+ "special": true
83
+ },
84
+ "10": {
85
+ "content": "<issue_closed>",
86
+ "lstrip": false,
87
+ "normalized": false,
88
+ "rstrip": false,
89
+ "single_word": false,
90
+ "special": true
91
+ },
92
+ "11": {
93
+ "content": "<jupyter_start>",
94
+ "lstrip": false,
95
+ "normalized": false,
96
+ "rstrip": false,
97
+ "single_word": false,
98
+ "special": true
99
+ },
100
+ "12": {
101
+ "content": "<jupyter_text>",
102
+ "lstrip": false,
103
+ "normalized": false,
104
+ "rstrip": false,
105
+ "single_word": false,
106
+ "special": true
107
+ },
108
+ "13": {
109
+ "content": "<jupyter_code>",
110
+ "lstrip": false,
111
+ "normalized": false,
112
+ "rstrip": false,
113
+ "single_word": false,
114
+ "special": true
115
+ },
116
+ "14": {
117
+ "content": "<jupyter_output>",
118
+ "lstrip": false,
119
+ "normalized": false,
120
+ "rstrip": false,
121
+ "single_word": false,
122
+ "special": true
123
+ },
124
+ "15": {
125
+ "content": "<jupyter_script>",
126
+ "lstrip": false,
127
+ "normalized": false,
128
+ "rstrip": false,
129
+ "single_word": false,
130
+ "special": true
131
+ },
132
+ "16": {
133
+ "content": "<empty_output>",
134
+ "lstrip": false,
135
+ "normalized": false,
136
+ "rstrip": false,
137
+ "single_word": false,
138
+ "special": true
139
+ }
140
+ },
141
+ "additional_special_tokens": [
142
+ "<|im_start|>",
143
+ "<|im_end|>"
144
+ ],
145
+ "bos_token": "<|im_start|>",
146
+ "chat_template": "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
147
+ "clean_up_tokenization_spaces": false,
148
+ "eos_token": "<|im_end|>",
149
+ "extra_special_tokens": {},
150
+ "model_max_length": 8192,
151
+ "pad_token": "<|im_end|>",
152
+ "padding_side": "left",
153
+ "tokenizer_class": "GPT2Tokenizer",
154
+ "unk_token": "<|endoftext|>",
155
+ "vocab_size": 49152
156
+ }
checkpoint-100/trainer_state.json ADDED
@@ -0,0 +1,1233 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 1.0204081632653061,
5
+ "eval_steps": 500,
6
+ "global_step": 100,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "completion_length": 160.0,
13
+ "epoch": 0.01020408163265306,
14
+ "grad_norm": 0.1883362978696823,
15
+ "kl": 0.0,
16
+ "learning_rate": 1.0000000000000002e-06,
17
+ "loss": 0.0,
18
+ "reward": 0.35154566168785095,
19
+ "reward_std": 0.18836843967437744,
20
+ "rewards/<lambda>": 0.35154566168785095,
21
+ "step": 1
22
+ },
23
+ {
24
+ "completion_length": 160.0,
25
+ "epoch": 0.02040816326530612,
26
+ "grad_norm": 0.08015607297420502,
27
+ "kl": 0.0,
28
+ "learning_rate": 2.0000000000000003e-06,
29
+ "loss": 0.0,
30
+ "reward": 0.24049028754234314,
31
+ "reward_std": 0.23294341564178467,
32
+ "rewards/<lambda>": 0.24049028754234314,
33
+ "step": 2
34
+ },
35
+ {
36
+ "completion_length": 160.0,
37
+ "epoch": 0.030612244897959183,
38
+ "grad_norm": 0.09276824444532394,
39
+ "kl": 0.0009410024504177272,
40
+ "learning_rate": 3e-06,
41
+ "loss": 0.0,
42
+ "reward": 0.32676321268081665,
43
+ "reward_std": 0.25492924451828003,
44
+ "rewards/<lambda>": 0.32676321268081665,
45
+ "step": 3
46
+ },
47
+ {
48
+ "completion_length": 160.0,
49
+ "epoch": 0.04081632653061224,
50
+ "grad_norm": 0.10654664784669876,
51
+ "kl": 0.001077650347724557,
52
+ "learning_rate": 4.000000000000001e-06,
53
+ "loss": 0.0,
54
+ "reward": 0.2009219378232956,
55
+ "reward_std": 0.21452507376670837,
56
+ "rewards/<lambda>": 0.2009219378232956,
57
+ "step": 4
58
+ },
59
+ {
60
+ "completion_length": 160.0,
61
+ "epoch": 0.05102040816326531,
62
+ "grad_norm": 0.11217533051967621,
63
+ "kl": 0.001070915488526225,
64
+ "learning_rate": 5e-06,
65
+ "loss": 0.0,
66
+ "reward": 0.3001704216003418,
67
+ "reward_std": 0.24798467755317688,
68
+ "rewards/<lambda>": 0.3001704216003418,
69
+ "step": 5
70
+ },
71
+ {
72
+ "completion_length": 160.0,
73
+ "epoch": 0.061224489795918366,
74
+ "grad_norm": 0.09305333346128464,
75
+ "kl": 0.0010154233314096928,
76
+ "learning_rate": 6e-06,
77
+ "loss": 0.0,
78
+ "reward": 0.2345525473356247,
79
+ "reward_std": 0.22145482897758484,
80
+ "rewards/<lambda>": 0.2345525473356247,
81
+ "step": 6
82
+ },
83
+ {
84
+ "completion_length": 160.0,
85
+ "epoch": 0.07142857142857142,
86
+ "grad_norm": 0.13233615458011627,
87
+ "kl": 0.0011612953385338187,
88
+ "learning_rate": 7.000000000000001e-06,
89
+ "loss": 0.0,
90
+ "reward": 0.3054887056350708,
91
+ "reward_std": 0.21511563658714294,
92
+ "rewards/<lambda>": 0.3054887056350708,
93
+ "step": 7
94
+ },
95
+ {
96
+ "completion_length": 160.0,
97
+ "epoch": 0.08163265306122448,
98
+ "grad_norm": 0.1244489848613739,
99
+ "kl": 0.000984564539976418,
100
+ "learning_rate": 8.000000000000001e-06,
101
+ "loss": 0.0,
102
+ "reward": 0.27341318130493164,
103
+ "reward_std": 0.23464292287826538,
104
+ "rewards/<lambda>": 0.27341318130493164,
105
+ "step": 8
106
+ },
107
+ {
108
+ "completion_length": 160.0,
109
+ "epoch": 0.09183673469387756,
110
+ "grad_norm": 0.09130726009607315,
111
+ "kl": 0.0008879535598680377,
112
+ "learning_rate": 9e-06,
113
+ "loss": 0.0,
114
+ "reward": 0.3593016564846039,
115
+ "reward_std": 0.2495943158864975,
116
+ "rewards/<lambda>": 0.3593016564846039,
117
+ "step": 9
118
+ },
119
+ {
120
+ "completion_length": 160.0,
121
+ "epoch": 0.10204081632653061,
122
+ "grad_norm": 0.15256966650485992,
123
+ "kl": 0.0011160458670929074,
124
+ "learning_rate": 1e-05,
125
+ "loss": 0.0,
126
+ "reward": 0.3049771785736084,
127
+ "reward_std": 0.26634567975997925,
128
+ "rewards/<lambda>": 0.3049771785736084,
129
+ "step": 10
130
+ },
131
+ {
132
+ "completion_length": 160.0,
133
+ "epoch": 0.11224489795918367,
134
+ "grad_norm": 0.09210026264190674,
135
+ "kl": 0.0010061098728328943,
136
+ "learning_rate": 1.1000000000000001e-05,
137
+ "loss": 0.0,
138
+ "reward": 0.2637690603733063,
139
+ "reward_std": 0.21869879961013794,
140
+ "rewards/<lambda>": 0.2637690603733063,
141
+ "step": 11
142
+ },
143
+ {
144
+ "completion_length": 160.0,
145
+ "epoch": 0.12244897959183673,
146
+ "grad_norm": 0.09713605791330338,
147
+ "kl": 0.0010679092956706882,
148
+ "learning_rate": 1.2e-05,
149
+ "loss": 0.0,
150
+ "reward": 0.36334261298179626,
151
+ "reward_std": 0.24027395248413086,
152
+ "rewards/<lambda>": 0.36334261298179626,
153
+ "step": 12
154
+ },
155
+ {
156
+ "completion_length": 160.0,
157
+ "epoch": 0.1326530612244898,
158
+ "grad_norm": 0.09611018002033234,
159
+ "kl": 0.000932489987462759,
160
+ "learning_rate": 1.3000000000000001e-05,
161
+ "loss": 0.0,
162
+ "reward": 0.2559860944747925,
163
+ "reward_std": 0.25332069396972656,
164
+ "rewards/<lambda>": 0.2559860944747925,
165
+ "step": 13
166
+ },
167
+ {
168
+ "completion_length": 160.0,
169
+ "epoch": 0.14285714285714285,
170
+ "grad_norm": 0.11913401633501053,
171
+ "kl": 0.0010431472910568118,
172
+ "learning_rate": 1.4000000000000001e-05,
173
+ "loss": 0.0,
174
+ "reward": 0.2593250274658203,
175
+ "reward_std": 0.25670933723449707,
176
+ "rewards/<lambda>": 0.2593250274658203,
177
+ "step": 14
178
+ },
179
+ {
180
+ "completion_length": 160.0,
181
+ "epoch": 0.15306122448979592,
182
+ "grad_norm": 0.09773930162191391,
183
+ "kl": 0.0009058397263288498,
184
+ "learning_rate": 1.5e-05,
185
+ "loss": 0.0,
186
+ "reward": 0.3413263261318207,
187
+ "reward_std": 0.23378777503967285,
188
+ "rewards/<lambda>": 0.3413263261318207,
189
+ "step": 15
190
+ },
191
+ {
192
+ "completion_length": 160.0,
193
+ "epoch": 0.16326530612244897,
194
+ "grad_norm": 0.10632280260324478,
195
+ "kl": 0.000995107227936387,
196
+ "learning_rate": 1.6000000000000003e-05,
197
+ "loss": 0.0,
198
+ "reward": 0.3327138423919678,
199
+ "reward_std": 0.2463734745979309,
200
+ "rewards/<lambda>": 0.3327138423919678,
201
+ "step": 16
202
+ },
203
+ {
204
+ "completion_length": 160.0,
205
+ "epoch": 0.17346938775510204,
206
+ "grad_norm": 0.0903797596693039,
207
+ "kl": 0.0008249408565461636,
208
+ "learning_rate": 1.7000000000000003e-05,
209
+ "loss": 0.0,
210
+ "reward": 0.25679031014442444,
211
+ "reward_std": 0.2034485787153244,
212
+ "rewards/<lambda>": 0.25679031014442444,
213
+ "step": 17
214
+ },
215
+ {
216
+ "completion_length": 160.0,
217
+ "epoch": 0.1836734693877551,
218
+ "grad_norm": 0.09401573985815048,
219
+ "kl": 0.0011675208806991577,
220
+ "learning_rate": 1.8e-05,
221
+ "loss": 0.0,
222
+ "reward": 0.24813005328178406,
223
+ "reward_std": 0.21580623090267181,
224
+ "rewards/<lambda>": 0.24813005328178406,
225
+ "step": 18
226
+ },
227
+ {
228
+ "completion_length": 160.0,
229
+ "epoch": 0.19387755102040816,
230
+ "grad_norm": 0.12456843256950378,
231
+ "kl": 0.0010836416622623801,
232
+ "learning_rate": 1.9e-05,
233
+ "loss": 0.0,
234
+ "reward": 0.3487837314605713,
235
+ "reward_std": 0.220088928937912,
236
+ "rewards/<lambda>": 0.3487837314605713,
237
+ "step": 19
238
+ },
239
+ {
240
+ "completion_length": 160.0,
241
+ "epoch": 0.20408163265306123,
242
+ "grad_norm": 0.0994577631354332,
243
+ "kl": 0.0012541261967271566,
244
+ "learning_rate": 2e-05,
245
+ "loss": 0.0001,
246
+ "reward": 0.2757805585861206,
247
+ "reward_std": 0.23097757995128632,
248
+ "rewards/<lambda>": 0.2757805585861206,
249
+ "step": 20
250
+ },
251
+ {
252
+ "completion_length": 160.0,
253
+ "epoch": 0.21428571428571427,
254
+ "grad_norm": 0.1026805192232132,
255
+ "kl": 0.0013662949204444885,
256
+ "learning_rate": 2.1e-05,
257
+ "loss": 0.0001,
258
+ "reward": 0.3016376793384552,
259
+ "reward_std": 0.26067519187927246,
260
+ "rewards/<lambda>": 0.3016376793384552,
261
+ "step": 21
262
+ },
263
+ {
264
+ "completion_length": 160.0,
265
+ "epoch": 0.22448979591836735,
266
+ "grad_norm": 0.11243847012519836,
267
+ "kl": 0.0012734383344650269,
268
+ "learning_rate": 2.2000000000000003e-05,
269
+ "loss": 0.0001,
270
+ "reward": 0.34152576327323914,
271
+ "reward_std": 0.24704596400260925,
272
+ "rewards/<lambda>": 0.34152576327323914,
273
+ "step": 22
274
+ },
275
+ {
276
+ "completion_length": 160.0,
277
+ "epoch": 0.23469387755102042,
278
+ "grad_norm": 0.11959439516067505,
279
+ "kl": 0.001076711225323379,
280
+ "learning_rate": 2.3000000000000003e-05,
281
+ "loss": 0.0,
282
+ "reward": 0.25360214710235596,
283
+ "reward_std": 0.24721531569957733,
284
+ "rewards/<lambda>": 0.25360214710235596,
285
+ "step": 23
286
+ },
287
+ {
288
+ "completion_length": 160.0,
289
+ "epoch": 0.24489795918367346,
290
+ "grad_norm": 0.12651875615119934,
291
+ "kl": 0.001166851376183331,
292
+ "learning_rate": 2.4e-05,
293
+ "loss": 0.0,
294
+ "reward": 0.23422789573669434,
295
+ "reward_std": 0.22290775179862976,
296
+ "rewards/<lambda>": 0.23422789573669434,
297
+ "step": 24
298
+ },
299
+ {
300
+ "completion_length": 160.0,
301
+ "epoch": 0.25510204081632654,
302
+ "grad_norm": 0.09411771595478058,
303
+ "kl": 0.0009251383016817272,
304
+ "learning_rate": 2.5e-05,
305
+ "loss": 0.0,
306
+ "reward": 0.3483571410179138,
307
+ "reward_std": 0.22439202666282654,
308
+ "rewards/<lambda>": 0.3483571410179138,
309
+ "step": 25
310
+ },
311
+ {
312
+ "completion_length": 160.0,
313
+ "epoch": 0.2653061224489796,
314
+ "grad_norm": 0.10857303440570831,
315
+ "kl": 0.0013178512454032898,
316
+ "learning_rate": 2.6000000000000002e-05,
317
+ "loss": 0.0001,
318
+ "reward": 0.27066630125045776,
319
+ "reward_std": 0.25778770446777344,
320
+ "rewards/<lambda>": 0.27066630125045776,
321
+ "step": 26
322
+ },
323
+ {
324
+ "completion_length": 160.0,
325
+ "epoch": 0.2755102040816326,
326
+ "grad_norm": 0.10724504292011261,
327
+ "kl": 0.0011154217645525932,
328
+ "learning_rate": 2.7000000000000002e-05,
329
+ "loss": 0.0,
330
+ "reward": 0.29464977979660034,
331
+ "reward_std": 0.22159242630004883,
332
+ "rewards/<lambda>": 0.29464977979660034,
333
+ "step": 27
334
+ },
335
+ {
336
+ "completion_length": 160.0,
337
+ "epoch": 0.2857142857142857,
338
+ "grad_norm": 0.14677317440509796,
339
+ "kl": 0.0010433748830109835,
340
+ "learning_rate": 2.8000000000000003e-05,
341
+ "loss": 0.0,
342
+ "reward": 0.3109304904937744,
343
+ "reward_std": 0.26592540740966797,
344
+ "rewards/<lambda>": 0.3109304904937744,
345
+ "step": 28
346
+ },
347
+ {
348
+ "completion_length": 160.0,
349
+ "epoch": 0.29591836734693877,
350
+ "grad_norm": 0.12714359164237976,
351
+ "kl": 0.001189422095194459,
352
+ "learning_rate": 2.9e-05,
353
+ "loss": 0.0,
354
+ "reward": 0.3446485698223114,
355
+ "reward_std": 0.22104832530021667,
356
+ "rewards/<lambda>": 0.3446485698223114,
357
+ "step": 29
358
+ },
359
+ {
360
+ "completion_length": 160.0,
361
+ "epoch": 0.30612244897959184,
362
+ "grad_norm": 0.093221515417099,
363
+ "kl": 0.0011938215466216207,
364
+ "learning_rate": 3e-05,
365
+ "loss": 0.0,
366
+ "reward": 0.3021507263183594,
367
+ "reward_std": 0.26753896474838257,
368
+ "rewards/<lambda>": 0.3021507263183594,
369
+ "step": 30
370
+ },
371
+ {
372
+ "completion_length": 160.0,
373
+ "epoch": 0.3163265306122449,
374
+ "grad_norm": 0.10894666612148285,
375
+ "kl": 0.0014481329126283526,
376
+ "learning_rate": 3.1e-05,
377
+ "loss": 0.0001,
378
+ "reward": 0.22669407725334167,
379
+ "reward_std": 0.21502387523651123,
380
+ "rewards/<lambda>": 0.22669407725334167,
381
+ "step": 31
382
+ },
383
+ {
384
+ "completion_length": 160.0,
385
+ "epoch": 0.32653061224489793,
386
+ "grad_norm": 0.11627420037984848,
387
+ "kl": 0.0011792851146310568,
388
+ "learning_rate": 3.2000000000000005e-05,
389
+ "loss": 0.0,
390
+ "reward": 0.2387671172618866,
391
+ "reward_std": 0.20545344054698944,
392
+ "rewards/<lambda>": 0.2387671172618866,
393
+ "step": 32
394
+ },
395
+ {
396
+ "completion_length": 160.0,
397
+ "epoch": 0.336734693877551,
398
+ "grad_norm": 0.1079491674900055,
399
+ "kl": 0.0013044481165707111,
400
+ "learning_rate": 3.3e-05,
401
+ "loss": 0.0001,
402
+ "reward": 0.2432921826839447,
403
+ "reward_std": 0.2530010938644409,
404
+ "rewards/<lambda>": 0.2432921826839447,
405
+ "step": 33
406
+ },
407
+ {
408
+ "completion_length": 160.0,
409
+ "epoch": 0.3469387755102041,
410
+ "grad_norm": 0.09688923507928848,
411
+ "kl": 0.0012661240762099624,
412
+ "learning_rate": 3.4000000000000007e-05,
413
+ "loss": 0.0001,
414
+ "reward": 0.32528483867645264,
415
+ "reward_std": 0.2360536754131317,
416
+ "rewards/<lambda>": 0.32528483867645264,
417
+ "step": 34
418
+ },
419
+ {
420
+ "completion_length": 160.0,
421
+ "epoch": 0.35714285714285715,
422
+ "grad_norm": 0.07901012152433395,
423
+ "kl": 0.001169042894616723,
424
+ "learning_rate": 3.5e-05,
425
+ "loss": 0.0,
426
+ "reward": 0.2810063362121582,
427
+ "reward_std": 0.22572952508926392,
428
+ "rewards/<lambda>": 0.2810063362121582,
429
+ "step": 35
430
+ },
431
+ {
432
+ "completion_length": 160.0,
433
+ "epoch": 0.3673469387755102,
434
+ "grad_norm": 0.12785927951335907,
435
+ "kl": 0.0014555864036083221,
436
+ "learning_rate": 3.6e-05,
437
+ "loss": 0.0001,
438
+ "reward": 0.2540737986564636,
439
+ "reward_std": 0.24591577053070068,
440
+ "rewards/<lambda>": 0.2540737986564636,
441
+ "step": 36
442
+ },
443
+ {
444
+ "completion_length": 160.0,
445
+ "epoch": 0.37755102040816324,
446
+ "grad_norm": 0.08787187188863754,
447
+ "kl": 0.001047058729454875,
448
+ "learning_rate": 3.7e-05,
449
+ "loss": 0.0,
450
+ "reward": 0.24186044931411743,
451
+ "reward_std": 0.2187865972518921,
452
+ "rewards/<lambda>": 0.24186044931411743,
453
+ "step": 37
454
+ },
455
+ {
456
+ "completion_length": 160.0,
457
+ "epoch": 0.3877551020408163,
458
+ "grad_norm": 0.102202869951725,
459
+ "kl": 0.0013880159240216017,
460
+ "learning_rate": 3.8e-05,
461
+ "loss": 0.0001,
462
+ "reward": 0.2575429379940033,
463
+ "reward_std": 0.2186940610408783,
464
+ "rewards/<lambda>": 0.2575429379940033,
465
+ "step": 38
466
+ },
467
+ {
468
+ "completion_length": 160.0,
469
+ "epoch": 0.3979591836734694,
470
+ "grad_norm": 0.08456045389175415,
471
+ "kl": 0.0011803264496847987,
472
+ "learning_rate": 3.9000000000000006e-05,
473
+ "loss": 0.0,
474
+ "reward": 0.31993257999420166,
475
+ "reward_std": 0.22598499059677124,
476
+ "rewards/<lambda>": 0.31993257999420166,
477
+ "step": 39
478
+ },
479
+ {
480
+ "completion_length": 160.0,
481
+ "epoch": 0.40816326530612246,
482
+ "grad_norm": 0.10486605018377304,
483
+ "kl": 0.0018042891751974821,
484
+ "learning_rate": 4e-05,
485
+ "loss": 0.0001,
486
+ "reward": 0.29636475443840027,
487
+ "reward_std": 0.24350598454475403,
488
+ "rewards/<lambda>": 0.29636475443840027,
489
+ "step": 40
490
+ },
491
+ {
492
+ "completion_length": 160.0,
493
+ "epoch": 0.41836734693877553,
494
+ "grad_norm": 0.0880642980337143,
495
+ "kl": 0.001442858949303627,
496
+ "learning_rate": 4.1e-05,
497
+ "loss": 0.0001,
498
+ "reward": 0.41248711943626404,
499
+ "reward_std": 0.23531243205070496,
500
+ "rewards/<lambda>": 0.41248711943626404,
501
+ "step": 41
502
+ },
503
+ {
504
+ "completion_length": 160.0,
505
+ "epoch": 0.42857142857142855,
506
+ "grad_norm": 0.11210600286722183,
507
+ "kl": 0.0027525494806468487,
508
+ "learning_rate": 4.2e-05,
509
+ "loss": 0.0001,
510
+ "reward": 0.33057302236557007,
511
+ "reward_std": 0.2597821354866028,
512
+ "rewards/<lambda>": 0.33057302236557007,
513
+ "step": 42
514
+ },
515
+ {
516
+ "completion_length": 160.0,
517
+ "epoch": 0.4387755102040816,
518
+ "grad_norm": 0.1131657138466835,
519
+ "kl": 0.0013009419199079275,
520
+ "learning_rate": 4.3e-05,
521
+ "loss": 0.0001,
522
+ "reward": 0.3796292245388031,
523
+ "reward_std": 0.23504015803337097,
524
+ "rewards/<lambda>": 0.3796292245388031,
525
+ "step": 43
526
+ },
527
+ {
528
+ "completion_length": 160.0,
529
+ "epoch": 0.4489795918367347,
530
+ "grad_norm": 0.09555080533027649,
531
+ "kl": 0.002401509787887335,
532
+ "learning_rate": 4.4000000000000006e-05,
533
+ "loss": 0.0001,
534
+ "reward": 0.28863847255706787,
535
+ "reward_std": 0.2549838721752167,
536
+ "rewards/<lambda>": 0.28863847255706787,
537
+ "step": 44
538
+ },
539
+ {
540
+ "completion_length": 160.0,
541
+ "epoch": 0.45918367346938777,
542
+ "grad_norm": 0.12921357154846191,
543
+ "kl": 0.003915494307875633,
544
+ "learning_rate": 4.5e-05,
545
+ "loss": 0.0002,
546
+ "reward": 0.25585973262786865,
547
+ "reward_std": 0.2098313271999359,
548
+ "rewards/<lambda>": 0.25585973262786865,
549
+ "step": 45
550
+ },
551
+ {
552
+ "completion_length": 160.0,
553
+ "epoch": 0.46938775510204084,
554
+ "grad_norm": 0.10449738055467606,
555
+ "kl": 0.0029986370354890823,
556
+ "learning_rate": 4.600000000000001e-05,
557
+ "loss": 0.0001,
558
+ "reward": 0.24023020267486572,
559
+ "reward_std": 0.21866671741008759,
560
+ "rewards/<lambda>": 0.24023020267486572,
561
+ "step": 46
562
+ },
563
+ {
564
+ "completion_length": 160.0,
565
+ "epoch": 0.47959183673469385,
566
+ "grad_norm": 0.11932526528835297,
567
+ "kl": 0.0029086011927574873,
568
+ "learning_rate": 4.7e-05,
569
+ "loss": 0.0001,
570
+ "reward": 0.35698994994163513,
571
+ "reward_std": 0.244903564453125,
572
+ "rewards/<lambda>": 0.35698994994163513,
573
+ "step": 47
574
+ },
575
+ {
576
+ "completion_length": 160.0,
577
+ "epoch": 0.4897959183673469,
578
+ "grad_norm": 0.11577532440423965,
579
+ "kl": 0.0025195125490427017,
580
+ "learning_rate": 4.8e-05,
581
+ "loss": 0.0001,
582
+ "reward": 0.3163990378379822,
583
+ "reward_std": 0.25772640109062195,
584
+ "rewards/<lambda>": 0.3163990378379822,
585
+ "step": 48
586
+ },
587
+ {
588
+ "completion_length": 160.0,
589
+ "epoch": 0.5,
590
+ "grad_norm": 0.12613017857074738,
591
+ "kl": 0.002560165012255311,
592
+ "learning_rate": 4.9e-05,
593
+ "loss": 0.0001,
594
+ "reward": 0.28095588088035583,
595
+ "reward_std": 0.2137780636548996,
596
+ "rewards/<lambda>": 0.28095588088035583,
597
+ "step": 49
598
+ },
599
+ {
600
+ "completion_length": 160.0,
601
+ "epoch": 0.5102040816326531,
602
+ "grad_norm": 0.11029908806085587,
603
+ "kl": 0.003053056076169014,
604
+ "learning_rate": 5e-05,
605
+ "loss": 0.0001,
606
+ "reward": 0.2593611478805542,
607
+ "reward_std": 0.17565123736858368,
608
+ "rewards/<lambda>": 0.2593611478805542,
609
+ "step": 50
610
+ },
611
+ {
612
+ "completion_length": 160.0,
613
+ "epoch": 0.5204081632653061,
614
+ "grad_norm": 0.10181179642677307,
615
+ "kl": 0.0032034586183726788,
616
+ "learning_rate": 5.1000000000000006e-05,
617
+ "loss": 0.0001,
618
+ "reward": 0.3496597707271576,
619
+ "reward_std": 0.23656174540519714,
620
+ "rewards/<lambda>": 0.3496597707271576,
621
+ "step": 51
622
+ },
623
+ {
624
+ "completion_length": 160.0,
625
+ "epoch": 0.5306122448979592,
626
+ "grad_norm": 0.11612030863761902,
627
+ "kl": 0.0029043052345514297,
628
+ "learning_rate": 5.2000000000000004e-05,
629
+ "loss": 0.0001,
630
+ "reward": 0.31435415148735046,
631
+ "reward_std": 0.26738691329956055,
632
+ "rewards/<lambda>": 0.31435415148735046,
633
+ "step": 52
634
+ },
635
+ {
636
+ "completion_length": 160.0,
637
+ "epoch": 0.5408163265306123,
638
+ "grad_norm": 0.11753087490797043,
639
+ "kl": 0.004537411965429783,
640
+ "learning_rate": 5.300000000000001e-05,
641
+ "loss": 0.0002,
642
+ "reward": 0.27415716648101807,
643
+ "reward_std": 0.2496672123670578,
644
+ "rewards/<lambda>": 0.27415716648101807,
645
+ "step": 53
646
+ },
647
+ {
648
+ "completion_length": 160.0,
649
+ "epoch": 0.5510204081632653,
650
+ "grad_norm": 0.09143570065498352,
651
+ "kl": 0.0020023572724312544,
652
+ "learning_rate": 5.4000000000000005e-05,
653
+ "loss": 0.0001,
654
+ "reward": 0.27505844831466675,
655
+ "reward_std": 0.2481660097837448,
656
+ "rewards/<lambda>": 0.27505844831466675,
657
+ "step": 54
658
+ },
659
+ {
660
+ "completion_length": 160.0,
661
+ "epoch": 0.5612244897959183,
662
+ "grad_norm": 0.11042241752147675,
663
+ "kl": 0.002455132082104683,
664
+ "learning_rate": 5.500000000000001e-05,
665
+ "loss": 0.0001,
666
+ "reward": 0.2277480959892273,
667
+ "reward_std": 0.2573385238647461,
668
+ "rewards/<lambda>": 0.2277480959892273,
669
+ "step": 55
670
+ },
671
+ {
672
+ "completion_length": 160.0,
673
+ "epoch": 0.5714285714285714,
674
+ "grad_norm": 0.1062241867184639,
675
+ "kl": 0.003958327695727348,
676
+ "learning_rate": 5.6000000000000006e-05,
677
+ "loss": 0.0002,
678
+ "reward": 0.3246029019355774,
679
+ "reward_std": 0.25107985734939575,
680
+ "rewards/<lambda>": 0.3246029019355774,
681
+ "step": 56
682
+ },
683
+ {
684
+ "completion_length": 160.0,
685
+ "epoch": 0.5816326530612245,
686
+ "grad_norm": 0.09364596009254456,
687
+ "kl": 0.0028939968906342983,
688
+ "learning_rate": 5.6999999999999996e-05,
689
+ "loss": 0.0001,
690
+ "reward": 0.33692467212677,
691
+ "reward_std": 0.2620271146297455,
692
+ "rewards/<lambda>": 0.33692467212677,
693
+ "step": 57
694
+ },
695
+ {
696
+ "completion_length": 160.0,
697
+ "epoch": 0.5918367346938775,
698
+ "grad_norm": 0.10371523350477219,
699
+ "kl": 0.0031635533086955547,
700
+ "learning_rate": 5.8e-05,
701
+ "loss": 0.0001,
702
+ "reward": 0.26338717341423035,
703
+ "reward_std": 0.25605374574661255,
704
+ "rewards/<lambda>": 0.26338717341423035,
705
+ "step": 58
706
+ },
707
+ {
708
+ "completion_length": 160.0,
709
+ "epoch": 0.6020408163265306,
710
+ "grad_norm": 0.119356170296669,
711
+ "kl": 0.004045659676194191,
712
+ "learning_rate": 5.9e-05,
713
+ "loss": 0.0002,
714
+ "reward": 0.26655131578445435,
715
+ "reward_std": 0.23418131470680237,
716
+ "rewards/<lambda>": 0.26655131578445435,
717
+ "step": 59
718
+ },
719
+ {
720
+ "completion_length": 160.0,
721
+ "epoch": 0.6122448979591837,
722
+ "grad_norm": 0.14208924770355225,
723
+ "kl": 0.005053609609603882,
724
+ "learning_rate": 6e-05,
725
+ "loss": 0.0002,
726
+ "reward": 0.3059399425983429,
727
+ "reward_std": 0.2509281635284424,
728
+ "rewards/<lambda>": 0.3059399425983429,
729
+ "step": 60
730
+ },
731
+ {
732
+ "completion_length": 160.0,
733
+ "epoch": 0.6224489795918368,
734
+ "grad_norm": 0.19251689314842224,
735
+ "kl": 0.005135521292686462,
736
+ "learning_rate": 6.1e-05,
737
+ "loss": 0.0002,
738
+ "reward": 0.3289321959018707,
739
+ "reward_std": 0.25894391536712646,
740
+ "rewards/<lambda>": 0.3289321959018707,
741
+ "step": 61
742
+ },
743
+ {
744
+ "completion_length": 160.0,
745
+ "epoch": 0.6326530612244898,
746
+ "grad_norm": 0.10605209320783615,
747
+ "kl": 0.004940195474773645,
748
+ "learning_rate": 6.2e-05,
749
+ "loss": 0.0002,
750
+ "reward": 0.3096313178539276,
751
+ "reward_std": 0.25553008913993835,
752
+ "rewards/<lambda>": 0.3096313178539276,
753
+ "step": 62
754
+ },
755
+ {
756
+ "completion_length": 160.0,
757
+ "epoch": 0.6428571428571429,
758
+ "grad_norm": 0.1347755789756775,
759
+ "kl": 0.007013984024524689,
760
+ "learning_rate": 6.3e-05,
761
+ "loss": 0.0003,
762
+ "reward": 0.3279077112674713,
763
+ "reward_std": 0.24912631511688232,
764
+ "rewards/<lambda>": 0.3279077112674713,
765
+ "step": 63
766
+ },
767
+ {
768
+ "completion_length": 160.0,
769
+ "epoch": 0.6530612244897959,
770
+ "grad_norm": 0.11074844002723694,
771
+ "kl": 0.005811232142150402,
772
+ "learning_rate": 6.400000000000001e-05,
773
+ "loss": 0.0002,
774
+ "reward": 0.26557987928390503,
775
+ "reward_std": 0.2595665454864502,
776
+ "rewards/<lambda>": 0.26557987928390503,
777
+ "step": 64
778
+ },
779
+ {
780
+ "completion_length": 160.0,
781
+ "epoch": 0.6632653061224489,
782
+ "grad_norm": 0.08949097245931625,
783
+ "kl": 0.0062391310930252075,
784
+ "learning_rate": 6.500000000000001e-05,
785
+ "loss": 0.0002,
786
+ "reward": 0.30340054631233215,
787
+ "reward_std": 0.23612141609191895,
788
+ "rewards/<lambda>": 0.30340054631233215,
789
+ "step": 65
790
+ },
791
+ {
792
+ "completion_length": 160.0,
793
+ "epoch": 0.673469387755102,
794
+ "grad_norm": 0.09197568893432617,
795
+ "kl": 0.006357334554195404,
796
+ "learning_rate": 6.6e-05,
797
+ "loss": 0.0003,
798
+ "reward": 0.305142879486084,
799
+ "reward_std": 0.25195544958114624,
800
+ "rewards/<lambda>": 0.305142879486084,
801
+ "step": 66
802
+ },
803
+ {
804
+ "completion_length": 160.0,
805
+ "epoch": 0.6836734693877551,
806
+ "grad_norm": 0.11795815080404282,
807
+ "kl": 0.006672342773526907,
808
+ "learning_rate": 6.7e-05,
809
+ "loss": 0.0003,
810
+ "reward": 0.2330521047115326,
811
+ "reward_std": 0.24398502707481384,
812
+ "rewards/<lambda>": 0.2330521047115326,
813
+ "step": 67
814
+ },
815
+ {
816
+ "completion_length": 160.0,
817
+ "epoch": 0.6938775510204082,
818
+ "grad_norm": 0.1380680352449417,
819
+ "kl": 0.008953109383583069,
820
+ "learning_rate": 6.800000000000001e-05,
821
+ "loss": 0.0004,
822
+ "reward": 0.3820633292198181,
823
+ "reward_std": 0.24111545085906982,
824
+ "rewards/<lambda>": 0.3820633292198181,
825
+ "step": 68
826
+ },
827
+ {
828
+ "completion_length": 160.0,
829
+ "epoch": 0.7040816326530612,
830
+ "grad_norm": 0.10920781642198563,
831
+ "kl": 0.006580730434507132,
832
+ "learning_rate": 6.9e-05,
833
+ "loss": 0.0003,
834
+ "reward": 0.30754148960113525,
835
+ "reward_std": 0.22947274148464203,
836
+ "rewards/<lambda>": 0.30754148960113525,
837
+ "step": 69
838
+ },
839
+ {
840
+ "completion_length": 160.0,
841
+ "epoch": 0.7142857142857143,
842
+ "grad_norm": 0.11049173772335052,
843
+ "kl": 0.006469545420259237,
844
+ "learning_rate": 7e-05,
845
+ "loss": 0.0003,
846
+ "reward": 0.2494296282529831,
847
+ "reward_std": 0.2338804006576538,
848
+ "rewards/<lambda>": 0.2494296282529831,
849
+ "step": 70
850
+ },
851
+ {
852
+ "completion_length": 160.0,
853
+ "epoch": 0.7244897959183674,
854
+ "grad_norm": 0.12242847681045532,
855
+ "kl": 0.0065796151757240295,
856
+ "learning_rate": 7.1e-05,
857
+ "loss": 0.0003,
858
+ "reward": 0.35586172342300415,
859
+ "reward_std": 0.22659151256084442,
860
+ "rewards/<lambda>": 0.35586172342300415,
861
+ "step": 71
862
+ },
863
+ {
864
+ "completion_length": 160.0,
865
+ "epoch": 0.7346938775510204,
866
+ "grad_norm": 0.10463358461856842,
867
+ "kl": 0.008036931976675987,
868
+ "learning_rate": 7.2e-05,
869
+ "loss": 0.0003,
870
+ "reward": 0.3306816518306732,
871
+ "reward_std": 0.21499347686767578,
872
+ "rewards/<lambda>": 0.3306816518306732,
873
+ "step": 72
874
+ },
875
+ {
876
+ "completion_length": 160.0,
877
+ "epoch": 0.7448979591836735,
878
+ "grad_norm": 0.10549326241016388,
879
+ "kl": 0.005967782810330391,
880
+ "learning_rate": 7.3e-05,
881
+ "loss": 0.0002,
882
+ "reward": 0.26001977920532227,
883
+ "reward_std": 0.20824775099754333,
884
+ "rewards/<lambda>": 0.26001977920532227,
885
+ "step": 73
886
+ },
887
+ {
888
+ "completion_length": 160.0,
889
+ "epoch": 0.7551020408163265,
890
+ "grad_norm": 0.10788781940937042,
891
+ "kl": 0.004906760528683662,
892
+ "learning_rate": 7.4e-05,
893
+ "loss": 0.0002,
894
+ "reward": 0.2874922752380371,
895
+ "reward_std": 0.24835163354873657,
896
+ "rewards/<lambda>": 0.2874922752380371,
897
+ "step": 74
898
+ },
899
+ {
900
+ "completion_length": 160.0,
901
+ "epoch": 0.7653061224489796,
902
+ "grad_norm": 0.09784362465143204,
903
+ "kl": 0.005803011357784271,
904
+ "learning_rate": 7.500000000000001e-05,
905
+ "loss": 0.0002,
906
+ "reward": 0.2684209942817688,
907
+ "reward_std": 0.2644413113594055,
908
+ "rewards/<lambda>": 0.2684209942817688,
909
+ "step": 75
910
+ },
911
+ {
912
+ "completion_length": 160.0,
913
+ "epoch": 0.7755102040816326,
914
+ "grad_norm": 0.16001495718955994,
915
+ "kl": 0.006752543151378632,
916
+ "learning_rate": 7.6e-05,
917
+ "loss": 0.0003,
918
+ "reward": 0.3604978919029236,
919
+ "reward_std": 0.21626180410385132,
920
+ "rewards/<lambda>": 0.3604978919029236,
921
+ "step": 76
922
+ },
923
+ {
924
+ "completion_length": 160.0,
925
+ "epoch": 0.7857142857142857,
926
+ "grad_norm": 0.13324351608753204,
927
+ "kl": 0.008819150738418102,
928
+ "learning_rate": 7.7e-05,
929
+ "loss": 0.0004,
930
+ "reward": 0.3122796416282654,
931
+ "reward_std": 0.23514878749847412,
932
+ "rewards/<lambda>": 0.3122796416282654,
933
+ "step": 77
934
+ },
935
+ {
936
+ "completion_length": 160.0,
937
+ "epoch": 0.7959183673469388,
938
+ "grad_norm": 0.11151094734668732,
939
+ "kl": 0.0060831112787127495,
940
+ "learning_rate": 7.800000000000001e-05,
941
+ "loss": 0.0002,
942
+ "reward": 0.3716966509819031,
943
+ "reward_std": 0.25390562415122986,
944
+ "rewards/<lambda>": 0.3716966509819031,
945
+ "step": 78
946
+ },
947
+ {
948
+ "completion_length": 160.0,
949
+ "epoch": 0.8061224489795918,
950
+ "grad_norm": 0.1258833110332489,
951
+ "kl": 0.00652284873649478,
952
+ "learning_rate": 7.900000000000001e-05,
953
+ "loss": 0.0003,
954
+ "reward": 0.31094950437545776,
955
+ "reward_std": 0.24983155727386475,
956
+ "rewards/<lambda>": 0.31094950437545776,
957
+ "step": 79
958
+ },
959
+ {
960
+ "completion_length": 160.0,
961
+ "epoch": 0.8163265306122449,
962
+ "grad_norm": 0.09824781864881516,
963
+ "kl": 0.007383039221167564,
964
+ "learning_rate": 8e-05,
965
+ "loss": 0.0003,
966
+ "reward": 0.3597716689109802,
967
+ "reward_std": 0.24361717700958252,
968
+ "rewards/<lambda>": 0.3597716689109802,
969
+ "step": 80
970
+ },
971
+ {
972
+ "completion_length": 160.0,
973
+ "epoch": 0.826530612244898,
974
+ "grad_norm": 0.08224570006132126,
975
+ "kl": 0.005881953053176403,
976
+ "learning_rate": 8.1e-05,
977
+ "loss": 0.0002,
978
+ "reward": 0.27791696786880493,
979
+ "reward_std": 0.2182818502187729,
980
+ "rewards/<lambda>": 0.27791696786880493,
981
+ "step": 81
982
+ },
983
+ {
984
+ "completion_length": 160.0,
985
+ "epoch": 0.8367346938775511,
986
+ "grad_norm": 0.12581421434879303,
987
+ "kl": 0.006681859493255615,
988
+ "learning_rate": 8.2e-05,
989
+ "loss": 0.0003,
990
+ "reward": 0.3261301517486572,
991
+ "reward_std": 0.2615125775337219,
992
+ "rewards/<lambda>": 0.3261301517486572,
993
+ "step": 82
994
+ },
995
+ {
996
+ "completion_length": 160.0,
997
+ "epoch": 0.8469387755102041,
998
+ "grad_norm": 0.10322140902280807,
999
+ "kl": 0.006312561221420765,
1000
+ "learning_rate": 8.3e-05,
1001
+ "loss": 0.0003,
1002
+ "reward": 0.3625139594078064,
1003
+ "reward_std": 0.24003905057907104,
1004
+ "rewards/<lambda>": 0.3625139594078064,
1005
+ "step": 83
1006
+ },
1007
+ {
1008
+ "completion_length": 160.0,
1009
+ "epoch": 0.8571428571428571,
1010
+ "grad_norm": 0.0966828241944313,
1011
+ "kl": 0.0061577907763421535,
1012
+ "learning_rate": 8.4e-05,
1013
+ "loss": 0.0002,
1014
+ "reward": 0.27837446331977844,
1015
+ "reward_std": 0.24895024299621582,
1016
+ "rewards/<lambda>": 0.27837446331977844,
1017
+ "step": 84
1018
+ },
1019
+ {
1020
+ "completion_length": 160.0,
1021
+ "epoch": 0.8673469387755102,
1022
+ "grad_norm": 0.09297024458646774,
1023
+ "kl": 0.005875007715076208,
1024
+ "learning_rate": 8.5e-05,
1025
+ "loss": 0.0002,
1026
+ "reward": 0.3240439295768738,
1027
+ "reward_std": 0.2554982602596283,
1028
+ "rewards/<lambda>": 0.3240439295768738,
1029
+ "step": 85
1030
+ },
1031
+ {
1032
+ "completion_length": 160.0,
1033
+ "epoch": 0.8775510204081632,
1034
+ "grad_norm": 0.12204565852880478,
1035
+ "kl": 0.006396137177944183,
1036
+ "learning_rate": 8.6e-05,
1037
+ "loss": 0.0003,
1038
+ "reward": 0.24172773957252502,
1039
+ "reward_std": 0.2518821060657501,
1040
+ "rewards/<lambda>": 0.24172773957252502,
1041
+ "step": 86
1042
+ },
1043
+ {
1044
+ "completion_length": 160.0,
1045
+ "epoch": 0.8877551020408163,
1046
+ "grad_norm": 0.08884488046169281,
1047
+ "kl": 0.00668664975091815,
1048
+ "learning_rate": 8.7e-05,
1049
+ "loss": 0.0003,
1050
+ "reward": 0.28681838512420654,
1051
+ "reward_std": 0.25753265619277954,
1052
+ "rewards/<lambda>": 0.28681838512420654,
1053
+ "step": 87
1054
+ },
1055
+ {
1056
+ "completion_length": 160.0,
1057
+ "epoch": 0.8979591836734694,
1058
+ "grad_norm": 0.08016186207532883,
1059
+ "kl": 0.006137486081570387,
1060
+ "learning_rate": 8.800000000000001e-05,
1061
+ "loss": 0.0002,
1062
+ "reward": 0.2647368907928467,
1063
+ "reward_std": 0.24885067343711853,
1064
+ "rewards/<lambda>": 0.2647368907928467,
1065
+ "step": 88
1066
+ },
1067
+ {
1068
+ "completion_length": 160.0,
1069
+ "epoch": 0.9081632653061225,
1070
+ "grad_norm": 0.10147328674793243,
1071
+ "kl": 0.00917094200849533,
1072
+ "learning_rate": 8.900000000000001e-05,
1073
+ "loss": 0.0004,
1074
+ "reward": 0.29730311036109924,
1075
+ "reward_std": 0.2591564953327179,
1076
+ "rewards/<lambda>": 0.29730311036109924,
1077
+ "step": 89
1078
+ },
1079
+ {
1080
+ "completion_length": 160.0,
1081
+ "epoch": 0.9183673469387755,
1082
+ "grad_norm": 0.12398728728294373,
1083
+ "kl": 0.007823077961802483,
1084
+ "learning_rate": 9e-05,
1085
+ "loss": 0.0003,
1086
+ "reward": 0.2589433193206787,
1087
+ "reward_std": 0.24704992771148682,
1088
+ "rewards/<lambda>": 0.2589433193206787,
1089
+ "step": 90
1090
+ },
1091
+ {
1092
+ "completion_length": 160.0,
1093
+ "epoch": 0.9285714285714286,
1094
+ "grad_norm": 0.1265306919813156,
1095
+ "kl": 0.009746653027832508,
1096
+ "learning_rate": 9.1e-05,
1097
+ "loss": 0.0004,
1098
+ "reward": 0.2624666094779968,
1099
+ "reward_std": 0.26040005683898926,
1100
+ "rewards/<lambda>": 0.2624666094779968,
1101
+ "step": 91
1102
+ },
1103
+ {
1104
+ "completion_length": 160.0,
1105
+ "epoch": 0.9387755102040817,
1106
+ "grad_norm": 0.09663106501102448,
1107
+ "kl": 0.009732738137245178,
1108
+ "learning_rate": 9.200000000000001e-05,
1109
+ "loss": 0.0004,
1110
+ "reward": 0.31370311975479126,
1111
+ "reward_std": 0.2633877992630005,
1112
+ "rewards/<lambda>": 0.31370311975479126,
1113
+ "step": 92
1114
+ },
1115
+ {
1116
+ "completion_length": 160.0,
1117
+ "epoch": 0.9489795918367347,
1118
+ "grad_norm": 0.12955360114574432,
1119
+ "kl": 0.011157519184052944,
1120
+ "learning_rate": 9.300000000000001e-05,
1121
+ "loss": 0.0004,
1122
+ "reward": 0.3296325206756592,
1123
+ "reward_std": 0.26135072112083435,
1124
+ "rewards/<lambda>": 0.3296325206756592,
1125
+ "step": 93
1126
+ },
1127
+ {
1128
+ "completion_length": 160.0,
1129
+ "epoch": 0.9591836734693877,
1130
+ "grad_norm": 0.07906248420476913,
1131
+ "kl": 0.00876379944384098,
1132
+ "learning_rate": 9.4e-05,
1133
+ "loss": 0.0004,
1134
+ "reward": 0.38581177592277527,
1135
+ "reward_std": 0.21951782703399658,
1136
+ "rewards/<lambda>": 0.38581177592277527,
1137
+ "step": 94
1138
+ },
1139
+ {
1140
+ "completion_length": 160.0,
1141
+ "epoch": 0.9693877551020408,
1142
+ "grad_norm": 0.10264045745134354,
1143
+ "kl": 0.013823926448822021,
1144
+ "learning_rate": 9.5e-05,
1145
+ "loss": 0.0006,
1146
+ "reward": 0.37457725405693054,
1147
+ "reward_std": 0.22653540968894958,
1148
+ "rewards/<lambda>": 0.37457725405693054,
1149
+ "step": 95
1150
+ },
1151
+ {
1152
+ "completion_length": 160.0,
1153
+ "epoch": 0.9795918367346939,
1154
+ "grad_norm": 0.09052637219429016,
1155
+ "kl": 0.011999586597084999,
1156
+ "learning_rate": 9.6e-05,
1157
+ "loss": 0.0005,
1158
+ "reward": 0.3207855224609375,
1159
+ "reward_std": 0.23623248934745789,
1160
+ "rewards/<lambda>": 0.3207855224609375,
1161
+ "step": 96
1162
+ },
1163
+ {
1164
+ "completion_length": 160.0,
1165
+ "epoch": 0.9897959183673469,
1166
+ "grad_norm": 0.11846129596233368,
1167
+ "kl": 0.011448493227362633,
1168
+ "learning_rate": 9.7e-05,
1169
+ "loss": 0.0005,
1170
+ "reward": 0.32892781496047974,
1171
+ "reward_std": 0.23297683894634247,
1172
+ "rewards/<lambda>": 0.32892781496047974,
1173
+ "step": 97
1174
+ },
1175
+ {
1176
+ "completion_length": 160.0,
1177
+ "epoch": 1.0,
1178
+ "grad_norm": 0.09638947248458862,
1179
+ "kl": 0.010949314571917057,
1180
+ "learning_rate": 9.8e-05,
1181
+ "loss": 0.0004,
1182
+ "reward": 0.4758386015892029,
1183
+ "reward_std": 0.2482748031616211,
1184
+ "rewards/<lambda>": 0.4758386015892029,
1185
+ "step": 98
1186
+ },
1187
+ {
1188
+ "completion_length": 160.0,
1189
+ "epoch": 1.010204081632653,
1190
+ "grad_norm": 0.09481366723775864,
1191
+ "kl": 0.01085271593183279,
1192
+ "learning_rate": 9.900000000000001e-05,
1193
+ "loss": 0.0004,
1194
+ "reward": 0.3523382544517517,
1195
+ "reward_std": 0.2240390032529831,
1196
+ "rewards/<lambda>": 0.3523382544517517,
1197
+ "step": 99
1198
+ },
1199
+ {
1200
+ "completion_length": 160.0,
1201
+ "epoch": 1.0204081632653061,
1202
+ "grad_norm": 0.09950532019138336,
1203
+ "kl": 0.012218523770570755,
1204
+ "learning_rate": 0.0001,
1205
+ "loss": 0.0005,
1206
+ "reward": 0.28119099140167236,
1207
+ "reward_std": 0.22250661253929138,
1208
+ "rewards/<lambda>": 0.28119099140167236,
1209
+ "step": 100
1210
+ }
1211
+ ],
1212
+ "logging_steps": 1,
1213
+ "max_steps": 294,
1214
+ "num_input_tokens_seen": 0,
1215
+ "num_train_epochs": 3,
1216
+ "save_steps": 100,
1217
+ "stateful_callbacks": {
1218
+ "TrainerControl": {
1219
+ "args": {
1220
+ "should_epoch_stop": false,
1221
+ "should_evaluate": false,
1222
+ "should_log": false,
1223
+ "should_save": true,
1224
+ "should_training_stop": false
1225
+ },
1226
+ "attributes": {}
1227
+ }
1228
+ },
1229
+ "total_flos": 0.0,
1230
+ "train_batch_size": 8,
1231
+ "trial_name": null,
1232
+ "trial_params": null
1233
+ }
checkpoint-100/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:10bbf0a23b8f4b09a58ff960658e6bc23228310be351cac2a5c1ed047b491f12
3
+ size 5560
checkpoint-100/vocab.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-200/README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: HuggingFaceTB/SmolLM2-360M
3
+ library_name: peft
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.14.0
checkpoint-200/adapter_config.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "HuggingFaceTB/SmolLM2-360M",
5
+ "bias": "none",
6
+ "eva_config": null,
7
+ "exclude_modules": null,
8
+ "fan_in_fan_out": false,
9
+ "inference_mode": true,
10
+ "init_lora_weights": true,
11
+ "layer_replication": null,
12
+ "layers_pattern": null,
13
+ "layers_to_transform": null,
14
+ "loftq_config": {},
15
+ "lora_alpha": 64,
16
+ "lora_bias": false,
17
+ "lora_dropout": 0.05,
18
+ "megatron_config": null,
19
+ "megatron_core": "megatron.core",
20
+ "modules_to_save": null,
21
+ "peft_type": "LORA",
22
+ "r": 32,
23
+ "rank_pattern": {},
24
+ "revision": null,
25
+ "target_modules": [
26
+ "k_proj",
27
+ "up_proj",
28
+ "v_proj",
29
+ "gate_proj",
30
+ "q_proj",
31
+ "down_proj",
32
+ "o_proj"
33
+ ],
34
+ "task_type": "CAUSAL_LM",
35
+ "use_dora": false,
36
+ "use_rslora": false
37
+ }
checkpoint-200/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c86fe316990db45f2615636367af79e298f133191741cb9217f15ece2ba7fb2c
3
+ size 69527352
checkpoint-200/merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-200/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:66636134445450dd72eeed7d9b5fb716409f792af7fcb3de584a20f44e3523de
3
+ size 139313234
checkpoint-200/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f6342846135f94c1e83447565a85d30dd89f714f3061c6b252aba6549e85adc
3
+ size 14244
checkpoint-200/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d298159cdcad213cd5f37f7528bef2babcacaf8836f0e4d1afc1d6de3993582
3
+ size 1064
checkpoint-200/special_tokens_map.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ {
4
+ "content": "<|im_start|>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ {
11
+ "content": "<|im_end|>",
12
+ "lstrip": false,
13
+ "normalized": false,
14
+ "rstrip": false,
15
+ "single_word": false
16
+ }
17
+ ],
18
+ "bos_token": "<|im_start|>",
19
+ "eos_token": "<|im_end|>",
20
+ "pad_token": "<|im_end|>",
21
+ "unk_token": {
22
+ "content": "<|endoftext|>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": false,
26
+ "single_word": false
27
+ }
28
+ }
checkpoint-200/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-200/tokenizer_config.json ADDED
@@ -0,0 +1,156 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "0": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": false,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "1": {
13
+ "content": "<|im_start|>",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "2": {
21
+ "content": "<|im_end|>",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "3": {
29
+ "content": "<repo_name>",
30
+ "lstrip": false,
31
+ "normalized": false,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "4": {
37
+ "content": "<reponame>",
38
+ "lstrip": false,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ },
44
+ "5": {
45
+ "content": "<file_sep>",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false,
50
+ "special": true
51
+ },
52
+ "6": {
53
+ "content": "<filename>",
54
+ "lstrip": false,
55
+ "normalized": false,
56
+ "rstrip": false,
57
+ "single_word": false,
58
+ "special": true
59
+ },
60
+ "7": {
61
+ "content": "<gh_stars>",
62
+ "lstrip": false,
63
+ "normalized": false,
64
+ "rstrip": false,
65
+ "single_word": false,
66
+ "special": true
67
+ },
68
+ "8": {
69
+ "content": "<issue_start>",
70
+ "lstrip": false,
71
+ "normalized": false,
72
+ "rstrip": false,
73
+ "single_word": false,
74
+ "special": true
75
+ },
76
+ "9": {
77
+ "content": "<issue_comment>",
78
+ "lstrip": false,
79
+ "normalized": false,
80
+ "rstrip": false,
81
+ "single_word": false,
82
+ "special": true
83
+ },
84
+ "10": {
85
+ "content": "<issue_closed>",
86
+ "lstrip": false,
87
+ "normalized": false,
88
+ "rstrip": false,
89
+ "single_word": false,
90
+ "special": true
91
+ },
92
+ "11": {
93
+ "content": "<jupyter_start>",
94
+ "lstrip": false,
95
+ "normalized": false,
96
+ "rstrip": false,
97
+ "single_word": false,
98
+ "special": true
99
+ },
100
+ "12": {
101
+ "content": "<jupyter_text>",
102
+ "lstrip": false,
103
+ "normalized": false,
104
+ "rstrip": false,
105
+ "single_word": false,
106
+ "special": true
107
+ },
108
+ "13": {
109
+ "content": "<jupyter_code>",
110
+ "lstrip": false,
111
+ "normalized": false,
112
+ "rstrip": false,
113
+ "single_word": false,
114
+ "special": true
115
+ },
116
+ "14": {
117
+ "content": "<jupyter_output>",
118
+ "lstrip": false,
119
+ "normalized": false,
120
+ "rstrip": false,
121
+ "single_word": false,
122
+ "special": true
123
+ },
124
+ "15": {
125
+ "content": "<jupyter_script>",
126
+ "lstrip": false,
127
+ "normalized": false,
128
+ "rstrip": false,
129
+ "single_word": false,
130
+ "special": true
131
+ },
132
+ "16": {
133
+ "content": "<empty_output>",
134
+ "lstrip": false,
135
+ "normalized": false,
136
+ "rstrip": false,
137
+ "single_word": false,
138
+ "special": true
139
+ }
140
+ },
141
+ "additional_special_tokens": [
142
+ "<|im_start|>",
143
+ "<|im_end|>"
144
+ ],
145
+ "bos_token": "<|im_start|>",
146
+ "chat_template": "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
147
+ "clean_up_tokenization_spaces": false,
148
+ "eos_token": "<|im_end|>",
149
+ "extra_special_tokens": {},
150
+ "model_max_length": 8192,
151
+ "pad_token": "<|im_end|>",
152
+ "padding_side": "left",
153
+ "tokenizer_class": "GPT2Tokenizer",
154
+ "unk_token": "<|endoftext|>",
155
+ "vocab_size": 49152
156
+ }
checkpoint-200/trainer_state.json ADDED
@@ -0,0 +1,2433 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 2.0408163265306123,
5
+ "eval_steps": 500,
6
+ "global_step": 200,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "completion_length": 160.0,
13
+ "epoch": 0.01020408163265306,
14
+ "grad_norm": 0.1883362978696823,
15
+ "kl": 0.0,
16
+ "learning_rate": 1.0000000000000002e-06,
17
+ "loss": 0.0,
18
+ "reward": 0.35154566168785095,
19
+ "reward_std": 0.18836843967437744,
20
+ "rewards/<lambda>": 0.35154566168785095,
21
+ "step": 1
22
+ },
23
+ {
24
+ "completion_length": 160.0,
25
+ "epoch": 0.02040816326530612,
26
+ "grad_norm": 0.08015607297420502,
27
+ "kl": 0.0,
28
+ "learning_rate": 2.0000000000000003e-06,
29
+ "loss": 0.0,
30
+ "reward": 0.24049028754234314,
31
+ "reward_std": 0.23294341564178467,
32
+ "rewards/<lambda>": 0.24049028754234314,
33
+ "step": 2
34
+ },
35
+ {
36
+ "completion_length": 160.0,
37
+ "epoch": 0.030612244897959183,
38
+ "grad_norm": 0.09276824444532394,
39
+ "kl": 0.0009410024504177272,
40
+ "learning_rate": 3e-06,
41
+ "loss": 0.0,
42
+ "reward": 0.32676321268081665,
43
+ "reward_std": 0.25492924451828003,
44
+ "rewards/<lambda>": 0.32676321268081665,
45
+ "step": 3
46
+ },
47
+ {
48
+ "completion_length": 160.0,
49
+ "epoch": 0.04081632653061224,
50
+ "grad_norm": 0.10654664784669876,
51
+ "kl": 0.001077650347724557,
52
+ "learning_rate": 4.000000000000001e-06,
53
+ "loss": 0.0,
54
+ "reward": 0.2009219378232956,
55
+ "reward_std": 0.21452507376670837,
56
+ "rewards/<lambda>": 0.2009219378232956,
57
+ "step": 4
58
+ },
59
+ {
60
+ "completion_length": 160.0,
61
+ "epoch": 0.05102040816326531,
62
+ "grad_norm": 0.11217533051967621,
63
+ "kl": 0.001070915488526225,
64
+ "learning_rate": 5e-06,
65
+ "loss": 0.0,
66
+ "reward": 0.3001704216003418,
67
+ "reward_std": 0.24798467755317688,
68
+ "rewards/<lambda>": 0.3001704216003418,
69
+ "step": 5
70
+ },
71
+ {
72
+ "completion_length": 160.0,
73
+ "epoch": 0.061224489795918366,
74
+ "grad_norm": 0.09305333346128464,
75
+ "kl": 0.0010154233314096928,
76
+ "learning_rate": 6e-06,
77
+ "loss": 0.0,
78
+ "reward": 0.2345525473356247,
79
+ "reward_std": 0.22145482897758484,
80
+ "rewards/<lambda>": 0.2345525473356247,
81
+ "step": 6
82
+ },
83
+ {
84
+ "completion_length": 160.0,
85
+ "epoch": 0.07142857142857142,
86
+ "grad_norm": 0.13233615458011627,
87
+ "kl": 0.0011612953385338187,
88
+ "learning_rate": 7.000000000000001e-06,
89
+ "loss": 0.0,
90
+ "reward": 0.3054887056350708,
91
+ "reward_std": 0.21511563658714294,
92
+ "rewards/<lambda>": 0.3054887056350708,
93
+ "step": 7
94
+ },
95
+ {
96
+ "completion_length": 160.0,
97
+ "epoch": 0.08163265306122448,
98
+ "grad_norm": 0.1244489848613739,
99
+ "kl": 0.000984564539976418,
100
+ "learning_rate": 8.000000000000001e-06,
101
+ "loss": 0.0,
102
+ "reward": 0.27341318130493164,
103
+ "reward_std": 0.23464292287826538,
104
+ "rewards/<lambda>": 0.27341318130493164,
105
+ "step": 8
106
+ },
107
+ {
108
+ "completion_length": 160.0,
109
+ "epoch": 0.09183673469387756,
110
+ "grad_norm": 0.09130726009607315,
111
+ "kl": 0.0008879535598680377,
112
+ "learning_rate": 9e-06,
113
+ "loss": 0.0,
114
+ "reward": 0.3593016564846039,
115
+ "reward_std": 0.2495943158864975,
116
+ "rewards/<lambda>": 0.3593016564846039,
117
+ "step": 9
118
+ },
119
+ {
120
+ "completion_length": 160.0,
121
+ "epoch": 0.10204081632653061,
122
+ "grad_norm": 0.15256966650485992,
123
+ "kl": 0.0011160458670929074,
124
+ "learning_rate": 1e-05,
125
+ "loss": 0.0,
126
+ "reward": 0.3049771785736084,
127
+ "reward_std": 0.26634567975997925,
128
+ "rewards/<lambda>": 0.3049771785736084,
129
+ "step": 10
130
+ },
131
+ {
132
+ "completion_length": 160.0,
133
+ "epoch": 0.11224489795918367,
134
+ "grad_norm": 0.09210026264190674,
135
+ "kl": 0.0010061098728328943,
136
+ "learning_rate": 1.1000000000000001e-05,
137
+ "loss": 0.0,
138
+ "reward": 0.2637690603733063,
139
+ "reward_std": 0.21869879961013794,
140
+ "rewards/<lambda>": 0.2637690603733063,
141
+ "step": 11
142
+ },
143
+ {
144
+ "completion_length": 160.0,
145
+ "epoch": 0.12244897959183673,
146
+ "grad_norm": 0.09713605791330338,
147
+ "kl": 0.0010679092956706882,
148
+ "learning_rate": 1.2e-05,
149
+ "loss": 0.0,
150
+ "reward": 0.36334261298179626,
151
+ "reward_std": 0.24027395248413086,
152
+ "rewards/<lambda>": 0.36334261298179626,
153
+ "step": 12
154
+ },
155
+ {
156
+ "completion_length": 160.0,
157
+ "epoch": 0.1326530612244898,
158
+ "grad_norm": 0.09611018002033234,
159
+ "kl": 0.000932489987462759,
160
+ "learning_rate": 1.3000000000000001e-05,
161
+ "loss": 0.0,
162
+ "reward": 0.2559860944747925,
163
+ "reward_std": 0.25332069396972656,
164
+ "rewards/<lambda>": 0.2559860944747925,
165
+ "step": 13
166
+ },
167
+ {
168
+ "completion_length": 160.0,
169
+ "epoch": 0.14285714285714285,
170
+ "grad_norm": 0.11913401633501053,
171
+ "kl": 0.0010431472910568118,
172
+ "learning_rate": 1.4000000000000001e-05,
173
+ "loss": 0.0,
174
+ "reward": 0.2593250274658203,
175
+ "reward_std": 0.25670933723449707,
176
+ "rewards/<lambda>": 0.2593250274658203,
177
+ "step": 14
178
+ },
179
+ {
180
+ "completion_length": 160.0,
181
+ "epoch": 0.15306122448979592,
182
+ "grad_norm": 0.09773930162191391,
183
+ "kl": 0.0009058397263288498,
184
+ "learning_rate": 1.5e-05,
185
+ "loss": 0.0,
186
+ "reward": 0.3413263261318207,
187
+ "reward_std": 0.23378777503967285,
188
+ "rewards/<lambda>": 0.3413263261318207,
189
+ "step": 15
190
+ },
191
+ {
192
+ "completion_length": 160.0,
193
+ "epoch": 0.16326530612244897,
194
+ "grad_norm": 0.10632280260324478,
195
+ "kl": 0.000995107227936387,
196
+ "learning_rate": 1.6000000000000003e-05,
197
+ "loss": 0.0,
198
+ "reward": 0.3327138423919678,
199
+ "reward_std": 0.2463734745979309,
200
+ "rewards/<lambda>": 0.3327138423919678,
201
+ "step": 16
202
+ },
203
+ {
204
+ "completion_length": 160.0,
205
+ "epoch": 0.17346938775510204,
206
+ "grad_norm": 0.0903797596693039,
207
+ "kl": 0.0008249408565461636,
208
+ "learning_rate": 1.7000000000000003e-05,
209
+ "loss": 0.0,
210
+ "reward": 0.25679031014442444,
211
+ "reward_std": 0.2034485787153244,
212
+ "rewards/<lambda>": 0.25679031014442444,
213
+ "step": 17
214
+ },
215
+ {
216
+ "completion_length": 160.0,
217
+ "epoch": 0.1836734693877551,
218
+ "grad_norm": 0.09401573985815048,
219
+ "kl": 0.0011675208806991577,
220
+ "learning_rate": 1.8e-05,
221
+ "loss": 0.0,
222
+ "reward": 0.24813005328178406,
223
+ "reward_std": 0.21580623090267181,
224
+ "rewards/<lambda>": 0.24813005328178406,
225
+ "step": 18
226
+ },
227
+ {
228
+ "completion_length": 160.0,
229
+ "epoch": 0.19387755102040816,
230
+ "grad_norm": 0.12456843256950378,
231
+ "kl": 0.0010836416622623801,
232
+ "learning_rate": 1.9e-05,
233
+ "loss": 0.0,
234
+ "reward": 0.3487837314605713,
235
+ "reward_std": 0.220088928937912,
236
+ "rewards/<lambda>": 0.3487837314605713,
237
+ "step": 19
238
+ },
239
+ {
240
+ "completion_length": 160.0,
241
+ "epoch": 0.20408163265306123,
242
+ "grad_norm": 0.0994577631354332,
243
+ "kl": 0.0012541261967271566,
244
+ "learning_rate": 2e-05,
245
+ "loss": 0.0001,
246
+ "reward": 0.2757805585861206,
247
+ "reward_std": 0.23097757995128632,
248
+ "rewards/<lambda>": 0.2757805585861206,
249
+ "step": 20
250
+ },
251
+ {
252
+ "completion_length": 160.0,
253
+ "epoch": 0.21428571428571427,
254
+ "grad_norm": 0.1026805192232132,
255
+ "kl": 0.0013662949204444885,
256
+ "learning_rate": 2.1e-05,
257
+ "loss": 0.0001,
258
+ "reward": 0.3016376793384552,
259
+ "reward_std": 0.26067519187927246,
260
+ "rewards/<lambda>": 0.3016376793384552,
261
+ "step": 21
262
+ },
263
+ {
264
+ "completion_length": 160.0,
265
+ "epoch": 0.22448979591836735,
266
+ "grad_norm": 0.11243847012519836,
267
+ "kl": 0.0012734383344650269,
268
+ "learning_rate": 2.2000000000000003e-05,
269
+ "loss": 0.0001,
270
+ "reward": 0.34152576327323914,
271
+ "reward_std": 0.24704596400260925,
272
+ "rewards/<lambda>": 0.34152576327323914,
273
+ "step": 22
274
+ },
275
+ {
276
+ "completion_length": 160.0,
277
+ "epoch": 0.23469387755102042,
278
+ "grad_norm": 0.11959439516067505,
279
+ "kl": 0.001076711225323379,
280
+ "learning_rate": 2.3000000000000003e-05,
281
+ "loss": 0.0,
282
+ "reward": 0.25360214710235596,
283
+ "reward_std": 0.24721531569957733,
284
+ "rewards/<lambda>": 0.25360214710235596,
285
+ "step": 23
286
+ },
287
+ {
288
+ "completion_length": 160.0,
289
+ "epoch": 0.24489795918367346,
290
+ "grad_norm": 0.12651875615119934,
291
+ "kl": 0.001166851376183331,
292
+ "learning_rate": 2.4e-05,
293
+ "loss": 0.0,
294
+ "reward": 0.23422789573669434,
295
+ "reward_std": 0.22290775179862976,
296
+ "rewards/<lambda>": 0.23422789573669434,
297
+ "step": 24
298
+ },
299
+ {
300
+ "completion_length": 160.0,
301
+ "epoch": 0.25510204081632654,
302
+ "grad_norm": 0.09411771595478058,
303
+ "kl": 0.0009251383016817272,
304
+ "learning_rate": 2.5e-05,
305
+ "loss": 0.0,
306
+ "reward": 0.3483571410179138,
307
+ "reward_std": 0.22439202666282654,
308
+ "rewards/<lambda>": 0.3483571410179138,
309
+ "step": 25
310
+ },
311
+ {
312
+ "completion_length": 160.0,
313
+ "epoch": 0.2653061224489796,
314
+ "grad_norm": 0.10857303440570831,
315
+ "kl": 0.0013178512454032898,
316
+ "learning_rate": 2.6000000000000002e-05,
317
+ "loss": 0.0001,
318
+ "reward": 0.27066630125045776,
319
+ "reward_std": 0.25778770446777344,
320
+ "rewards/<lambda>": 0.27066630125045776,
321
+ "step": 26
322
+ },
323
+ {
324
+ "completion_length": 160.0,
325
+ "epoch": 0.2755102040816326,
326
+ "grad_norm": 0.10724504292011261,
327
+ "kl": 0.0011154217645525932,
328
+ "learning_rate": 2.7000000000000002e-05,
329
+ "loss": 0.0,
330
+ "reward": 0.29464977979660034,
331
+ "reward_std": 0.22159242630004883,
332
+ "rewards/<lambda>": 0.29464977979660034,
333
+ "step": 27
334
+ },
335
+ {
336
+ "completion_length": 160.0,
337
+ "epoch": 0.2857142857142857,
338
+ "grad_norm": 0.14677317440509796,
339
+ "kl": 0.0010433748830109835,
340
+ "learning_rate": 2.8000000000000003e-05,
341
+ "loss": 0.0,
342
+ "reward": 0.3109304904937744,
343
+ "reward_std": 0.26592540740966797,
344
+ "rewards/<lambda>": 0.3109304904937744,
345
+ "step": 28
346
+ },
347
+ {
348
+ "completion_length": 160.0,
349
+ "epoch": 0.29591836734693877,
350
+ "grad_norm": 0.12714359164237976,
351
+ "kl": 0.001189422095194459,
352
+ "learning_rate": 2.9e-05,
353
+ "loss": 0.0,
354
+ "reward": 0.3446485698223114,
355
+ "reward_std": 0.22104832530021667,
356
+ "rewards/<lambda>": 0.3446485698223114,
357
+ "step": 29
358
+ },
359
+ {
360
+ "completion_length": 160.0,
361
+ "epoch": 0.30612244897959184,
362
+ "grad_norm": 0.093221515417099,
363
+ "kl": 0.0011938215466216207,
364
+ "learning_rate": 3e-05,
365
+ "loss": 0.0,
366
+ "reward": 0.3021507263183594,
367
+ "reward_std": 0.26753896474838257,
368
+ "rewards/<lambda>": 0.3021507263183594,
369
+ "step": 30
370
+ },
371
+ {
372
+ "completion_length": 160.0,
373
+ "epoch": 0.3163265306122449,
374
+ "grad_norm": 0.10894666612148285,
375
+ "kl": 0.0014481329126283526,
376
+ "learning_rate": 3.1e-05,
377
+ "loss": 0.0001,
378
+ "reward": 0.22669407725334167,
379
+ "reward_std": 0.21502387523651123,
380
+ "rewards/<lambda>": 0.22669407725334167,
381
+ "step": 31
382
+ },
383
+ {
384
+ "completion_length": 160.0,
385
+ "epoch": 0.32653061224489793,
386
+ "grad_norm": 0.11627420037984848,
387
+ "kl": 0.0011792851146310568,
388
+ "learning_rate": 3.2000000000000005e-05,
389
+ "loss": 0.0,
390
+ "reward": 0.2387671172618866,
391
+ "reward_std": 0.20545344054698944,
392
+ "rewards/<lambda>": 0.2387671172618866,
393
+ "step": 32
394
+ },
395
+ {
396
+ "completion_length": 160.0,
397
+ "epoch": 0.336734693877551,
398
+ "grad_norm": 0.1079491674900055,
399
+ "kl": 0.0013044481165707111,
400
+ "learning_rate": 3.3e-05,
401
+ "loss": 0.0001,
402
+ "reward": 0.2432921826839447,
403
+ "reward_std": 0.2530010938644409,
404
+ "rewards/<lambda>": 0.2432921826839447,
405
+ "step": 33
406
+ },
407
+ {
408
+ "completion_length": 160.0,
409
+ "epoch": 0.3469387755102041,
410
+ "grad_norm": 0.09688923507928848,
411
+ "kl": 0.0012661240762099624,
412
+ "learning_rate": 3.4000000000000007e-05,
413
+ "loss": 0.0001,
414
+ "reward": 0.32528483867645264,
415
+ "reward_std": 0.2360536754131317,
416
+ "rewards/<lambda>": 0.32528483867645264,
417
+ "step": 34
418
+ },
419
+ {
420
+ "completion_length": 160.0,
421
+ "epoch": 0.35714285714285715,
422
+ "grad_norm": 0.07901012152433395,
423
+ "kl": 0.001169042894616723,
424
+ "learning_rate": 3.5e-05,
425
+ "loss": 0.0,
426
+ "reward": 0.2810063362121582,
427
+ "reward_std": 0.22572952508926392,
428
+ "rewards/<lambda>": 0.2810063362121582,
429
+ "step": 35
430
+ },
431
+ {
432
+ "completion_length": 160.0,
433
+ "epoch": 0.3673469387755102,
434
+ "grad_norm": 0.12785927951335907,
435
+ "kl": 0.0014555864036083221,
436
+ "learning_rate": 3.6e-05,
437
+ "loss": 0.0001,
438
+ "reward": 0.2540737986564636,
439
+ "reward_std": 0.24591577053070068,
440
+ "rewards/<lambda>": 0.2540737986564636,
441
+ "step": 36
442
+ },
443
+ {
444
+ "completion_length": 160.0,
445
+ "epoch": 0.37755102040816324,
446
+ "grad_norm": 0.08787187188863754,
447
+ "kl": 0.001047058729454875,
448
+ "learning_rate": 3.7e-05,
449
+ "loss": 0.0,
450
+ "reward": 0.24186044931411743,
451
+ "reward_std": 0.2187865972518921,
452
+ "rewards/<lambda>": 0.24186044931411743,
453
+ "step": 37
454
+ },
455
+ {
456
+ "completion_length": 160.0,
457
+ "epoch": 0.3877551020408163,
458
+ "grad_norm": 0.102202869951725,
459
+ "kl": 0.0013880159240216017,
460
+ "learning_rate": 3.8e-05,
461
+ "loss": 0.0001,
462
+ "reward": 0.2575429379940033,
463
+ "reward_std": 0.2186940610408783,
464
+ "rewards/<lambda>": 0.2575429379940033,
465
+ "step": 38
466
+ },
467
+ {
468
+ "completion_length": 160.0,
469
+ "epoch": 0.3979591836734694,
470
+ "grad_norm": 0.08456045389175415,
471
+ "kl": 0.0011803264496847987,
472
+ "learning_rate": 3.9000000000000006e-05,
473
+ "loss": 0.0,
474
+ "reward": 0.31993257999420166,
475
+ "reward_std": 0.22598499059677124,
476
+ "rewards/<lambda>": 0.31993257999420166,
477
+ "step": 39
478
+ },
479
+ {
480
+ "completion_length": 160.0,
481
+ "epoch": 0.40816326530612246,
482
+ "grad_norm": 0.10486605018377304,
483
+ "kl": 0.0018042891751974821,
484
+ "learning_rate": 4e-05,
485
+ "loss": 0.0001,
486
+ "reward": 0.29636475443840027,
487
+ "reward_std": 0.24350598454475403,
488
+ "rewards/<lambda>": 0.29636475443840027,
489
+ "step": 40
490
+ },
491
+ {
492
+ "completion_length": 160.0,
493
+ "epoch": 0.41836734693877553,
494
+ "grad_norm": 0.0880642980337143,
495
+ "kl": 0.001442858949303627,
496
+ "learning_rate": 4.1e-05,
497
+ "loss": 0.0001,
498
+ "reward": 0.41248711943626404,
499
+ "reward_std": 0.23531243205070496,
500
+ "rewards/<lambda>": 0.41248711943626404,
501
+ "step": 41
502
+ },
503
+ {
504
+ "completion_length": 160.0,
505
+ "epoch": 0.42857142857142855,
506
+ "grad_norm": 0.11210600286722183,
507
+ "kl": 0.0027525494806468487,
508
+ "learning_rate": 4.2e-05,
509
+ "loss": 0.0001,
510
+ "reward": 0.33057302236557007,
511
+ "reward_std": 0.2597821354866028,
512
+ "rewards/<lambda>": 0.33057302236557007,
513
+ "step": 42
514
+ },
515
+ {
516
+ "completion_length": 160.0,
517
+ "epoch": 0.4387755102040816,
518
+ "grad_norm": 0.1131657138466835,
519
+ "kl": 0.0013009419199079275,
520
+ "learning_rate": 4.3e-05,
521
+ "loss": 0.0001,
522
+ "reward": 0.3796292245388031,
523
+ "reward_std": 0.23504015803337097,
524
+ "rewards/<lambda>": 0.3796292245388031,
525
+ "step": 43
526
+ },
527
+ {
528
+ "completion_length": 160.0,
529
+ "epoch": 0.4489795918367347,
530
+ "grad_norm": 0.09555080533027649,
531
+ "kl": 0.002401509787887335,
532
+ "learning_rate": 4.4000000000000006e-05,
533
+ "loss": 0.0001,
534
+ "reward": 0.28863847255706787,
535
+ "reward_std": 0.2549838721752167,
536
+ "rewards/<lambda>": 0.28863847255706787,
537
+ "step": 44
538
+ },
539
+ {
540
+ "completion_length": 160.0,
541
+ "epoch": 0.45918367346938777,
542
+ "grad_norm": 0.12921357154846191,
543
+ "kl": 0.003915494307875633,
544
+ "learning_rate": 4.5e-05,
545
+ "loss": 0.0002,
546
+ "reward": 0.25585973262786865,
547
+ "reward_std": 0.2098313271999359,
548
+ "rewards/<lambda>": 0.25585973262786865,
549
+ "step": 45
550
+ },
551
+ {
552
+ "completion_length": 160.0,
553
+ "epoch": 0.46938775510204084,
554
+ "grad_norm": 0.10449738055467606,
555
+ "kl": 0.0029986370354890823,
556
+ "learning_rate": 4.600000000000001e-05,
557
+ "loss": 0.0001,
558
+ "reward": 0.24023020267486572,
559
+ "reward_std": 0.21866671741008759,
560
+ "rewards/<lambda>": 0.24023020267486572,
561
+ "step": 46
562
+ },
563
+ {
564
+ "completion_length": 160.0,
565
+ "epoch": 0.47959183673469385,
566
+ "grad_norm": 0.11932526528835297,
567
+ "kl": 0.0029086011927574873,
568
+ "learning_rate": 4.7e-05,
569
+ "loss": 0.0001,
570
+ "reward": 0.35698994994163513,
571
+ "reward_std": 0.244903564453125,
572
+ "rewards/<lambda>": 0.35698994994163513,
573
+ "step": 47
574
+ },
575
+ {
576
+ "completion_length": 160.0,
577
+ "epoch": 0.4897959183673469,
578
+ "grad_norm": 0.11577532440423965,
579
+ "kl": 0.0025195125490427017,
580
+ "learning_rate": 4.8e-05,
581
+ "loss": 0.0001,
582
+ "reward": 0.3163990378379822,
583
+ "reward_std": 0.25772640109062195,
584
+ "rewards/<lambda>": 0.3163990378379822,
585
+ "step": 48
586
+ },
587
+ {
588
+ "completion_length": 160.0,
589
+ "epoch": 0.5,
590
+ "grad_norm": 0.12613017857074738,
591
+ "kl": 0.002560165012255311,
592
+ "learning_rate": 4.9e-05,
593
+ "loss": 0.0001,
594
+ "reward": 0.28095588088035583,
595
+ "reward_std": 0.2137780636548996,
596
+ "rewards/<lambda>": 0.28095588088035583,
597
+ "step": 49
598
+ },
599
+ {
600
+ "completion_length": 160.0,
601
+ "epoch": 0.5102040816326531,
602
+ "grad_norm": 0.11029908806085587,
603
+ "kl": 0.003053056076169014,
604
+ "learning_rate": 5e-05,
605
+ "loss": 0.0001,
606
+ "reward": 0.2593611478805542,
607
+ "reward_std": 0.17565123736858368,
608
+ "rewards/<lambda>": 0.2593611478805542,
609
+ "step": 50
610
+ },
611
+ {
612
+ "completion_length": 160.0,
613
+ "epoch": 0.5204081632653061,
614
+ "grad_norm": 0.10181179642677307,
615
+ "kl": 0.0032034586183726788,
616
+ "learning_rate": 5.1000000000000006e-05,
617
+ "loss": 0.0001,
618
+ "reward": 0.3496597707271576,
619
+ "reward_std": 0.23656174540519714,
620
+ "rewards/<lambda>": 0.3496597707271576,
621
+ "step": 51
622
+ },
623
+ {
624
+ "completion_length": 160.0,
625
+ "epoch": 0.5306122448979592,
626
+ "grad_norm": 0.11612030863761902,
627
+ "kl": 0.0029043052345514297,
628
+ "learning_rate": 5.2000000000000004e-05,
629
+ "loss": 0.0001,
630
+ "reward": 0.31435415148735046,
631
+ "reward_std": 0.26738691329956055,
632
+ "rewards/<lambda>": 0.31435415148735046,
633
+ "step": 52
634
+ },
635
+ {
636
+ "completion_length": 160.0,
637
+ "epoch": 0.5408163265306123,
638
+ "grad_norm": 0.11753087490797043,
639
+ "kl": 0.004537411965429783,
640
+ "learning_rate": 5.300000000000001e-05,
641
+ "loss": 0.0002,
642
+ "reward": 0.27415716648101807,
643
+ "reward_std": 0.2496672123670578,
644
+ "rewards/<lambda>": 0.27415716648101807,
645
+ "step": 53
646
+ },
647
+ {
648
+ "completion_length": 160.0,
649
+ "epoch": 0.5510204081632653,
650
+ "grad_norm": 0.09143570065498352,
651
+ "kl": 0.0020023572724312544,
652
+ "learning_rate": 5.4000000000000005e-05,
653
+ "loss": 0.0001,
654
+ "reward": 0.27505844831466675,
655
+ "reward_std": 0.2481660097837448,
656
+ "rewards/<lambda>": 0.27505844831466675,
657
+ "step": 54
658
+ },
659
+ {
660
+ "completion_length": 160.0,
661
+ "epoch": 0.5612244897959183,
662
+ "grad_norm": 0.11042241752147675,
663
+ "kl": 0.002455132082104683,
664
+ "learning_rate": 5.500000000000001e-05,
665
+ "loss": 0.0001,
666
+ "reward": 0.2277480959892273,
667
+ "reward_std": 0.2573385238647461,
668
+ "rewards/<lambda>": 0.2277480959892273,
669
+ "step": 55
670
+ },
671
+ {
672
+ "completion_length": 160.0,
673
+ "epoch": 0.5714285714285714,
674
+ "grad_norm": 0.1062241867184639,
675
+ "kl": 0.003958327695727348,
676
+ "learning_rate": 5.6000000000000006e-05,
677
+ "loss": 0.0002,
678
+ "reward": 0.3246029019355774,
679
+ "reward_std": 0.25107985734939575,
680
+ "rewards/<lambda>": 0.3246029019355774,
681
+ "step": 56
682
+ },
683
+ {
684
+ "completion_length": 160.0,
685
+ "epoch": 0.5816326530612245,
686
+ "grad_norm": 0.09364596009254456,
687
+ "kl": 0.0028939968906342983,
688
+ "learning_rate": 5.6999999999999996e-05,
689
+ "loss": 0.0001,
690
+ "reward": 0.33692467212677,
691
+ "reward_std": 0.2620271146297455,
692
+ "rewards/<lambda>": 0.33692467212677,
693
+ "step": 57
694
+ },
695
+ {
696
+ "completion_length": 160.0,
697
+ "epoch": 0.5918367346938775,
698
+ "grad_norm": 0.10371523350477219,
699
+ "kl": 0.0031635533086955547,
700
+ "learning_rate": 5.8e-05,
701
+ "loss": 0.0001,
702
+ "reward": 0.26338717341423035,
703
+ "reward_std": 0.25605374574661255,
704
+ "rewards/<lambda>": 0.26338717341423035,
705
+ "step": 58
706
+ },
707
+ {
708
+ "completion_length": 160.0,
709
+ "epoch": 0.6020408163265306,
710
+ "grad_norm": 0.119356170296669,
711
+ "kl": 0.004045659676194191,
712
+ "learning_rate": 5.9e-05,
713
+ "loss": 0.0002,
714
+ "reward": 0.26655131578445435,
715
+ "reward_std": 0.23418131470680237,
716
+ "rewards/<lambda>": 0.26655131578445435,
717
+ "step": 59
718
+ },
719
+ {
720
+ "completion_length": 160.0,
721
+ "epoch": 0.6122448979591837,
722
+ "grad_norm": 0.14208924770355225,
723
+ "kl": 0.005053609609603882,
724
+ "learning_rate": 6e-05,
725
+ "loss": 0.0002,
726
+ "reward": 0.3059399425983429,
727
+ "reward_std": 0.2509281635284424,
728
+ "rewards/<lambda>": 0.3059399425983429,
729
+ "step": 60
730
+ },
731
+ {
732
+ "completion_length": 160.0,
733
+ "epoch": 0.6224489795918368,
734
+ "grad_norm": 0.19251689314842224,
735
+ "kl": 0.005135521292686462,
736
+ "learning_rate": 6.1e-05,
737
+ "loss": 0.0002,
738
+ "reward": 0.3289321959018707,
739
+ "reward_std": 0.25894391536712646,
740
+ "rewards/<lambda>": 0.3289321959018707,
741
+ "step": 61
742
+ },
743
+ {
744
+ "completion_length": 160.0,
745
+ "epoch": 0.6326530612244898,
746
+ "grad_norm": 0.10605209320783615,
747
+ "kl": 0.004940195474773645,
748
+ "learning_rate": 6.2e-05,
749
+ "loss": 0.0002,
750
+ "reward": 0.3096313178539276,
751
+ "reward_std": 0.25553008913993835,
752
+ "rewards/<lambda>": 0.3096313178539276,
753
+ "step": 62
754
+ },
755
+ {
756
+ "completion_length": 160.0,
757
+ "epoch": 0.6428571428571429,
758
+ "grad_norm": 0.1347755789756775,
759
+ "kl": 0.007013984024524689,
760
+ "learning_rate": 6.3e-05,
761
+ "loss": 0.0003,
762
+ "reward": 0.3279077112674713,
763
+ "reward_std": 0.24912631511688232,
764
+ "rewards/<lambda>": 0.3279077112674713,
765
+ "step": 63
766
+ },
767
+ {
768
+ "completion_length": 160.0,
769
+ "epoch": 0.6530612244897959,
770
+ "grad_norm": 0.11074844002723694,
771
+ "kl": 0.005811232142150402,
772
+ "learning_rate": 6.400000000000001e-05,
773
+ "loss": 0.0002,
774
+ "reward": 0.26557987928390503,
775
+ "reward_std": 0.2595665454864502,
776
+ "rewards/<lambda>": 0.26557987928390503,
777
+ "step": 64
778
+ },
779
+ {
780
+ "completion_length": 160.0,
781
+ "epoch": 0.6632653061224489,
782
+ "grad_norm": 0.08949097245931625,
783
+ "kl": 0.0062391310930252075,
784
+ "learning_rate": 6.500000000000001e-05,
785
+ "loss": 0.0002,
786
+ "reward": 0.30340054631233215,
787
+ "reward_std": 0.23612141609191895,
788
+ "rewards/<lambda>": 0.30340054631233215,
789
+ "step": 65
790
+ },
791
+ {
792
+ "completion_length": 160.0,
793
+ "epoch": 0.673469387755102,
794
+ "grad_norm": 0.09197568893432617,
795
+ "kl": 0.006357334554195404,
796
+ "learning_rate": 6.6e-05,
797
+ "loss": 0.0003,
798
+ "reward": 0.305142879486084,
799
+ "reward_std": 0.25195544958114624,
800
+ "rewards/<lambda>": 0.305142879486084,
801
+ "step": 66
802
+ },
803
+ {
804
+ "completion_length": 160.0,
805
+ "epoch": 0.6836734693877551,
806
+ "grad_norm": 0.11795815080404282,
807
+ "kl": 0.006672342773526907,
808
+ "learning_rate": 6.7e-05,
809
+ "loss": 0.0003,
810
+ "reward": 0.2330521047115326,
811
+ "reward_std": 0.24398502707481384,
812
+ "rewards/<lambda>": 0.2330521047115326,
813
+ "step": 67
814
+ },
815
+ {
816
+ "completion_length": 160.0,
817
+ "epoch": 0.6938775510204082,
818
+ "grad_norm": 0.1380680352449417,
819
+ "kl": 0.008953109383583069,
820
+ "learning_rate": 6.800000000000001e-05,
821
+ "loss": 0.0004,
822
+ "reward": 0.3820633292198181,
823
+ "reward_std": 0.24111545085906982,
824
+ "rewards/<lambda>": 0.3820633292198181,
825
+ "step": 68
826
+ },
827
+ {
828
+ "completion_length": 160.0,
829
+ "epoch": 0.7040816326530612,
830
+ "grad_norm": 0.10920781642198563,
831
+ "kl": 0.006580730434507132,
832
+ "learning_rate": 6.9e-05,
833
+ "loss": 0.0003,
834
+ "reward": 0.30754148960113525,
835
+ "reward_std": 0.22947274148464203,
836
+ "rewards/<lambda>": 0.30754148960113525,
837
+ "step": 69
838
+ },
839
+ {
840
+ "completion_length": 160.0,
841
+ "epoch": 0.7142857142857143,
842
+ "grad_norm": 0.11049173772335052,
843
+ "kl": 0.006469545420259237,
844
+ "learning_rate": 7e-05,
845
+ "loss": 0.0003,
846
+ "reward": 0.2494296282529831,
847
+ "reward_std": 0.2338804006576538,
848
+ "rewards/<lambda>": 0.2494296282529831,
849
+ "step": 70
850
+ },
851
+ {
852
+ "completion_length": 160.0,
853
+ "epoch": 0.7244897959183674,
854
+ "grad_norm": 0.12242847681045532,
855
+ "kl": 0.0065796151757240295,
856
+ "learning_rate": 7.1e-05,
857
+ "loss": 0.0003,
858
+ "reward": 0.35586172342300415,
859
+ "reward_std": 0.22659151256084442,
860
+ "rewards/<lambda>": 0.35586172342300415,
861
+ "step": 71
862
+ },
863
+ {
864
+ "completion_length": 160.0,
865
+ "epoch": 0.7346938775510204,
866
+ "grad_norm": 0.10463358461856842,
867
+ "kl": 0.008036931976675987,
868
+ "learning_rate": 7.2e-05,
869
+ "loss": 0.0003,
870
+ "reward": 0.3306816518306732,
871
+ "reward_std": 0.21499347686767578,
872
+ "rewards/<lambda>": 0.3306816518306732,
873
+ "step": 72
874
+ },
875
+ {
876
+ "completion_length": 160.0,
877
+ "epoch": 0.7448979591836735,
878
+ "grad_norm": 0.10549326241016388,
879
+ "kl": 0.005967782810330391,
880
+ "learning_rate": 7.3e-05,
881
+ "loss": 0.0002,
882
+ "reward": 0.26001977920532227,
883
+ "reward_std": 0.20824775099754333,
884
+ "rewards/<lambda>": 0.26001977920532227,
885
+ "step": 73
886
+ },
887
+ {
888
+ "completion_length": 160.0,
889
+ "epoch": 0.7551020408163265,
890
+ "grad_norm": 0.10788781940937042,
891
+ "kl": 0.004906760528683662,
892
+ "learning_rate": 7.4e-05,
893
+ "loss": 0.0002,
894
+ "reward": 0.2874922752380371,
895
+ "reward_std": 0.24835163354873657,
896
+ "rewards/<lambda>": 0.2874922752380371,
897
+ "step": 74
898
+ },
899
+ {
900
+ "completion_length": 160.0,
901
+ "epoch": 0.7653061224489796,
902
+ "grad_norm": 0.09784362465143204,
903
+ "kl": 0.005803011357784271,
904
+ "learning_rate": 7.500000000000001e-05,
905
+ "loss": 0.0002,
906
+ "reward": 0.2684209942817688,
907
+ "reward_std": 0.2644413113594055,
908
+ "rewards/<lambda>": 0.2684209942817688,
909
+ "step": 75
910
+ },
911
+ {
912
+ "completion_length": 160.0,
913
+ "epoch": 0.7755102040816326,
914
+ "grad_norm": 0.16001495718955994,
915
+ "kl": 0.006752543151378632,
916
+ "learning_rate": 7.6e-05,
917
+ "loss": 0.0003,
918
+ "reward": 0.3604978919029236,
919
+ "reward_std": 0.21626180410385132,
920
+ "rewards/<lambda>": 0.3604978919029236,
921
+ "step": 76
922
+ },
923
+ {
924
+ "completion_length": 160.0,
925
+ "epoch": 0.7857142857142857,
926
+ "grad_norm": 0.13324351608753204,
927
+ "kl": 0.008819150738418102,
928
+ "learning_rate": 7.7e-05,
929
+ "loss": 0.0004,
930
+ "reward": 0.3122796416282654,
931
+ "reward_std": 0.23514878749847412,
932
+ "rewards/<lambda>": 0.3122796416282654,
933
+ "step": 77
934
+ },
935
+ {
936
+ "completion_length": 160.0,
937
+ "epoch": 0.7959183673469388,
938
+ "grad_norm": 0.11151094734668732,
939
+ "kl": 0.0060831112787127495,
940
+ "learning_rate": 7.800000000000001e-05,
941
+ "loss": 0.0002,
942
+ "reward": 0.3716966509819031,
943
+ "reward_std": 0.25390562415122986,
944
+ "rewards/<lambda>": 0.3716966509819031,
945
+ "step": 78
946
+ },
947
+ {
948
+ "completion_length": 160.0,
949
+ "epoch": 0.8061224489795918,
950
+ "grad_norm": 0.1258833110332489,
951
+ "kl": 0.00652284873649478,
952
+ "learning_rate": 7.900000000000001e-05,
953
+ "loss": 0.0003,
954
+ "reward": 0.31094950437545776,
955
+ "reward_std": 0.24983155727386475,
956
+ "rewards/<lambda>": 0.31094950437545776,
957
+ "step": 79
958
+ },
959
+ {
960
+ "completion_length": 160.0,
961
+ "epoch": 0.8163265306122449,
962
+ "grad_norm": 0.09824781864881516,
963
+ "kl": 0.007383039221167564,
964
+ "learning_rate": 8e-05,
965
+ "loss": 0.0003,
966
+ "reward": 0.3597716689109802,
967
+ "reward_std": 0.24361717700958252,
968
+ "rewards/<lambda>": 0.3597716689109802,
969
+ "step": 80
970
+ },
971
+ {
972
+ "completion_length": 160.0,
973
+ "epoch": 0.826530612244898,
974
+ "grad_norm": 0.08224570006132126,
975
+ "kl": 0.005881953053176403,
976
+ "learning_rate": 8.1e-05,
977
+ "loss": 0.0002,
978
+ "reward": 0.27791696786880493,
979
+ "reward_std": 0.2182818502187729,
980
+ "rewards/<lambda>": 0.27791696786880493,
981
+ "step": 81
982
+ },
983
+ {
984
+ "completion_length": 160.0,
985
+ "epoch": 0.8367346938775511,
986
+ "grad_norm": 0.12581421434879303,
987
+ "kl": 0.006681859493255615,
988
+ "learning_rate": 8.2e-05,
989
+ "loss": 0.0003,
990
+ "reward": 0.3261301517486572,
991
+ "reward_std": 0.2615125775337219,
992
+ "rewards/<lambda>": 0.3261301517486572,
993
+ "step": 82
994
+ },
995
+ {
996
+ "completion_length": 160.0,
997
+ "epoch": 0.8469387755102041,
998
+ "grad_norm": 0.10322140902280807,
999
+ "kl": 0.006312561221420765,
1000
+ "learning_rate": 8.3e-05,
1001
+ "loss": 0.0003,
1002
+ "reward": 0.3625139594078064,
1003
+ "reward_std": 0.24003905057907104,
1004
+ "rewards/<lambda>": 0.3625139594078064,
1005
+ "step": 83
1006
+ },
1007
+ {
1008
+ "completion_length": 160.0,
1009
+ "epoch": 0.8571428571428571,
1010
+ "grad_norm": 0.0966828241944313,
1011
+ "kl": 0.0061577907763421535,
1012
+ "learning_rate": 8.4e-05,
1013
+ "loss": 0.0002,
1014
+ "reward": 0.27837446331977844,
1015
+ "reward_std": 0.24895024299621582,
1016
+ "rewards/<lambda>": 0.27837446331977844,
1017
+ "step": 84
1018
+ },
1019
+ {
1020
+ "completion_length": 160.0,
1021
+ "epoch": 0.8673469387755102,
1022
+ "grad_norm": 0.09297024458646774,
1023
+ "kl": 0.005875007715076208,
1024
+ "learning_rate": 8.5e-05,
1025
+ "loss": 0.0002,
1026
+ "reward": 0.3240439295768738,
1027
+ "reward_std": 0.2554982602596283,
1028
+ "rewards/<lambda>": 0.3240439295768738,
1029
+ "step": 85
1030
+ },
1031
+ {
1032
+ "completion_length": 160.0,
1033
+ "epoch": 0.8775510204081632,
1034
+ "grad_norm": 0.12204565852880478,
1035
+ "kl": 0.006396137177944183,
1036
+ "learning_rate": 8.6e-05,
1037
+ "loss": 0.0003,
1038
+ "reward": 0.24172773957252502,
1039
+ "reward_std": 0.2518821060657501,
1040
+ "rewards/<lambda>": 0.24172773957252502,
1041
+ "step": 86
1042
+ },
1043
+ {
1044
+ "completion_length": 160.0,
1045
+ "epoch": 0.8877551020408163,
1046
+ "grad_norm": 0.08884488046169281,
1047
+ "kl": 0.00668664975091815,
1048
+ "learning_rate": 8.7e-05,
1049
+ "loss": 0.0003,
1050
+ "reward": 0.28681838512420654,
1051
+ "reward_std": 0.25753265619277954,
1052
+ "rewards/<lambda>": 0.28681838512420654,
1053
+ "step": 87
1054
+ },
1055
+ {
1056
+ "completion_length": 160.0,
1057
+ "epoch": 0.8979591836734694,
1058
+ "grad_norm": 0.08016186207532883,
1059
+ "kl": 0.006137486081570387,
1060
+ "learning_rate": 8.800000000000001e-05,
1061
+ "loss": 0.0002,
1062
+ "reward": 0.2647368907928467,
1063
+ "reward_std": 0.24885067343711853,
1064
+ "rewards/<lambda>": 0.2647368907928467,
1065
+ "step": 88
1066
+ },
1067
+ {
1068
+ "completion_length": 160.0,
1069
+ "epoch": 0.9081632653061225,
1070
+ "grad_norm": 0.10147328674793243,
1071
+ "kl": 0.00917094200849533,
1072
+ "learning_rate": 8.900000000000001e-05,
1073
+ "loss": 0.0004,
1074
+ "reward": 0.29730311036109924,
1075
+ "reward_std": 0.2591564953327179,
1076
+ "rewards/<lambda>": 0.29730311036109924,
1077
+ "step": 89
1078
+ },
1079
+ {
1080
+ "completion_length": 160.0,
1081
+ "epoch": 0.9183673469387755,
1082
+ "grad_norm": 0.12398728728294373,
1083
+ "kl": 0.007823077961802483,
1084
+ "learning_rate": 9e-05,
1085
+ "loss": 0.0003,
1086
+ "reward": 0.2589433193206787,
1087
+ "reward_std": 0.24704992771148682,
1088
+ "rewards/<lambda>": 0.2589433193206787,
1089
+ "step": 90
1090
+ },
1091
+ {
1092
+ "completion_length": 160.0,
1093
+ "epoch": 0.9285714285714286,
1094
+ "grad_norm": 0.1265306919813156,
1095
+ "kl": 0.009746653027832508,
1096
+ "learning_rate": 9.1e-05,
1097
+ "loss": 0.0004,
1098
+ "reward": 0.2624666094779968,
1099
+ "reward_std": 0.26040005683898926,
1100
+ "rewards/<lambda>": 0.2624666094779968,
1101
+ "step": 91
1102
+ },
1103
+ {
1104
+ "completion_length": 160.0,
1105
+ "epoch": 0.9387755102040817,
1106
+ "grad_norm": 0.09663106501102448,
1107
+ "kl": 0.009732738137245178,
1108
+ "learning_rate": 9.200000000000001e-05,
1109
+ "loss": 0.0004,
1110
+ "reward": 0.31370311975479126,
1111
+ "reward_std": 0.2633877992630005,
1112
+ "rewards/<lambda>": 0.31370311975479126,
1113
+ "step": 92
1114
+ },
1115
+ {
1116
+ "completion_length": 160.0,
1117
+ "epoch": 0.9489795918367347,
1118
+ "grad_norm": 0.12955360114574432,
1119
+ "kl": 0.011157519184052944,
1120
+ "learning_rate": 9.300000000000001e-05,
1121
+ "loss": 0.0004,
1122
+ "reward": 0.3296325206756592,
1123
+ "reward_std": 0.26135072112083435,
1124
+ "rewards/<lambda>": 0.3296325206756592,
1125
+ "step": 93
1126
+ },
1127
+ {
1128
+ "completion_length": 160.0,
1129
+ "epoch": 0.9591836734693877,
1130
+ "grad_norm": 0.07906248420476913,
1131
+ "kl": 0.00876379944384098,
1132
+ "learning_rate": 9.4e-05,
1133
+ "loss": 0.0004,
1134
+ "reward": 0.38581177592277527,
1135
+ "reward_std": 0.21951782703399658,
1136
+ "rewards/<lambda>": 0.38581177592277527,
1137
+ "step": 94
1138
+ },
1139
+ {
1140
+ "completion_length": 160.0,
1141
+ "epoch": 0.9693877551020408,
1142
+ "grad_norm": 0.10264045745134354,
1143
+ "kl": 0.013823926448822021,
1144
+ "learning_rate": 9.5e-05,
1145
+ "loss": 0.0006,
1146
+ "reward": 0.37457725405693054,
1147
+ "reward_std": 0.22653540968894958,
1148
+ "rewards/<lambda>": 0.37457725405693054,
1149
+ "step": 95
1150
+ },
1151
+ {
1152
+ "completion_length": 160.0,
1153
+ "epoch": 0.9795918367346939,
1154
+ "grad_norm": 0.09052637219429016,
1155
+ "kl": 0.011999586597084999,
1156
+ "learning_rate": 9.6e-05,
1157
+ "loss": 0.0005,
1158
+ "reward": 0.3207855224609375,
1159
+ "reward_std": 0.23623248934745789,
1160
+ "rewards/<lambda>": 0.3207855224609375,
1161
+ "step": 96
1162
+ },
1163
+ {
1164
+ "completion_length": 160.0,
1165
+ "epoch": 0.9897959183673469,
1166
+ "grad_norm": 0.11846129596233368,
1167
+ "kl": 0.011448493227362633,
1168
+ "learning_rate": 9.7e-05,
1169
+ "loss": 0.0005,
1170
+ "reward": 0.32892781496047974,
1171
+ "reward_std": 0.23297683894634247,
1172
+ "rewards/<lambda>": 0.32892781496047974,
1173
+ "step": 97
1174
+ },
1175
+ {
1176
+ "completion_length": 160.0,
1177
+ "epoch": 1.0,
1178
+ "grad_norm": 0.09638947248458862,
1179
+ "kl": 0.010949314571917057,
1180
+ "learning_rate": 9.8e-05,
1181
+ "loss": 0.0004,
1182
+ "reward": 0.4758386015892029,
1183
+ "reward_std": 0.2482748031616211,
1184
+ "rewards/<lambda>": 0.4758386015892029,
1185
+ "step": 98
1186
+ },
1187
+ {
1188
+ "completion_length": 160.0,
1189
+ "epoch": 1.010204081632653,
1190
+ "grad_norm": 0.09481366723775864,
1191
+ "kl": 0.01085271593183279,
1192
+ "learning_rate": 9.900000000000001e-05,
1193
+ "loss": 0.0004,
1194
+ "reward": 0.3523382544517517,
1195
+ "reward_std": 0.2240390032529831,
1196
+ "rewards/<lambda>": 0.3523382544517517,
1197
+ "step": 99
1198
+ },
1199
+ {
1200
+ "completion_length": 160.0,
1201
+ "epoch": 1.0204081632653061,
1202
+ "grad_norm": 0.09950532019138336,
1203
+ "kl": 0.012218523770570755,
1204
+ "learning_rate": 0.0001,
1205
+ "loss": 0.0005,
1206
+ "reward": 0.28119099140167236,
1207
+ "reward_std": 0.22250661253929138,
1208
+ "rewards/<lambda>": 0.28119099140167236,
1209
+ "step": 100
1210
+ },
1211
+ {
1212
+ "completion_length": 160.0,
1213
+ "epoch": 1.030612244897959,
1214
+ "grad_norm": 0.07458172738552094,
1215
+ "kl": 0.009257722645998001,
1216
+ "learning_rate": 9.999344418328162e-05,
1217
+ "loss": 0.0004,
1218
+ "reward": 0.33414292335510254,
1219
+ "reward_std": 0.21670334041118622,
1220
+ "rewards/<lambda>": 0.33414292335510254,
1221
+ "step": 101
1222
+ },
1223
+ {
1224
+ "completion_length": 160.0,
1225
+ "epoch": 1.0408163265306123,
1226
+ "grad_norm": 0.08385725319385529,
1227
+ "kl": 0.011008420959115028,
1228
+ "learning_rate": 9.997377845227576e-05,
1229
+ "loss": 0.0004,
1230
+ "reward": 0.3389889895915985,
1231
+ "reward_std": 0.24514511227607727,
1232
+ "rewards/<lambda>": 0.3389889895915985,
1233
+ "step": 102
1234
+ },
1235
+ {
1236
+ "completion_length": 160.0,
1237
+ "epoch": 1.0510204081632653,
1238
+ "grad_norm": 0.10061275213956833,
1239
+ "kl": 0.014137129299342632,
1240
+ "learning_rate": 9.994100796397954e-05,
1241
+ "loss": 0.0006,
1242
+ "reward": 0.4110638499259949,
1243
+ "reward_std": 0.22362101078033447,
1244
+ "rewards/<lambda>": 0.4110638499259949,
1245
+ "step": 103
1246
+ },
1247
+ {
1248
+ "completion_length": 160.0,
1249
+ "epoch": 1.0612244897959184,
1250
+ "grad_norm": 0.10247852653265,
1251
+ "kl": 0.015538258478045464,
1252
+ "learning_rate": 9.989514131188559e-05,
1253
+ "loss": 0.0006,
1254
+ "reward": 0.3177061676979065,
1255
+ "reward_std": 0.26253318786621094,
1256
+ "rewards/<lambda>": 0.3177061676979065,
1257
+ "step": 104
1258
+ },
1259
+ {
1260
+ "completion_length": 160.0,
1261
+ "epoch": 1.0714285714285714,
1262
+ "grad_norm": 0.11179685592651367,
1263
+ "kl": 0.018445193767547607,
1264
+ "learning_rate": 9.983619052372848e-05,
1265
+ "loss": 0.0007,
1266
+ "reward": 0.2772364020347595,
1267
+ "reward_std": 0.25032472610473633,
1268
+ "rewards/<lambda>": 0.2772364020347595,
1269
+ "step": 105
1270
+ },
1271
+ {
1272
+ "completion_length": 160.0,
1273
+ "epoch": 1.0816326530612246,
1274
+ "grad_norm": 0.09749067574739456,
1275
+ "kl": 0.01747281290590763,
1276
+ "learning_rate": 9.97641710583307e-05,
1277
+ "loss": 0.0007,
1278
+ "reward": 0.3064037561416626,
1279
+ "reward_std": 0.255950927734375,
1280
+ "rewards/<lambda>": 0.3064037561416626,
1281
+ "step": 106
1282
+ },
1283
+ {
1284
+ "completion_length": 160.0,
1285
+ "epoch": 1.0918367346938775,
1286
+ "grad_norm": 0.08926989883184433,
1287
+ "kl": 0.018639955669641495,
1288
+ "learning_rate": 9.967910180154889e-05,
1289
+ "loss": 0.0007,
1290
+ "reward": 0.33175772428512573,
1291
+ "reward_std": 0.2530345320701599,
1292
+ "rewards/<lambda>": 0.33175772428512573,
1293
+ "step": 107
1294
+ },
1295
+ {
1296
+ "completion_length": 160.0,
1297
+ "epoch": 1.1020408163265305,
1298
+ "grad_norm": 0.10069949179887772,
1299
+ "kl": 0.017288221046328545,
1300
+ "learning_rate": 9.958100506132127e-05,
1301
+ "loss": 0.0007,
1302
+ "reward": 0.2976396083831787,
1303
+ "reward_std": 0.2146736979484558,
1304
+ "rewards/<lambda>": 0.2976396083831787,
1305
+ "step": 108
1306
+ },
1307
+ {
1308
+ "completion_length": 160.0,
1309
+ "epoch": 1.1122448979591837,
1310
+ "grad_norm": 0.08820276707410812,
1311
+ "kl": 0.013430261984467506,
1312
+ "learning_rate": 9.946990656181781e-05,
1313
+ "loss": 0.0005,
1314
+ "reward": 0.27721720933914185,
1315
+ "reward_std": 0.2337382286787033,
1316
+ "rewards/<lambda>": 0.27721720933914185,
1317
+ "step": 109
1318
+ },
1319
+ {
1320
+ "completion_length": 160.0,
1321
+ "epoch": 1.1224489795918366,
1322
+ "grad_norm": 0.09090136736631393,
1323
+ "kl": 0.016656771302223206,
1324
+ "learning_rate": 9.934583543669453e-05,
1325
+ "loss": 0.0007,
1326
+ "reward": 0.3464582562446594,
1327
+ "reward_std": 0.26044878363609314,
1328
+ "rewards/<lambda>": 0.3464582562446594,
1329
+ "step": 110
1330
+ },
1331
+ {
1332
+ "completion_length": 160.0,
1333
+ "epoch": 1.1326530612244898,
1334
+ "grad_norm": 0.09186483174562454,
1335
+ "kl": 0.0160512812435627,
1336
+ "learning_rate": 9.920882422145372e-05,
1337
+ "loss": 0.0006,
1338
+ "reward": 0.3516361713409424,
1339
+ "reward_std": 0.2280379831790924,
1340
+ "rewards/<lambda>": 0.3516361713409424,
1341
+ "step": 111
1342
+ },
1343
+ {
1344
+ "completion_length": 160.0,
1345
+ "epoch": 1.1428571428571428,
1346
+ "grad_norm": 0.0900593101978302,
1347
+ "kl": 0.018652722239494324,
1348
+ "learning_rate": 9.905890884491195e-05,
1349
+ "loss": 0.0007,
1350
+ "reward": 0.26406681537628174,
1351
+ "reward_std": 0.23392222821712494,
1352
+ "rewards/<lambda>": 0.26406681537628174,
1353
+ "step": 112
1354
+ },
1355
+ {
1356
+ "completion_length": 160.0,
1357
+ "epoch": 1.153061224489796,
1358
+ "grad_norm": 0.10555818676948547,
1359
+ "kl": 0.01904328167438507,
1360
+ "learning_rate": 9.889612861977853e-05,
1361
+ "loss": 0.0008,
1362
+ "reward": 0.2887289226055145,
1363
+ "reward_std": 0.21101483702659607,
1364
+ "rewards/<lambda>": 0.2887289226055145,
1365
+ "step": 113
1366
+ },
1367
+ {
1368
+ "completion_length": 160.0,
1369
+ "epoch": 1.163265306122449,
1370
+ "grad_norm": 0.10259734839200974,
1371
+ "kl": 0.021371304988861084,
1372
+ "learning_rate": 9.872052623234632e-05,
1373
+ "loss": 0.0009,
1374
+ "reward": 0.28545108437538147,
1375
+ "reward_std": 0.2336777299642563,
1376
+ "rewards/<lambda>": 0.28545108437538147,
1377
+ "step": 114
1378
+ },
1379
+ {
1380
+ "completion_length": 160.0,
1381
+ "epoch": 1.1734693877551021,
1382
+ "grad_norm": 0.10765431821346283,
1383
+ "kl": 0.022964194416999817,
1384
+ "learning_rate": 9.853214773129796e-05,
1385
+ "loss": 0.0009,
1386
+ "reward": 0.2757786214351654,
1387
+ "reward_std": 0.2596602439880371,
1388
+ "rewards/<lambda>": 0.2757786214351654,
1389
+ "step": 115
1390
+ },
1391
+ {
1392
+ "completion_length": 160.0,
1393
+ "epoch": 1.183673469387755,
1394
+ "grad_norm": 0.10045947879552841,
1395
+ "kl": 0.01650853641331196,
1396
+ "learning_rate": 9.833104251563056e-05,
1397
+ "loss": 0.0007,
1398
+ "reward": 0.37531301379203796,
1399
+ "reward_std": 0.19301483035087585,
1400
+ "rewards/<lambda>": 0.37531301379203796,
1401
+ "step": 116
1402
+ },
1403
+ {
1404
+ "completion_length": 160.0,
1405
+ "epoch": 1.193877551020408,
1406
+ "grad_norm": 0.08847310394048691,
1407
+ "kl": 0.021333348006010056,
1408
+ "learning_rate": 9.811726332170153e-05,
1409
+ "loss": 0.0009,
1410
+ "reward": 0.3334275186061859,
1411
+ "reward_std": 0.22464393079280853,
1412
+ "rewards/<lambda>": 0.3334275186061859,
1413
+ "step": 117
1414
+ },
1415
+ {
1416
+ "completion_length": 160.0,
1417
+ "epoch": 1.2040816326530612,
1418
+ "grad_norm": 0.09106040000915527,
1419
+ "kl": 0.019402675330638885,
1420
+ "learning_rate": 9.789086620939936e-05,
1421
+ "loss": 0.0008,
1422
+ "reward": 0.3201903998851776,
1423
+ "reward_std": 0.24303846061229706,
1424
+ "rewards/<lambda>": 0.3201903998851776,
1425
+ "step": 118
1426
+ },
1427
+ {
1428
+ "completion_length": 160.0,
1429
+ "epoch": 1.2142857142857142,
1430
+ "grad_norm": 0.09148433804512024,
1431
+ "kl": 0.020076055079698563,
1432
+ "learning_rate": 9.765191054744305e-05,
1433
+ "loss": 0.0008,
1434
+ "reward": 0.26528307795524597,
1435
+ "reward_std": 0.20828311145305634,
1436
+ "rewards/<lambda>": 0.26528307795524597,
1437
+ "step": 119
1438
+ },
1439
+ {
1440
+ "completion_length": 160.0,
1441
+ "epoch": 1.2244897959183674,
1442
+ "grad_norm": 0.09016289561986923,
1443
+ "kl": 0.02068479359149933,
1444
+ "learning_rate": 9.740045899781352e-05,
1445
+ "loss": 0.0008,
1446
+ "reward": 0.3603302836418152,
1447
+ "reward_std": 0.2604081332683563,
1448
+ "rewards/<lambda>": 0.3603302836418152,
1449
+ "step": 120
1450
+ },
1451
+ {
1452
+ "completion_length": 160.0,
1453
+ "epoch": 1.2346938775510203,
1454
+ "grad_norm": 0.08944398164749146,
1455
+ "kl": 0.02130693942308426,
1456
+ "learning_rate": 9.713657749932172e-05,
1457
+ "loss": 0.0009,
1458
+ "reward": 0.345991313457489,
1459
+ "reward_std": 0.2438054382801056,
1460
+ "rewards/<lambda>": 0.345991313457489,
1461
+ "step": 121
1462
+ },
1463
+ {
1464
+ "completion_length": 160.0,
1465
+ "epoch": 1.2448979591836735,
1466
+ "grad_norm": 0.08998019993305206,
1467
+ "kl": 0.018094293773174286,
1468
+ "learning_rate": 9.686033525031719e-05,
1469
+ "loss": 0.0007,
1470
+ "reward": 0.23993568122386932,
1471
+ "reward_std": 0.2522079348564148,
1472
+ "rewards/<lambda>": 0.23993568122386932,
1473
+ "step": 122
1474
+ },
1475
+ {
1476
+ "completion_length": 160.0,
1477
+ "epoch": 1.2551020408163265,
1478
+ "grad_norm": 0.09132429212331772,
1479
+ "kl": 0.021364744752645493,
1480
+ "learning_rate": 9.657180469054213e-05,
1481
+ "loss": 0.0009,
1482
+ "reward": 0.3082301616668701,
1483
+ "reward_std": 0.24264320731163025,
1484
+ "rewards/<lambda>": 0.3082301616668701,
1485
+ "step": 123
1486
+ },
1487
+ {
1488
+ "completion_length": 160.0,
1489
+ "epoch": 1.2653061224489797,
1490
+ "grad_norm": 0.07840445637702942,
1491
+ "kl": 0.01507033221423626,
1492
+ "learning_rate": 9.627106148213522e-05,
1493
+ "loss": 0.0006,
1494
+ "reward": 0.2848488390445709,
1495
+ "reward_std": 0.21467337012290955,
1496
+ "rewards/<lambda>": 0.2848488390445709,
1497
+ "step": 124
1498
+ },
1499
+ {
1500
+ "completion_length": 160.0,
1501
+ "epoch": 1.2755102040816326,
1502
+ "grad_norm": 0.08806001394987106,
1503
+ "kl": 0.018644016236066818,
1504
+ "learning_rate": 9.595818448979061e-05,
1505
+ "loss": 0.0007,
1506
+ "reward": 0.2717777192592621,
1507
+ "reward_std": 0.2168688029050827,
1508
+ "rewards/<lambda>": 0.2717777192592621,
1509
+ "step": 125
1510
+ },
1511
+ {
1512
+ "completion_length": 160.0,
1513
+ "epoch": 1.2857142857142856,
1514
+ "grad_norm": 0.09057774394750595,
1515
+ "kl": 0.019513610750436783,
1516
+ "learning_rate": 9.563325576007701e-05,
1517
+ "loss": 0.0008,
1518
+ "reward": 0.3114008903503418,
1519
+ "reward_std": 0.2647177577018738,
1520
+ "rewards/<lambda>": 0.3114008903503418,
1521
+ "step": 126
1522
+ },
1523
+ {
1524
+ "completion_length": 160.0,
1525
+ "epoch": 1.2959183673469388,
1526
+ "grad_norm": 0.09328091889619827,
1527
+ "kl": 0.02476987987756729,
1528
+ "learning_rate": 9.529636049992234e-05,
1529
+ "loss": 0.001,
1530
+ "reward": 0.35005778074264526,
1531
+ "reward_std": 0.2296982854604721,
1532
+ "rewards/<lambda>": 0.35005778074264526,
1533
+ "step": 127
1534
+ },
1535
+ {
1536
+ "completion_length": 160.0,
1537
+ "epoch": 1.306122448979592,
1538
+ "grad_norm": 0.09036798775196075,
1539
+ "kl": 0.0257786326110363,
1540
+ "learning_rate": 9.494758705426978e-05,
1541
+ "loss": 0.001,
1542
+ "reward": 0.3622300624847412,
1543
+ "reward_std": 0.23824195563793182,
1544
+ "rewards/<lambda>": 0.3622300624847412,
1545
+ "step": 128
1546
+ },
1547
+ {
1548
+ "completion_length": 160.0,
1549
+ "epoch": 1.316326530612245,
1550
+ "grad_norm": 0.11859830468893051,
1551
+ "kl": 0.021266868337988853,
1552
+ "learning_rate": 9.458702688291073e-05,
1553
+ "loss": 0.0009,
1554
+ "reward": 0.2144925594329834,
1555
+ "reward_std": 0.2241775244474411,
1556
+ "rewards/<lambda>": 0.2144925594329834,
1557
+ "step": 129
1558
+ },
1559
+ {
1560
+ "completion_length": 160.0,
1561
+ "epoch": 1.3265306122448979,
1562
+ "grad_norm": 0.08116462826728821,
1563
+ "kl": 0.0258474238216877,
1564
+ "learning_rate": 9.421477453650118e-05,
1565
+ "loss": 0.001,
1566
+ "reward": 0.29674050211906433,
1567
+ "reward_std": 0.22861354053020477,
1568
+ "rewards/<lambda>": 0.29674050211906433,
1569
+ "step": 130
1570
+ },
1571
+ {
1572
+ "completion_length": 160.0,
1573
+ "epoch": 1.336734693877551,
1574
+ "grad_norm": 0.08280783146619797,
1575
+ "kl": 0.024379916489124298,
1576
+ "learning_rate": 9.38309276317674e-05,
1577
+ "loss": 0.001,
1578
+ "reward": 0.3025866746902466,
1579
+ "reward_std": 0.2633967101573944,
1580
+ "rewards/<lambda>": 0.3025866746902466,
1581
+ "step": 131
1582
+ },
1583
+ {
1584
+ "completion_length": 160.0,
1585
+ "epoch": 1.346938775510204,
1586
+ "grad_norm": 0.09579820185899734,
1587
+ "kl": 0.02445649728178978,
1588
+ "learning_rate": 9.343558682590756e-05,
1589
+ "loss": 0.001,
1590
+ "reward": 0.28383010625839233,
1591
+ "reward_std": 0.24883991479873657,
1592
+ "rewards/<lambda>": 0.28383010625839233,
1593
+ "step": 132
1594
+ },
1595
+ {
1596
+ "completion_length": 160.0,
1597
+ "epoch": 1.3571428571428572,
1598
+ "grad_norm": 0.08781164884567261,
1599
+ "kl": 0.023096945136785507,
1600
+ "learning_rate": 9.302885579019627e-05,
1601
+ "loss": 0.0009,
1602
+ "reward": 0.27924996614456177,
1603
+ "reward_std": 0.2151711881160736,
1604
+ "rewards/<lambda>": 0.27924996614456177,
1605
+ "step": 133
1606
+ },
1607
+ {
1608
+ "completion_length": 160.0,
1609
+ "epoch": 1.3673469387755102,
1610
+ "grad_norm": 0.09480538964271545,
1611
+ "kl": 0.02117522805929184,
1612
+ "learning_rate": 9.261084118279847e-05,
1613
+ "loss": 0.0008,
1614
+ "reward": 0.37825846672058105,
1615
+ "reward_std": 0.2239784449338913,
1616
+ "rewards/<lambda>": 0.37825846672058105,
1617
+ "step": 134
1618
+ },
1619
+ {
1620
+ "completion_length": 160.0,
1621
+ "epoch": 1.3775510204081631,
1622
+ "grad_norm": 0.09008127450942993,
1623
+ "kl": 0.023548265919089317,
1624
+ "learning_rate": 9.218165262080023e-05,
1625
+ "loss": 0.0009,
1626
+ "reward": 0.26018446683883667,
1627
+ "reward_std": 0.23662327229976654,
1628
+ "rewards/<lambda>": 0.26018446683883667,
1629
+ "step": 135
1630
+ },
1631
+ {
1632
+ "completion_length": 160.0,
1633
+ "epoch": 1.3877551020408163,
1634
+ "grad_norm": 0.10781973600387573,
1635
+ "kl": 0.024034924805164337,
1636
+ "learning_rate": 9.174140265146356e-05,
1637
+ "loss": 0.001,
1638
+ "reward": 0.3326035141944885,
1639
+ "reward_std": 0.2397647500038147,
1640
+ "rewards/<lambda>": 0.3326035141944885,
1641
+ "step": 136
1642
+ },
1643
+ {
1644
+ "completion_length": 160.0,
1645
+ "epoch": 1.3979591836734695,
1646
+ "grad_norm": 0.09277699142694473,
1647
+ "kl": 0.026356618851423264,
1648
+ "learning_rate": 9.129020672271283e-05,
1649
+ "loss": 0.0011,
1650
+ "reward": 0.33098840713500977,
1651
+ "reward_std": 0.2678415775299072,
1652
+ "rewards/<lambda>": 0.33098840713500977,
1653
+ "step": 137
1654
+ },
1655
+ {
1656
+ "completion_length": 160.0,
1657
+ "epoch": 1.4081632653061225,
1658
+ "grad_norm": 0.09566947817802429,
1659
+ "kl": 0.025415873154997826,
1660
+ "learning_rate": 9.082818315286055e-05,
1661
+ "loss": 0.001,
1662
+ "reward": 0.30005940794944763,
1663
+ "reward_std": 0.2622036635875702,
1664
+ "rewards/<lambda>": 0.30005940794944763,
1665
+ "step": 138
1666
+ },
1667
+ {
1668
+ "completion_length": 160.0,
1669
+ "epoch": 1.4183673469387754,
1670
+ "grad_norm": 0.09971340000629425,
1671
+ "kl": 0.023971717804670334,
1672
+ "learning_rate": 9.035545309958046e-05,
1673
+ "loss": 0.001,
1674
+ "reward": 0.29274553060531616,
1675
+ "reward_std": 0.2605469822883606,
1676
+ "rewards/<lambda>": 0.29274553060531616,
1677
+ "step": 139
1678
+ },
1679
+ {
1680
+ "completion_length": 160.0,
1681
+ "epoch": 1.4285714285714286,
1682
+ "grad_norm": 0.17278257012367249,
1683
+ "kl": 0.036384448409080505,
1684
+ "learning_rate": 8.987214052813604e-05,
1685
+ "loss": 0.0015,
1686
+ "reward": 0.3640914857387543,
1687
+ "reward_std": 0.22631150484085083,
1688
+ "rewards/<lambda>": 0.3640914857387543,
1689
+ "step": 140
1690
+ },
1691
+ {
1692
+ "completion_length": 160.0,
1693
+ "epoch": 1.4387755102040816,
1694
+ "grad_norm": 0.0879712924361229,
1695
+ "kl": 0.022140463814139366,
1696
+ "learning_rate": 8.937837217887273e-05,
1697
+ "loss": 0.0009,
1698
+ "reward": 0.3031749427318573,
1699
+ "reward_std": 0.2355748862028122,
1700
+ "rewards/<lambda>": 0.3031749427318573,
1701
+ "step": 141
1702
+ },
1703
+ {
1704
+ "completion_length": 160.0,
1705
+ "epoch": 1.4489795918367347,
1706
+ "grad_norm": 0.09844816476106644,
1707
+ "kl": 0.02770843915641308,
1708
+ "learning_rate": 8.887427753398248e-05,
1709
+ "loss": 0.0011,
1710
+ "reward": 0.333422988653183,
1711
+ "reward_std": 0.2207636535167694,
1712
+ "rewards/<lambda>": 0.333422988653183,
1713
+ "step": 142
1714
+ },
1715
+ {
1716
+ "completion_length": 160.0,
1717
+ "epoch": 1.4591836734693877,
1718
+ "grad_norm": 0.08592235296964645,
1719
+ "kl": 0.02235778421163559,
1720
+ "learning_rate": 8.835998878354931e-05,
1721
+ "loss": 0.0009,
1722
+ "reward": 0.3474034070968628,
1723
+ "reward_std": 0.22502724826335907,
1724
+ "rewards/<lambda>": 0.3474034070968628,
1725
+ "step": 143
1726
+ },
1727
+ {
1728
+ "completion_length": 160.0,
1729
+ "epoch": 1.469387755102041,
1730
+ "grad_norm": 0.10064583271741867,
1731
+ "kl": 0.01792587712407112,
1732
+ "learning_rate": 8.783564079088477e-05,
1733
+ "loss": 0.0007,
1734
+ "reward": 0.25580060482025146,
1735
+ "reward_std": 0.2477901428937912,
1736
+ "rewards/<lambda>": 0.25580060482025146,
1737
+ "step": 144
1738
+ },
1739
+ {
1740
+ "completion_length": 160.0,
1741
+ "epoch": 1.4795918367346939,
1742
+ "grad_norm": 0.10648388415575027,
1743
+ "kl": 0.020750422030687332,
1744
+ "learning_rate": 8.73013710571623e-05,
1745
+ "loss": 0.0008,
1746
+ "reward": 0.3896491527557373,
1747
+ "reward_std": 0.24791425466537476,
1748
+ "rewards/<lambda>": 0.3896491527557373,
1749
+ "step": 145
1750
+ },
1751
+ {
1752
+ "completion_length": 160.0,
1753
+ "epoch": 1.489795918367347,
1754
+ "grad_norm": 0.09451978653669357,
1755
+ "kl": 0.024195339530706406,
1756
+ "learning_rate": 8.675731968536002e-05,
1757
+ "loss": 0.001,
1758
+ "reward": 0.29740995168685913,
1759
+ "reward_std": 0.26704221963882446,
1760
+ "rewards/<lambda>": 0.29740995168685913,
1761
+ "step": 146
1762
+ },
1763
+ {
1764
+ "completion_length": 160.0,
1765
+ "epoch": 1.5,
1766
+ "grad_norm": 0.08318852633237839,
1767
+ "kl": 0.02090907096862793,
1768
+ "learning_rate": 8.620362934352109e-05,
1769
+ "loss": 0.0008,
1770
+ "reward": 0.2733314037322998,
1771
+ "reward_std": 0.2534288763999939,
1772
+ "rewards/<lambda>": 0.2733314037322998,
1773
+ "step": 147
1774
+ },
1775
+ {
1776
+ "completion_length": 160.0,
1777
+ "epoch": 1.510204081632653,
1778
+ "grad_norm": 0.09782999753952026,
1779
+ "kl": 0.019019391387701035,
1780
+ "learning_rate": 8.564044522734147e-05,
1781
+ "loss": 0.0008,
1782
+ "reward": 0.32081273198127747,
1783
+ "reward_std": 0.2417653501033783,
1784
+ "rewards/<lambda>": 0.32081273198127747,
1785
+ "step": 148
1786
+ },
1787
+ {
1788
+ "completion_length": 160.0,
1789
+ "epoch": 1.5204081632653061,
1790
+ "grad_norm": 0.08283797651529312,
1791
+ "kl": 0.024606352671980858,
1792
+ "learning_rate": 8.506791502209496e-05,
1793
+ "loss": 0.001,
1794
+ "reward": 0.2620502710342407,
1795
+ "reward_std": 0.26248690485954285,
1796
+ "rewards/<lambda>": 0.2620502710342407,
1797
+ "step": 149
1798
+ },
1799
+ {
1800
+ "completion_length": 160.0,
1801
+ "epoch": 1.5306122448979593,
1802
+ "grad_norm": 0.09963952004909515,
1803
+ "kl": 0.023783504962921143,
1804
+ "learning_rate": 8.448618886390522e-05,
1805
+ "loss": 0.001,
1806
+ "reward": 0.30407556891441345,
1807
+ "reward_std": 0.2542496621608734,
1808
+ "rewards/<lambda>": 0.30407556891441345,
1809
+ "step": 150
1810
+ },
1811
+ {
1812
+ "completion_length": 160.0,
1813
+ "epoch": 1.5408163265306123,
1814
+ "grad_norm": 0.08900399506092072,
1815
+ "kl": 0.018189266324043274,
1816
+ "learning_rate": 8.389541930037516e-05,
1817
+ "loss": 0.0007,
1818
+ "reward": 0.34692129492759705,
1819
+ "reward_std": 0.25238391757011414,
1820
+ "rewards/<lambda>": 0.34692129492759705,
1821
+ "step": 151
1822
+ },
1823
+ {
1824
+ "completion_length": 160.0,
1825
+ "epoch": 1.5510204081632653,
1826
+ "grad_norm": 0.08934988081455231,
1827
+ "kl": 0.02292565628886223,
1828
+ "learning_rate": 8.329576125058406e-05,
1829
+ "loss": 0.0009,
1830
+ "reward": 0.36156541109085083,
1831
+ "reward_std": 0.23901474475860596,
1832
+ "rewards/<lambda>": 0.36156541109085083,
1833
+ "step": 152
1834
+ },
1835
+ {
1836
+ "completion_length": 160.0,
1837
+ "epoch": 1.5612244897959182,
1838
+ "grad_norm": 0.08225736767053604,
1839
+ "kl": 0.018968891352415085,
1840
+ "learning_rate": 8.268737196446264e-05,
1841
+ "loss": 0.0008,
1842
+ "reward": 0.29287439584732056,
1843
+ "reward_std": 0.2568100690841675,
1844
+ "rewards/<lambda>": 0.29287439584732056,
1845
+ "step": 153
1846
+ },
1847
+ {
1848
+ "completion_length": 160.0,
1849
+ "epoch": 1.5714285714285714,
1850
+ "grad_norm": 0.0784936249256134,
1851
+ "kl": 0.017963310703635216,
1852
+ "learning_rate": 8.2070410981557e-05,
1853
+ "loss": 0.0007,
1854
+ "reward": 0.3246784210205078,
1855
+ "reward_std": 0.25057297945022583,
1856
+ "rewards/<lambda>": 0.3246784210205078,
1857
+ "step": 154
1858
+ },
1859
+ {
1860
+ "completion_length": 160.0,
1861
+ "epoch": 1.5816326530612246,
1862
+ "grad_norm": 0.07634963095188141,
1863
+ "kl": 0.01993548683822155,
1864
+ "learning_rate": 8.144504008919222e-05,
1865
+ "loss": 0.0008,
1866
+ "reward": 0.32869774103164673,
1867
+ "reward_std": 0.22997353971004486,
1868
+ "rewards/<lambda>": 0.32869774103164673,
1869
+ "step": 155
1870
+ },
1871
+ {
1872
+ "completion_length": 160.0,
1873
+ "epoch": 1.5918367346938775,
1874
+ "grad_norm": 0.2396547645330429,
1875
+ "kl": 0.06700322777032852,
1876
+ "learning_rate": 8.081142328004637e-05,
1877
+ "loss": 0.0027,
1878
+ "reward": 0.35755395889282227,
1879
+ "reward_std": 0.21904043853282928,
1880
+ "rewards/<lambda>": 0.35755395889282227,
1881
+ "step": 156
1882
+ },
1883
+ {
1884
+ "completion_length": 160.0,
1885
+ "epoch": 1.6020408163265305,
1886
+ "grad_norm": 0.08202355355024338,
1887
+ "kl": 0.01816694065928459,
1888
+ "learning_rate": 8.016972670914624e-05,
1889
+ "loss": 0.0007,
1890
+ "reward": 0.34367379546165466,
1891
+ "reward_std": 0.24324068427085876,
1892
+ "rewards/<lambda>": 0.34367379546165466,
1893
+ "step": 157
1894
+ },
1895
+ {
1896
+ "completion_length": 160.0,
1897
+ "epoch": 1.6122448979591837,
1898
+ "grad_norm": 0.10310099273920059,
1899
+ "kl": 0.023697612807154655,
1900
+ "learning_rate": 7.952011865029614e-05,
1901
+ "loss": 0.0009,
1902
+ "reward": 0.2922106683254242,
1903
+ "reward_std": 0.26478201150894165,
1904
+ "rewards/<lambda>": 0.2922106683254242,
1905
+ "step": 158
1906
+ },
1907
+ {
1908
+ "completion_length": 160.0,
1909
+ "epoch": 1.6224489795918369,
1910
+ "grad_norm": 0.07689829915761948,
1911
+ "kl": 0.01890609599649906,
1912
+ "learning_rate": 7.886276945195099e-05,
1913
+ "loss": 0.0008,
1914
+ "reward": 0.3211023807525635,
1915
+ "reward_std": 0.24797722697257996,
1916
+ "rewards/<lambda>": 0.3211023807525635,
1917
+ "step": 159
1918
+ },
1919
+ {
1920
+ "completion_length": 160.0,
1921
+ "epoch": 1.6326530612244898,
1922
+ "grad_norm": 0.07625985890626907,
1923
+ "kl": 0.014439202845096588,
1924
+ "learning_rate": 7.819785149254532e-05,
1925
+ "loss": 0.0006,
1926
+ "reward": 0.36535224318504333,
1927
+ "reward_std": 0.26052916049957275,
1928
+ "rewards/<lambda>": 0.36535224318504333,
1929
+ "step": 160
1930
+ },
1931
+ {
1932
+ "completion_length": 160.0,
1933
+ "epoch": 1.6428571428571428,
1934
+ "grad_norm": 0.09949609637260437,
1935
+ "kl": 0.015952972695231438,
1936
+ "learning_rate": 7.752553913529018e-05,
1937
+ "loss": 0.0006,
1938
+ "reward": 0.34558984637260437,
1939
+ "reward_std": 0.24385800957679749,
1940
+ "rewards/<lambda>": 0.34558984637260437,
1941
+ "step": 161
1942
+ },
1943
+ {
1944
+ "completion_length": 160.0,
1945
+ "epoch": 1.6530612244897958,
1946
+ "grad_norm": 0.07270115613937378,
1947
+ "kl": 0.017309173941612244,
1948
+ "learning_rate": 7.68460086824492e-05,
1949
+ "loss": 0.0007,
1950
+ "reward": 0.3196105659008026,
1951
+ "reward_std": 0.24196776747703552,
1952
+ "rewards/<lambda>": 0.3196105659008026,
1953
+ "step": 162
1954
+ },
1955
+ {
1956
+ "completion_length": 160.0,
1957
+ "epoch": 1.663265306122449,
1958
+ "grad_norm": 0.08732082694768906,
1959
+ "kl": 0.02441173419356346,
1960
+ "learning_rate": 7.61594383291065e-05,
1961
+ "loss": 0.001,
1962
+ "reward": 0.317618191242218,
1963
+ "reward_std": 0.2187143862247467,
1964
+ "rewards/<lambda>": 0.317618191242218,
1965
+ "step": 163
1966
+ },
1967
+ {
1968
+ "completion_length": 160.0,
1969
+ "epoch": 1.6734693877551021,
1970
+ "grad_norm": 0.07517392933368683,
1971
+ "kl": 0.015940185636281967,
1972
+ "learning_rate": 7.546600811643816e-05,
1973
+ "loss": 0.0006,
1974
+ "reward": 0.2270391881465912,
1975
+ "reward_std": 0.24194824695587158,
1976
+ "rewards/<lambda>": 0.2270391881465912,
1977
+ "step": 164
1978
+ },
1979
+ {
1980
+ "completion_length": 160.0,
1981
+ "epoch": 1.683673469387755,
1982
+ "grad_norm": 0.07552263885736465,
1983
+ "kl": 0.01631421223282814,
1984
+ "learning_rate": 7.476589988449939e-05,
1985
+ "loss": 0.0007,
1986
+ "reward": 0.3129787743091583,
1987
+ "reward_std": 0.22886860370635986,
1988
+ "rewards/<lambda>": 0.3129787743091583,
1989
+ "step": 165
1990
+ },
1991
+ {
1992
+ "completion_length": 160.0,
1993
+ "epoch": 1.693877551020408,
1994
+ "grad_norm": 0.08246885985136032,
1995
+ "kl": 0.01746886596083641,
1996
+ "learning_rate": 7.405929722454026e-05,
1997
+ "loss": 0.0007,
1998
+ "reward": 0.2953646779060364,
1999
+ "reward_std": 0.2490847110748291,
2000
+ "rewards/<lambda>": 0.2953646779060364,
2001
+ "step": 166
2002
+ },
2003
+ {
2004
+ "completion_length": 160.0,
2005
+ "epoch": 1.7040816326530612,
2006
+ "grad_norm": 0.07670585811138153,
2007
+ "kl": 0.021257830783724785,
2008
+ "learning_rate": 7.334638543086203e-05,
2009
+ "loss": 0.0009,
2010
+ "reward": 0.39490482211112976,
2011
+ "reward_std": 0.2520644962787628,
2012
+ "rewards/<lambda>": 0.39490482211112976,
2013
+ "step": 167
2014
+ },
2015
+ {
2016
+ "completion_length": 160.0,
2017
+ "epoch": 1.7142857142857144,
2018
+ "grad_norm": 0.07335949689149857,
2019
+ "kl": 0.012844810262322426,
2020
+ "learning_rate": 7.262735145222696e-05,
2021
+ "loss": 0.0005,
2022
+ "reward": 0.3359541893005371,
2023
+ "reward_std": 0.22056405246257782,
2024
+ "rewards/<lambda>": 0.3359541893005371,
2025
+ "step": 168
2026
+ },
2027
+ {
2028
+ "completion_length": 160.0,
2029
+ "epoch": 1.7244897959183674,
2030
+ "grad_norm": 0.10231731832027435,
2031
+ "kl": 0.020412620157003403,
2032
+ "learning_rate": 7.190238384283412e-05,
2033
+ "loss": 0.0008,
2034
+ "reward": 0.2858106791973114,
2035
+ "reward_std": 0.22994212806224823,
2036
+ "rewards/<lambda>": 0.2858106791973114,
2037
+ "step": 169
2038
+ },
2039
+ {
2040
+ "completion_length": 160.0,
2041
+ "epoch": 1.7346938775510203,
2042
+ "grad_norm": 0.0776882916688919,
2043
+ "kl": 0.015348056331276894,
2044
+ "learning_rate": 7.117167271287453e-05,
2045
+ "loss": 0.0006,
2046
+ "reward": 0.32561856508255005,
2047
+ "reward_std": 0.24203304946422577,
2048
+ "rewards/<lambda>": 0.32561856508255005,
2049
+ "step": 170
2050
+ },
2051
+ {
2052
+ "completion_length": 160.0,
2053
+ "epoch": 1.7448979591836735,
2054
+ "grad_norm": 0.09082949161529541,
2055
+ "kl": 0.021862512454390526,
2056
+ "learning_rate": 7.043540967867782e-05,
2057
+ "loss": 0.0009,
2058
+ "reward": 0.3552192449569702,
2059
+ "reward_std": 0.2229807823896408,
2060
+ "rewards/<lambda>": 0.3552192449569702,
2061
+ "step": 171
2062
+ },
2063
+ {
2064
+ "completion_length": 160.0,
2065
+ "epoch": 1.7551020408163265,
2066
+ "grad_norm": 0.09254392236471176,
2067
+ "kl": 0.023197144269943237,
2068
+ "learning_rate": 6.969378781246436e-05,
2069
+ "loss": 0.0009,
2070
+ "reward": 0.3580297827720642,
2071
+ "reward_std": 0.25305554270744324,
2072
+ "rewards/<lambda>": 0.3580297827720642,
2073
+ "step": 172
2074
+ },
2075
+ {
2076
+ "completion_length": 160.0,
2077
+ "epoch": 1.7653061224489797,
2078
+ "grad_norm": 0.09547614306211472,
2079
+ "kl": 0.016374874860048294,
2080
+ "learning_rate": 6.894700159171534e-05,
2081
+ "loss": 0.0007,
2082
+ "reward": 0.3224062919616699,
2083
+ "reward_std": 0.22778098285198212,
2084
+ "rewards/<lambda>": 0.3224062919616699,
2085
+ "step": 173
2086
+ },
2087
+ {
2088
+ "completion_length": 160.0,
2089
+ "epoch": 1.7755102040816326,
2090
+ "grad_norm": 0.07411126792430878,
2091
+ "kl": 0.025782398879528046,
2092
+ "learning_rate": 6.819524684817438e-05,
2093
+ "loss": 0.001,
2094
+ "reward": 0.3161015212535858,
2095
+ "reward_std": 0.24362775683403015,
2096
+ "rewards/<lambda>": 0.3161015212535858,
2097
+ "step": 174
2098
+ },
2099
+ {
2100
+ "completion_length": 160.0,
2101
+ "epoch": 1.7857142857142856,
2102
+ "grad_norm": 0.06989577412605286,
2103
+ "kl": 0.014967722818255424,
2104
+ "learning_rate": 6.743872071649411e-05,
2105
+ "loss": 0.0006,
2106
+ "reward": 0.3597205877304077,
2107
+ "reward_std": 0.22002741694450378,
2108
+ "rewards/<lambda>": 0.3597205877304077,
2109
+ "step": 175
2110
+ },
2111
+ {
2112
+ "completion_length": 160.0,
2113
+ "epoch": 1.7959183673469388,
2114
+ "grad_norm": 0.07689698040485382,
2115
+ "kl": 0.014289500191807747,
2116
+ "learning_rate": 6.667762158254104e-05,
2117
+ "loss": 0.0006,
2118
+ "reward": 0.349145770072937,
2119
+ "reward_std": 0.25986701250076294,
2120
+ "rewards/<lambda>": 0.349145770072937,
2121
+ "step": 176
2122
+ },
2123
+ {
2124
+ "completion_length": 160.0,
2125
+ "epoch": 1.806122448979592,
2126
+ "grad_norm": 0.08618627488613129,
2127
+ "kl": 0.01953529752790928,
2128
+ "learning_rate": 6.59121490313722e-05,
2129
+ "loss": 0.0008,
2130
+ "reward": 0.2717389762401581,
2131
+ "reward_std": 0.24193832278251648,
2132
+ "rewards/<lambda>": 0.2717389762401581,
2133
+ "step": 177
2134
+ },
2135
+ {
2136
+ "completion_length": 160.0,
2137
+ "epoch": 1.816326530612245,
2138
+ "grad_norm": 0.08920174837112427,
2139
+ "kl": 0.02378169447183609,
2140
+ "learning_rate": 6.514250379489753e-05,
2141
+ "loss": 0.001,
2142
+ "reward": 0.3774096667766571,
2143
+ "reward_std": 0.20812949538230896,
2144
+ "rewards/<lambda>": 0.3774096667766571,
2145
+ "step": 178
2146
+ },
2147
+ {
2148
+ "completion_length": 160.0,
2149
+ "epoch": 1.8265306122448979,
2150
+ "grad_norm": 0.07194539904594421,
2151
+ "kl": 0.013617664575576782,
2152
+ "learning_rate": 6.436888769924142e-05,
2153
+ "loss": 0.0005,
2154
+ "reward": 0.36836332082748413,
2155
+ "reward_std": 0.24121849238872528,
2156
+ "rewards/<lambda>": 0.36836332082748413,
2157
+ "step": 179
2158
+ },
2159
+ {
2160
+ "completion_length": 160.0,
2161
+ "epoch": 1.836734693877551,
2162
+ "grad_norm": 0.08749783784151077,
2163
+ "kl": 0.019728073850274086,
2164
+ "learning_rate": 6.359150361181715e-05,
2165
+ "loss": 0.0008,
2166
+ "reward": 0.3138796091079712,
2167
+ "reward_std": 0.21334460377693176,
2168
+ "rewards/<lambda>": 0.3138796091079712,
2169
+ "step": 180
2170
+ },
2171
+ {
2172
+ "completion_length": 160.0,
2173
+ "epoch": 1.8469387755102042,
2174
+ "grad_norm": 0.08991340547800064,
2175
+ "kl": 0.019904986023902893,
2176
+ "learning_rate": 6.281055538812861e-05,
2177
+ "loss": 0.0008,
2178
+ "reward": 0.28394806385040283,
2179
+ "reward_std": 0.26574069261550903,
2180
+ "rewards/<lambda>": 0.28394806385040283,
2181
+ "step": 181
2182
+ },
2183
+ {
2184
+ "completion_length": 160.0,
2185
+ "epoch": 1.8571428571428572,
2186
+ "grad_norm": 0.07849021255970001,
2187
+ "kl": 0.022857721894979477,
2188
+ "learning_rate": 6.202624781831268e-05,
2189
+ "loss": 0.0009,
2190
+ "reward": 0.31418418884277344,
2191
+ "reward_std": 0.2225717306137085,
2192
+ "rewards/<lambda>": 0.31418418884277344,
2193
+ "step": 182
2194
+ },
2195
+ {
2196
+ "completion_length": 160.0,
2197
+ "epoch": 1.8673469387755102,
2198
+ "grad_norm": 0.12505197525024414,
2199
+ "kl": 0.019057810306549072,
2200
+ "learning_rate": 6.123878657343648e-05,
2201
+ "loss": 0.0008,
2202
+ "reward": 0.29661425948143005,
2203
+ "reward_std": 0.22678154706954956,
2204
+ "rewards/<lambda>": 0.29661425948143005,
2205
+ "step": 183
2206
+ },
2207
+ {
2208
+ "completion_length": 160.0,
2209
+ "epoch": 1.8775510204081631,
2210
+ "grad_norm": 0.08824272453784943,
2211
+ "kl": 0.024682914838194847,
2212
+ "learning_rate": 6.044837815156377e-05,
2213
+ "loss": 0.001,
2214
+ "reward": 0.30208826065063477,
2215
+ "reward_std": 0.25953543186187744,
2216
+ "rewards/<lambda>": 0.30208826065063477,
2217
+ "step": 184
2218
+ },
2219
+ {
2220
+ "completion_length": 160.0,
2221
+ "epoch": 1.8877551020408163,
2222
+ "grad_norm": 0.07919792085886002,
2223
+ "kl": 0.017491590231657028,
2224
+ "learning_rate": 5.9655229823604406e-05,
2225
+ "loss": 0.0007,
2226
+ "reward": 0.2938078045845032,
2227
+ "reward_std": 0.24010448157787323,
2228
+ "rewards/<lambda>": 0.2938078045845032,
2229
+ "step": 185
2230
+ },
2231
+ {
2232
+ "completion_length": 160.0,
2233
+ "epoch": 1.8979591836734695,
2234
+ "grad_norm": 0.08519456535577774,
2235
+ "kl": 0.014628434553742409,
2236
+ "learning_rate": 5.885954957896115e-05,
2237
+ "loss": 0.0006,
2238
+ "reward": 0.3203762471675873,
2239
+ "reward_std": 0.21388846635818481,
2240
+ "rewards/<lambda>": 0.3203762471675873,
2241
+ "step": 186
2242
+ },
2243
+ {
2244
+ "completion_length": 160.0,
2245
+ "epoch": 1.9081632653061225,
2246
+ "grad_norm": 0.09762715548276901,
2247
+ "kl": 0.021606557071208954,
2248
+ "learning_rate": 5.8061546070987994e-05,
2249
+ "loss": 0.0009,
2250
+ "reward": 0.2728058099746704,
2251
+ "reward_std": 0.25408631563186646,
2252
+ "rewards/<lambda>": 0.2728058099746704,
2253
+ "step": 187
2254
+ },
2255
+ {
2256
+ "completion_length": 160.0,
2257
+ "epoch": 1.9183673469387754,
2258
+ "grad_norm": 0.1670609414577484,
2259
+ "kl": 0.03298768401145935,
2260
+ "learning_rate": 5.726142856227452e-05,
2261
+ "loss": 0.0013,
2262
+ "reward": 0.3018071949481964,
2263
+ "reward_std": 0.24476169049739838,
2264
+ "rewards/<lambda>": 0.3018071949481964,
2265
+ "step": 188
2266
+ },
2267
+ {
2268
+ "completion_length": 160.0,
2269
+ "epoch": 1.9285714285714286,
2270
+ "grad_norm": 0.08198920637369156,
2271
+ "kl": 0.016482073813676834,
2272
+ "learning_rate": 5.645940686977033e-05,
2273
+ "loss": 0.0007,
2274
+ "reward": 0.33921799063682556,
2275
+ "reward_std": 0.2550126910209656,
2276
+ "rewards/<lambda>": 0.33921799063682556,
2277
+ "step": 189
2278
+ },
2279
+ {
2280
+ "completion_length": 160.0,
2281
+ "epoch": 1.9387755102040818,
2282
+ "grad_norm": 0.07509396970272064,
2283
+ "kl": 0.014598744921386242,
2284
+ "learning_rate": 5.565569130976422e-05,
2285
+ "loss": 0.0006,
2286
+ "reward": 0.36032330989837646,
2287
+ "reward_std": 0.2456628531217575,
2288
+ "rewards/<lambda>": 0.36032330989837646,
2289
+ "step": 190
2290
+ },
2291
+ {
2292
+ "completion_length": 160.0,
2293
+ "epoch": 1.9489795918367347,
2294
+ "grad_norm": 0.07215838134288788,
2295
+ "kl": 0.012914094142615795,
2296
+ "learning_rate": 5.4850492642732406e-05,
2297
+ "loss": 0.0005,
2298
+ "reward": 0.3160780072212219,
2299
+ "reward_std": 0.260766863822937,
2300
+ "rewards/<lambda>": 0.3160780072212219,
2301
+ "step": 191
2302
+ },
2303
+ {
2304
+ "completion_length": 160.0,
2305
+ "epoch": 1.9591836734693877,
2306
+ "grad_norm": 0.08896195888519287,
2307
+ "kl": 0.019211173057556152,
2308
+ "learning_rate": 5.4044022018070214e-05,
2309
+ "loss": 0.0008,
2310
+ "reward": 0.3490823209285736,
2311
+ "reward_std": 0.26013457775115967,
2312
+ "rewards/<lambda>": 0.3490823209285736,
2313
+ "step": 192
2314
+ },
2315
+ {
2316
+ "completion_length": 160.0,
2317
+ "epoch": 1.9693877551020407,
2318
+ "grad_norm": 0.08373931050300598,
2319
+ "kl": 0.01714364066720009,
2320
+ "learning_rate": 5.3236490918721794e-05,
2321
+ "loss": 0.0007,
2322
+ "reward": 0.3829745948314667,
2323
+ "reward_std": 0.20967933535575867,
2324
+ "rewards/<lambda>": 0.3829745948314667,
2325
+ "step": 193
2326
+ },
2327
+ {
2328
+ "completion_length": 160.0,
2329
+ "epoch": 1.9795918367346939,
2330
+ "grad_norm": 0.07414960116147995,
2331
+ "kl": 0.010791709646582603,
2332
+ "learning_rate": 5.242811110572242e-05,
2333
+ "loss": 0.0004,
2334
+ "reward": 0.3071313798427582,
2335
+ "reward_std": 0.23706994950771332,
2336
+ "rewards/<lambda>": 0.3071313798427582,
2337
+ "step": 194
2338
+ },
2339
+ {
2340
+ "completion_length": 160.0,
2341
+ "epoch": 1.989795918367347,
2342
+ "grad_norm": 0.08582977205514908,
2343
+ "kl": 0.022279752418398857,
2344
+ "learning_rate": 5.1619094562667804e-05,
2345
+ "loss": 0.0009,
2346
+ "reward": 0.23579248785972595,
2347
+ "reward_std": 0.19861236214637756,
2348
+ "rewards/<lambda>": 0.23579248785972595,
2349
+ "step": 195
2350
+ },
2351
+ {
2352
+ "completion_length": 160.0,
2353
+ "epoch": 2.0,
2354
+ "grad_norm": 0.09055842459201813,
2355
+ "kl": 0.013352158479392529,
2356
+ "learning_rate": 5.080965344012508e-05,
2357
+ "loss": 0.0005,
2358
+ "reward": 0.34037166833877563,
2359
+ "reward_std": 0.26631197333335876,
2360
+ "rewards/<lambda>": 0.34037166833877563,
2361
+ "step": 196
2362
+ },
2363
+ {
2364
+ "completion_length": 160.0,
2365
+ "epoch": 2.010204081632653,
2366
+ "grad_norm": 0.08190540969371796,
2367
+ "kl": 0.01571495272219181,
2368
+ "learning_rate": 5e-05,
2369
+ "loss": 0.0006,
2370
+ "reward": 0.3017084002494812,
2371
+ "reward_std": 0.24429354071617126,
2372
+ "rewards/<lambda>": 0.3017084002494812,
2373
+ "step": 197
2374
+ },
2375
+ {
2376
+ "completion_length": 160.0,
2377
+ "epoch": 2.020408163265306,
2378
+ "grad_norm": 0.07793857902288437,
2379
+ "kl": 0.012462570331990719,
2380
+ "learning_rate": 4.919034655987493e-05,
2381
+ "loss": 0.0005,
2382
+ "reward": 0.2924309968948364,
2383
+ "reward_std": 0.2147884964942932,
2384
+ "rewards/<lambda>": 0.2924309968948364,
2385
+ "step": 198
2386
+ },
2387
+ {
2388
+ "completion_length": 160.0,
2389
+ "epoch": 2.0306122448979593,
2390
+ "grad_norm": 0.08943740278482437,
2391
+ "kl": 0.016143202781677246,
2392
+ "learning_rate": 4.838090543733222e-05,
2393
+ "loss": 0.0006,
2394
+ "reward": 0.3363001346588135,
2395
+ "reward_std": 0.252535343170166,
2396
+ "rewards/<lambda>": 0.3363001346588135,
2397
+ "step": 199
2398
+ },
2399
+ {
2400
+ "completion_length": 160.0,
2401
+ "epoch": 2.0408163265306123,
2402
+ "grad_norm": 0.07238653302192688,
2403
+ "kl": 0.01266445405781269,
2404
+ "learning_rate": 4.7571888894277604e-05,
2405
+ "loss": 0.0005,
2406
+ "reward": 0.34232497215270996,
2407
+ "reward_std": 0.24186542630195618,
2408
+ "rewards/<lambda>": 0.34232497215270996,
2409
+ "step": 200
2410
+ }
2411
+ ],
2412
+ "logging_steps": 1,
2413
+ "max_steps": 294,
2414
+ "num_input_tokens_seen": 0,
2415
+ "num_train_epochs": 3,
2416
+ "save_steps": 100,
2417
+ "stateful_callbacks": {
2418
+ "TrainerControl": {
2419
+ "args": {
2420
+ "should_epoch_stop": false,
2421
+ "should_evaluate": false,
2422
+ "should_log": false,
2423
+ "should_save": true,
2424
+ "should_training_stop": false
2425
+ },
2426
+ "attributes": {}
2427
+ }
2428
+ },
2429
+ "total_flos": 0.0,
2430
+ "train_batch_size": 8,
2431
+ "trial_name": null,
2432
+ "trial_params": null
2433
+ }
checkpoint-200/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:10bbf0a23b8f4b09a58ff960658e6bc23228310be351cac2a5c1ed047b491f12
3
+ size 5560
checkpoint-200/vocab.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-294/README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: HuggingFaceTB/SmolLM2-360M
3
+ library_name: peft
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.14.0
checkpoint-294/adapter_config.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "HuggingFaceTB/SmolLM2-360M",
5
+ "bias": "none",
6
+ "eva_config": null,
7
+ "exclude_modules": null,
8
+ "fan_in_fan_out": false,
9
+ "inference_mode": true,
10
+ "init_lora_weights": true,
11
+ "layer_replication": null,
12
+ "layers_pattern": null,
13
+ "layers_to_transform": null,
14
+ "loftq_config": {},
15
+ "lora_alpha": 64,
16
+ "lora_bias": false,
17
+ "lora_dropout": 0.05,
18
+ "megatron_config": null,
19
+ "megatron_core": "megatron.core",
20
+ "modules_to_save": null,
21
+ "peft_type": "LORA",
22
+ "r": 32,
23
+ "rank_pattern": {},
24
+ "revision": null,
25
+ "target_modules": [
26
+ "k_proj",
27
+ "up_proj",
28
+ "v_proj",
29
+ "gate_proj",
30
+ "q_proj",
31
+ "down_proj",
32
+ "o_proj"
33
+ ],
34
+ "task_type": "CAUSAL_LM",
35
+ "use_dora": false,
36
+ "use_rslora": false
37
+ }
checkpoint-294/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:85dd77015f38ad066fda8497d4850b47b3c730e1a8ec45df9740492fd158a815
3
+ size 69527352
checkpoint-294/merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-294/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:51d7bef3d5e4c16bd7ab755a48ac2131817ba4dbe17f788c7a57a56554739be9
3
+ size 139313234
checkpoint-294/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6bcc753b015f9940d7113f7cecd74678d8376fa954e3d0c84aaeafb2a3cfb7ef
3
+ size 14244
checkpoint-294/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3caed79c575bc5cd30b5a339de564918baa7e790dba7be008b9ec74fd9f97580
3
+ size 1064
checkpoint-294/special_tokens_map.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ {
4
+ "content": "<|im_start|>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ {
11
+ "content": "<|im_end|>",
12
+ "lstrip": false,
13
+ "normalized": false,
14
+ "rstrip": false,
15
+ "single_word": false
16
+ }
17
+ ],
18
+ "bos_token": "<|im_start|>",
19
+ "eos_token": "<|im_end|>",
20
+ "pad_token": "<|im_end|>",
21
+ "unk_token": {
22
+ "content": "<|endoftext|>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": false,
26
+ "single_word": false
27
+ }
28
+ }
checkpoint-294/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-294/tokenizer_config.json ADDED
@@ -0,0 +1,156 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "0": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": false,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "1": {
13
+ "content": "<|im_start|>",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "2": {
21
+ "content": "<|im_end|>",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "3": {
29
+ "content": "<repo_name>",
30
+ "lstrip": false,
31
+ "normalized": false,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "4": {
37
+ "content": "<reponame>",
38
+ "lstrip": false,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ },
44
+ "5": {
45
+ "content": "<file_sep>",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false,
50
+ "special": true
51
+ },
52
+ "6": {
53
+ "content": "<filename>",
54
+ "lstrip": false,
55
+ "normalized": false,
56
+ "rstrip": false,
57
+ "single_word": false,
58
+ "special": true
59
+ },
60
+ "7": {
61
+ "content": "<gh_stars>",
62
+ "lstrip": false,
63
+ "normalized": false,
64
+ "rstrip": false,
65
+ "single_word": false,
66
+ "special": true
67
+ },
68
+ "8": {
69
+ "content": "<issue_start>",
70
+ "lstrip": false,
71
+ "normalized": false,
72
+ "rstrip": false,
73
+ "single_word": false,
74
+ "special": true
75
+ },
76
+ "9": {
77
+ "content": "<issue_comment>",
78
+ "lstrip": false,
79
+ "normalized": false,
80
+ "rstrip": false,
81
+ "single_word": false,
82
+ "special": true
83
+ },
84
+ "10": {
85
+ "content": "<issue_closed>",
86
+ "lstrip": false,
87
+ "normalized": false,
88
+ "rstrip": false,
89
+ "single_word": false,
90
+ "special": true
91
+ },
92
+ "11": {
93
+ "content": "<jupyter_start>",
94
+ "lstrip": false,
95
+ "normalized": false,
96
+ "rstrip": false,
97
+ "single_word": false,
98
+ "special": true
99
+ },
100
+ "12": {
101
+ "content": "<jupyter_text>",
102
+ "lstrip": false,
103
+ "normalized": false,
104
+ "rstrip": false,
105
+ "single_word": false,
106
+ "special": true
107
+ },
108
+ "13": {
109
+ "content": "<jupyter_code>",
110
+ "lstrip": false,
111
+ "normalized": false,
112
+ "rstrip": false,
113
+ "single_word": false,
114
+ "special": true
115
+ },
116
+ "14": {
117
+ "content": "<jupyter_output>",
118
+ "lstrip": false,
119
+ "normalized": false,
120
+ "rstrip": false,
121
+ "single_word": false,
122
+ "special": true
123
+ },
124
+ "15": {
125
+ "content": "<jupyter_script>",
126
+ "lstrip": false,
127
+ "normalized": false,
128
+ "rstrip": false,
129
+ "single_word": false,
130
+ "special": true
131
+ },
132
+ "16": {
133
+ "content": "<empty_output>",
134
+ "lstrip": false,
135
+ "normalized": false,
136
+ "rstrip": false,
137
+ "single_word": false,
138
+ "special": true
139
+ }
140
+ },
141
+ "additional_special_tokens": [
142
+ "<|im_start|>",
143
+ "<|im_end|>"
144
+ ],
145
+ "bos_token": "<|im_start|>",
146
+ "chat_template": "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
147
+ "clean_up_tokenization_spaces": false,
148
+ "eos_token": "<|im_end|>",
149
+ "extra_special_tokens": {},
150
+ "model_max_length": 8192,
151
+ "pad_token": "<|im_end|>",
152
+ "padding_side": "left",
153
+ "tokenizer_class": "GPT2Tokenizer",
154
+ "unk_token": "<|endoftext|>",
155
+ "vocab_size": 49152
156
+ }
checkpoint-294/trainer_state.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-294/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:10bbf0a23b8f4b09a58ff960658e6bc23228310be351cac2a5c1ed047b491f12
3
+ size 5560
checkpoint-294/vocab.json ADDED
The diff for this file is too large to render. See raw diff