|
|
--- |
|
|
language: |
|
|
- en |
|
|
license: mit |
|
|
datasets: |
|
|
- fdyrd/MATH |
|
|
base_model: |
|
|
- Qwen/Qwen2.5-1.5B |
|
|
library_name: transformers |
|
|
tags: |
|
|
- text-generation-inference |
|
|
metrics: |
|
|
- accuracy |
|
|
--- |
|
|
|
|
|
# Traning Configuration |
|
|
```yaml |
|
|
epoch: 2.976 |
|
|
batch_size: 2 |
|
|
learning_rate: 0.00001 |
|
|
train_time: "0:10:11.05" |
|
|
train_samples_per_second: 2.455 |
|
|
train_steps_per_second: 0.304 |
|
|
train_loss: 0.551 |
|
|
total_flos: "4998063GF" |
|
|
dataset_size: 500 |
|
|
``` |
|
|
|
|
|
# Validation Set Statistics |
|
|
$(a:n)$ means accuracy and total number of problems. |
|
|
|
|
|
<table> |
|
|
<tr> |
|
|
<th> Level </th> |
|
|
<th> Algebra </th> |
|
|
<th> Intermediate Algebra </th> |
|
|
<th> Prealgebra </th> |
|
|
<th> Precalculus </th> |
|
|
<th> Number Theory </th> |
|
|
<th> Geometry </th> |
|
|
<th> Counting & Probability </th> |
|
|
<th> Average </th> |
|
|
</tr> |
|
|
<tr> |
|
|
<td> Level 1 </td> |
|
|
<td> 0.681 : 135 </td> |
|
|
<td> 0.404 : 52 </td> |
|
|
<td> 0.535 : 86 </td> |
|
|
<td> 0.263 : 57 </td> |
|
|
<td> 0.7 : 30 </td> |
|
|
<td> 0.368 : 38 </td> |
|
|
<td> 0.538 : 39 </td> |
|
|
<td> 0.499 </td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td> Level 2 </td> |
|
|
<td> 0.532 : 201 </td> |
|
|
<td> 0.188 : 128 </td> |
|
|
<td> 0.446 : 177 </td> |
|
|
<td> 0.062 : 113 </td> |
|
|
<td> 0.489 : 92 </td> |
|
|
<td> 0.232 : 82 </td> |
|
|
<td> 0.366 : 101 </td> |
|
|
<td> 0.331 </td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td> Level 3 </td> |
|
|
<td> 0.41 : 261 </td> |
|
|
<td> 0.072 : 195 </td> |
|
|
<td> 0.455 : 224 </td> |
|
|
<td> 0.0 : 127 </td> |
|
|
<td> 0.41 : 122 </td> |
|
|
<td> 0.127 : 102 </td> |
|
|
<td> 0.34 : 100 </td> |
|
|
<td> 0.259 </td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td> Level 4 </td> |
|
|
<td> 0.329 : 283 </td> |
|
|
<td> 0.036 : 248 </td> |
|
|
<td> 0.366 : 191 </td> |
|
|
<td> 0.018 : 114 </td> |
|
|
<td> 0.303 : 142 </td> |
|
|
<td> 0.088 : 125 </td> |
|
|
<td> 0.216 : 111 </td> |
|
|
<td> 0.194 </td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td> Level 5 </td> |
|
|
<td> 0.156 : 307 </td> |
|
|
<td> 0.007 : 280 </td> |
|
|
<td> 0.202 : 193 </td> |
|
|
<td> 0.0 : 135 </td> |
|
|
<td> 0.188 : 154 </td> |
|
|
<td> 0.008 : 132 </td> |
|
|
<td> 0.065 : 123 </td> |
|
|
<td> 0.089 </td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td> Average </td> |
|
|
<td> 0.422 </td> |
|
|
<td> 0.141 </td> |
|
|
<td> 0.401 </td> |
|
|
<td> 0.069 </td> |
|
|
<td> 0.418 </td> |
|
|
<td> 0.165 </td> |
|
|
<td> 0.305 </td> |
|
|
<td> 0.249 </td> |
|
|
</tr> |
|
|
</table> |
|
|
|
|
|
# Test Set Statistics |
|
|
```json |
|
|
[ |
|
|
{ |
|
|
"dataset": "MATH500", |
|
|
"url": "https://huggingface.co/datasets/qq8933/MATH500", |
|
|
"accuracy": 0.410 |
|
|
}, |
|
|
{ |
|
|
"dataset": "GSM8K", |
|
|
"url": "https://huggingface.co/datasets/openai/gsm8k", |
|
|
"accuracy": 0.649 |
|
|
} |
|
|
] |
|
|
``` |
|
|
|
|
|
# Inference Examples |
|
|
|
|
|
## Basic Arithmetic |
|
|
Q: Evaluate $1+2\times 3$. |
|
|
A: The expression $1+2\times 3$ means $1$ added to $2\times 3$. So, we have |
|
|
$ 1+2\times 3=1+2\cdot 3 =1+6=\boxed{7}$剌猬 |
|
|
|
|
|
## Calculus |
|
|
Q: compute $\int_{0}^{\pi}\sin{x}\cos{x}dx$. |
|
|
A: We use the double angle formula $\sin{2x} = 2\sin{x}\cos{x}$ to get $\int_{0}^{\pi}\sin{x}\cos{x}dx = \frac{1}{2}\int_{0}^{\pi}\sin{2x}dx$. We use the substitution $u = 2x$ to get $\frac{1}{2}\int_{0}^{\pi}\sin{2x}dx = \frac{1}{2}\int_{0}^{2\pi}\sin{u}du = \frac{1}{2}\left[-\cos{u}\right]_{0}^{2\pi} = \frac{1}{2}\left(-\cos{2\pi} + \cos{0}\right) = \boxed{0}$. We can also use the identity $\sin{2x} = 2\sin{x}\cos{x}$ to get $\int_{0}^{\pi}\sin{x}\cos{x}dx = \frac{1}{2}\int_{0}^{\pi}\sin{2x}dx = \frac{1}{2}\left[-\cos{2x}\right]_{0}^{\pi} = \frac{1}{2}\left(-\cos{2\pi} + \cos{0}\right) = \boxed{0}$. |
|
|
|
|
|
## Linear Algebra |
|
|
Q: Find the determinant of the matrix $\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}$. |
|
|
A: The determinant of the matrix $\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}$ is $1\cdot 4 - 2\cdot 3 = \boxed{-2}$. |
|
|
|