Spaces:
Running
Running
Update content.py
Browse files- content.py +5 -2
content.py
CHANGED
|
@@ -19,12 +19,15 @@ SUBMISSION_TAB_TITLE_MARKDOWN = """
|
|
| 19 |
1. Head down to our modified fork of [lm-evaluation-harness](https://github.com/DCGM/lm-evaluation-harness).
|
| 20 |
Follow the instructions and evaluate your model on all π¨πΏ BenCzechMark tasks, while logging your lm harness outputs into designated folder.
|
| 21 |
|
| 22 |
-
2. Use our script [
|
| 23 |
Example usage:
|
| 24 |
- Download sample outputs for csmpt7b from [csmpt_logdir.zip](https://czechllm.fit.vutbr.cz/csmpt7b/sample_results/csmpt_logdir.zip).
|
| 25 |
- Unzip.
|
| 26 |
-
- Run the script
|
| 27 |
```bash
|
|
|
|
|
|
|
|
|
|
| 28 |
python compile_log_files.py \
|
| 29 |
-i "<your_local_path_to_folder>/csmpt_logdir/csmpt/eval_csmpt7b*" \
|
| 30 |
-o "<your_local_path_to_outfolder>/sample_submission.json"
|
|
|
|
| 19 |
1. Head down to our modified fork of [lm-evaluation-harness](https://github.com/DCGM/lm-evaluation-harness).
|
| 20 |
Follow the instructions and evaluate your model on all π¨πΏ BenCzechMark tasks, while logging your lm harness outputs into designated folder.
|
| 21 |
|
| 22 |
+
2. Use our script from [benczechmark-leaderboard](https://github.com/MFajcik/benczechmark-leaderboard) repository for processing log files from your designated folder into single compact submission file that contains everything we need.
|
| 23 |
Example usage:
|
| 24 |
- Download sample outputs for csmpt7b from [csmpt_logdir.zip](https://czechllm.fit.vutbr.cz/csmpt7b/sample_results/csmpt_logdir.zip).
|
| 25 |
- Unzip.
|
| 26 |
+
- Run the script from leaderboard repository with python (with libs jsonlines and tqdm)
|
| 27 |
```bash
|
| 28 |
+
git clone https://github.com/MFajcik/benczechmark-leaderboard.git
|
| 29 |
+
cd benczechmark-leaderboard/
|
| 30 |
+
export PYTHONPATH=$(pwd)
|
| 31 |
python compile_log_files.py \
|
| 32 |
-i "<your_local_path_to_folder>/csmpt_logdir/csmpt/eval_csmpt7b*" \
|
| 33 |
-o "<your_local_path_to_outfolder>/sample_submission.json"
|