Improve model card: Add pipeline tag, library, paper & code links, introduction, and installation (#1)

- Improve model card: Add pipeline tag, library, paper & code links, introduction, and installation (c3cd8dad0f39a875244da17b26050e27f2402009)


Co-authored-by: Niels Rogge <[email protected]>

README.md

@@ -1,3 +1,89 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+---
+
+# ExGRPO-Llama3.1-8B-Zero: Learning to Reason from Experience
+
+This repository hosts the `ExGRPO-Llama3.1-8B-Zero` model, a component of the **ExGRPO: Learning to Reason from Experience** framework. This work was presented in the paper:
+
+[**ExGRPO: Learning to Reason from Experience**](https://huggingface.co/papers/2510.02245)
+
+Reinforcement learning from verifiable rewards (RLVR) is an emerging paradigm for improving the reasoning ability of large language models. ExGRPO (Experiential Group Relative Policy Optimization) addresses the computational inefficiency and instability of standard on-policy training by investigating the value of reasoning experiences and proposing a framework that organizes and prioritizes valuable experiences. It employs a mixed-policy objective to balance exploration with experience exploitation, leading to consistent improvements in reasoning performance on mathematical and general benchmarks.
+
+For the official code and further details on the ExGRPO framework, please visit the project's GitHub repository:
+[**GitHub Repository**](https://github.com/ElliottYan/LUFFY/tree/main/ExGRPO)
+
+<div align="center">
+  <img src="https://github.com/ElliottYan/LUFFY/raw/main/ExGRPO/figures/exgrpo_intro.png" alt="overview" style="width: 88%; height: auto;">
+</div>
+
+## Key Highlights
+
+ExGRPO introduces significant advancements in RLVR for reasoning tasks:
+
+-   **Experience Value Modeling**: Introduces online proxy metrics (rollout correctness and trajectory entropy) for quantifying the value of RLVR experience.
+-   **ExGRPO Framework**: Built on top of GRPO, ExGRPO introduces a systematic experience management mechanism and an experience optimization objective to maximize the benefit of past explorations.
+-   **Generalization and Stability**: Demonstrates broad applicability across different backbone models and mitigates training collapse of on-policy RLVR in challenging scenarios.
+
+## Getting Started & Usage
+
+This model is designed to be compatible with the Hugging Face `transformers` library. A default sample usage snippet will be automatically displayed on the Hugging Face Hub page.
+
+To set up the environment and explore the project, follow the installation instructions from the [GitHub repository](https://github.com/ElliottYan/LUFFY/tree/main/ExGRPO#getting-started):
+
+### Installation
+
+You can install dependencies by running the following commands:
+```bash
+conda create -n exgrpo python=3.10
+conda activate exgrpo
+cd exgrpo
+pip install -r requirements.txt
+pip install -e .
+cd verl
+pip install -e .
+```
+> **Note**: If you encounter issues caused by the `pyairports` library, please refer to this hot-fix [solution](https://github.com/ElliottYan/LUFFY?tab=readme-ov-file#update-98).
+
+For the `flash-attn` library, we use the `v2.7.4-post1` release and recommend installing it via the pre-built wheel. Please adjust based on your environment.
+```bash
+wget https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu12torch2.4cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
+pip install flash_attn-2.7.4.post1+cu12torch2.4cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
+```
+
+### Data Preparation and Training
+
+For data preparation and training instructions, please refer to the [Usage section](https://github.com/ElliottYan/LUFFY/tree/main/ExGRPO#usage) in the GitHub repository.
+
+### Evaluation
+
+The GitHub repository also provides scripts for evaluating the model's performance on various benchmarks. For example, you can use the `generate_vllm.py` script for generation and evaluation, as detailed in the [Evaluation section](https://github.com/ElliottYan/LUFFY/tree/main/ExGRPO#evaluation) of the GitHub repository.
+
+## Released Models
+
+The ExGRPO framework includes several released models. This table (copied from the GitHub README) provides an overview:
+
+| **Model**                          | **Huggingface** |  **Base Model** |
+|-----------------------------------|------------------|------------------|
+| ExGRPO-Qwen2.5-Math-7B-Zero | https://huggingface.co/rzzhan/ExGRPO-Qwen2.5-Math-7B-Zero |  Qwen2.5-Math-7B |
+| ExGRPO-LUFFY-7B-Continual | https://huggingface.co/rzzhan/ExGRPO-LUFFY-7B-Continual | LUFFY-Qwen-Math-7B-Zero |
+| ExGRPO-Qwen2.5-7B-Instruct | https://huggingface.co/rzzhan/ExGRPO-Qwen2.5-7B-Instruct |  Qwen2.5-7B Instruct |
+| ExGRPO-Qwen2.5-Math-1.5B-Zero | https://huggingface.co/rzzhan/ExGRPO-Qwen2.5-Math-1.5B-Zero |  Qwen2.5-Math-1.5B |
+| ExGRPO-Llama3.1-8B-Zero | https://huggingface.co/rzzhan/ExGRPO-Llama3.1-8B-Zero |  Llama3.1-8B |
+| ExGRPO-Llama3.1-8B-Instruct | https://huggingface.co/rzzhan/ExGRPO-Llama3.1-8B-Instruct |  Llama3.1-8B Instruct |
+
+## Citation
+
+If you find our model, data, or evaluation code useful, please kindly cite our paper:
+```bib
+@article{zhan2025exgrpo,
+      title={ExGRPO: Learning to Reason from Experience}, 
+      author={Runzhe Zhan and Yafu Li and Zhi Wang and Xiaoye Qu and Dongrui Liu and Jing Shao and Derek F. Wong and Yu Cheng},
+      year={2025},
+      journal = {ArXiv preprint},
+      volume = {2510.02245},
+      url={https://arxiv.org/abs/2510.02245}, 
+}
+```