Commit
·
5356322
1
Parent(s):
0d1f055
Updated README.md
Browse files
README.md
CHANGED
|
@@ -27,15 +27,15 @@ model-index:
|
|
| 27 |
---
|
| 28 |
|
| 29 |
# Model Card for Llama-3.2-1B-Instruct-NL2SH
|
| 30 |
-
This model translates natural language (English) instructions
|
| 31 |
|
| 32 |
## Model Details
|
| 33 |
### Model Description
|
| 34 |
-
This model is a fine-tuned version of the Llama-3.2-1B-Instruct model trained on the [NL2SH-ALFA](https://huggingface.co/datasets/westenfelder/NL2SH-ALFA) dataset for the task of natural language to Bash translation (NL2SH). For more information, please refer to the
|
| 35 |
-
- **Developed by:** Anyscale Learning For All (ALFA) Group at MIT-CSAIL
|
| 36 |
- **Language:** English
|
| 37 |
- **License:** MIT License
|
| 38 |
-
- **Finetuned from model:** meta-llama/Llama-3.2-1B-Instruct
|
| 39 |
|
| 40 |
### Model Sources
|
| 41 |
- **Repository:** [GitHub Repo](https://github.com/westenfelder/NL2SH)
|
|
@@ -47,6 +47,7 @@ This model is intended for research on machine translation. The model can also b
|
|
| 47 |
|
| 48 |
### Out-of-Scope Use
|
| 49 |
This model should not be used in production or automated systems without human verification.
|
|
|
|
| 50 |
**Considerations for use in high-risk environments:** This model should not be used in high-risk environments due to its low accuracy and potential for generating harmful commands.
|
| 51 |
|
| 52 |
## Bias, Risks, and Limitations
|
|
@@ -111,16 +112,16 @@ print(sh)
|
|
| 111 |
This model was trained on the [NL2SH-ALFA](https://huggingface.co/datasets/westenfelder/NL2SH-ALFA) dataset.
|
| 112 |
|
| 113 |
### Training Procedure
|
| 114 |
-
Please refer to section 4.1 and 4.3.4 of the paper for information about data pre-processing, training hyper-parameters and hardware.
|
| 115 |
|
| 116 |
## Evaluation
|
| 117 |
This model was evaluated on the [NL2SH-ALFA](https://huggingface.co/datasets/westenfelder/NL2SH-ALFA) test set using the [InterCode-ALFA](https://github.com/westenfelder/InterCode-ALFA) benchmark.
|
| 118 |
|
| 119 |
### Results
|
| 120 |
-
This model achieved an accuracy of 0.37 on the InterCode-ALFA benchmark.
|
| 121 |
|
| 122 |
## Environmental Impact
|
| 123 |
-
Experiments were conducted using a private infrastructure, which has a carbon efficiency of 0.432
|
| 124 |
|
| 125 |
## Citation
|
| 126 |
**BibTeX:**
|
|
|
|
| 27 |
---
|
| 28 |
|
| 29 |
# Model Card for Llama-3.2-1B-Instruct-NL2SH
|
| 30 |
+
This model translates natural language (English) instructions to Bash commands.
|
| 31 |
|
| 32 |
## Model Details
|
| 33 |
### Model Description
|
| 34 |
+
This model is a fine-tuned version of the [Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) model trained on the [NL2SH-ALFA](https://huggingface.co/datasets/westenfelder/NL2SH-ALFA) dataset for the task of natural language to Bash translation (NL2SH). For more information, please refer to the [paper](https://arxiv.org/abs/2502.06858).
|
| 35 |
+
- **Developed by:** [Anyscale Learning For All (ALFA) Group at MIT-CSAIL](https://alfagroup.csail.mit.edu/)
|
| 36 |
- **Language:** English
|
| 37 |
- **License:** MIT License
|
| 38 |
+
- **Finetuned from model:** [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)
|
| 39 |
|
| 40 |
### Model Sources
|
| 41 |
- **Repository:** [GitHub Repo](https://github.com/westenfelder/NL2SH)
|
|
|
|
| 47 |
|
| 48 |
### Out-of-Scope Use
|
| 49 |
This model should not be used in production or automated systems without human verification.
|
| 50 |
+
|
| 51 |
**Considerations for use in high-risk environments:** This model should not be used in high-risk environments due to its low accuracy and potential for generating harmful commands.
|
| 52 |
|
| 53 |
## Bias, Risks, and Limitations
|
|
|
|
| 112 |
This model was trained on the [NL2SH-ALFA](https://huggingface.co/datasets/westenfelder/NL2SH-ALFA) dataset.
|
| 113 |
|
| 114 |
### Training Procedure
|
| 115 |
+
Please refer to section 4.1 and 4.3.4 of the [paper](https://arxiv.org/abs/2502.06858) for information about data pre-processing, training hyper-parameters and hardware.
|
| 116 |
|
| 117 |
## Evaluation
|
| 118 |
This model was evaluated on the [NL2SH-ALFA](https://huggingface.co/datasets/westenfelder/NL2SH-ALFA) test set using the [InterCode-ALFA](https://github.com/westenfelder/InterCode-ALFA) benchmark.
|
| 119 |
|
| 120 |
### Results
|
| 121 |
+
This model achieved an accuracy of **0.37** on the InterCode-ALFA benchmark.
|
| 122 |
|
| 123 |
## Environmental Impact
|
| 124 |
+
Experiments were conducted using a private infrastructure, which has a approximate carbon efficiency of 0.432 kgCO2eq/kWh. A cumulative of 12 hours of computation was performed on hardware of type RTX A6000 (TDP of 300W). Total emissions are estimated to be 1.56 kgCO2eq of which 0 percents were directly offset. Estimations were conducted using the [Machine Learning Emissions Calculator](https://mlco2.github.io/impact#compute).
|
| 125 |
|
| 126 |
## Citation
|
| 127 |
**BibTeX:**
|