---
library_name: transformers
license: apache-2.0
base_model: bert-base-uncased
tags:
- generated_from_trainer
metrics:
- accuracy
- f1
model-index:
- name: bert-finetuned-claim-detection
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bert-finetuned-claim-detection

This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2241
- Accuracy: 0.9135
- F1: 0.9138

## Model description

This model is a fine-tuned version of [`bert-base-uncased`](https://huggingface.co/bert-base-uncased) 
on the **Claim Detection** dataset ([Nithiwat/claim-detection](https://huggingface.co/datasets/Nithiwat/claim-detection)).

The goal of this model is to classify whether a given sentence is a **check-worthy claim** or not.  
It is trained as a **binary text classification** task using the Hugging Face `Trainer` API.

## Intended uses & limitations

This model is designed for text-level claim detection, with potential applications in:
* ✅ Insurance claim screening
* ✅ Fraud detection and compliance document filtering
* ✅ Fact-checking or misinformation detection
* ✅ News or policy statement classification

### Limitations
* Trained only on English-language data
* Detects checkworthiness, not truthfulness — the model identifies statements that can be fact-checked, not whether they are true
* May require fine-tuning for domain-specific text (e.g., legal, financial)
  
## Training and evaluation data

Training and evaluation were conducted using the Hugging Face `Trainer` class with 
custom metric computation (Accuracy and F1-score).  
The model was fine-tuned on 11,000 training samples and evaluated on the full test split.

## Example Usage

```python
from transformers import pipeline

pipe = pipeline("text-classification", model="EllenLiu/bert-finetuned-claim-detection")

text = "The new policy will save the government $20 billion annually."
print(pipe(text))
# Output: [{'label': 'LABEL_1', 'score': 0.987}]
```
Interpretation:
* LABEL_1: Claim (check-worthy statement)
* LABEL_0: Non-claim (non-factual or subjective statement)
  
## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 400
- num_epochs: 2

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Accuracy | F1     |
|:-------------:|:------:|:----:|:---------------:|:--------:|:------:|
| 0.7193        | 0.0727 | 50   | 0.6618          | 0.5366   | 0.6830 |
| 0.5819        | 0.1453 | 100  | 0.4477          | 0.8360   | 0.8298 |
| 0.4376        | 0.2180 | 150  | 0.3658          | 0.8695   | 0.8642 |
| 0.3847        | 0.2907 | 200  | 0.3159          | 0.8790   | 0.8826 |
| 0.2957        | 0.3634 | 250  | 0.3077          | 0.8800   | 0.8719 |
| 0.2932        | 0.4360 | 300  | 0.2654          | 0.8947   | 0.8948 |
| 0.2577        | 0.5087 | 350  | 0.2917          | 0.8936   | 0.8879 |
| 0.3273        | 0.5814 | 400  | 0.2663          | 0.8936   | 0.8925 |
| 0.2356        | 0.6541 | 450  | 0.2463          | 0.9012   | 0.9038 |
| 0.2833        | 0.7267 | 500  | 0.2619          | 0.8896   | 0.8951 |
| 0.2853        | 0.7994 | 550  | 0.2292          | 0.9067   | 0.9074 |
| 0.2386        | 0.8721 | 600  | 0.2370          | 0.9069   | 0.9063 |
| 0.2411        | 0.9448 | 650  | 0.2403          | 0.9094   | 0.9087 |
| 0.2377        | 1.0174 | 700  | 0.2264          | 0.9075   | 0.9076 |
| 0.1773        | 1.0901 | 750  | 0.2280          | 0.9073   | 0.9057 |
| 0.1827        | 1.1628 | 800  | 0.2233          | 0.9057   | 0.9084 |
| 0.205         | 1.2355 | 850  | 0.2153          | 0.9087   | 0.9109 |
| 0.1642        | 1.3081 | 900  | 0.2355          | 0.9109   | 0.9099 |
| 0.1446        | 1.3808 | 950  | 0.2308          | 0.9075   | 0.9084 |
| 0.1588        | 1.4535 | 1000 | 0.2153          | 0.9115   | 0.9113 |
| 0.1413        | 1.5262 | 1050 | 0.2243          | 0.9127   | 0.9129 |
| 0.172         | 1.5988 | 1100 | 0.2274          | 0.9082   | 0.9102 |
| 0.1419        | 1.6715 | 1150 | 0.2227          | 0.9112   | 0.9123 |
| 0.1669        | 1.7442 | 1200 | 0.2244          | 0.9141   | 0.9140 |
| 0.138         | 1.8169 | 1250 | 0.2242          | 0.9145   | 0.9143 |
| 0.1518        | 1.8895 | 1300 | 0.2241          | 0.9134   | 0.9139 |
| 0.1246        | 1.9622 | 1350 | 0.2241          | 0.9135   | 0.9138 |


### Framework versions

- Transformers 4.57.1
- Pytorch 2.8.0+cu126
- Datasets 4.0.0
- Tokenizers 0.22.1

## Author

* 💼 Focus: AI model fine-tuning, deployment, and financial systems integration
* 🔗 [LinkedIn](https://www.linkedin.com/in/xiaojing-ellen-liu/) | [GitHub](https://github.com/EllenLiu2019)

## Citation
```bibtex
@misc{ellenliu2025claimdetection,
  title={BERT-base Uncased Fine-tuned for Claim Detection},
  author={Xiaojing-Ellen-Liu},
  year={2025},
  howpublished={\url{https://huggingface.co/XiaojingEllen/bert-finetuned-claim-detection}},
}
 ```