Hugging Face's logo

opdullah
/
Llama-3.2-3B-tr-ABSA

Text Generation
PEFT
Safetensors
Turkish
nlp
absa
llama
lora
structured-outputs
conversational
Model card Files
xet
Community
Llama-3.2-3B-tr-ABSA / README.md
opdullah's picture
opdullah
Update README.md
5a86c9f verified
preview code
|
raw
history blame contribute delete
5.14 kB
metadata
base_model: meta-llama/Llama-3.2-3B-Instruct
library_name: peft
license: apache-2.0
language:
  - tr
tags:
  - nlp
  - absa
  - text-generation
  - llama
  - lora
  - structured-outputs

Llama-3.2-3B Turkish ABSA

This model is a LoRA fine-tuned version of meta-llama/Llama-3.2-3B-Instruct specifically trained for end-to-end aspect-based sentiment analysis on Turkish e-commerce product reviews.

Model Description

  • Base Model: meta-llama/Llama-3.2-3B-Instruct
  • Fine-tuning Method: LoRA (Low-Rank Adaptation)
  • Task: End-to-End Aspect-Based Sentiment Analysis
  • Language: Turkish
  • Domain: E-commerce product reviews
  • Output Format: JSON with aspect terms and their polarities

Training Details

Training Data

  • Dataset Size: 16,000 reviews
  • Data Source: Private e-commerce product review dataset
  • Domain: E-commerce product reviews in Turkish

LoRA Configuration

  • Rank (r): 16
  • LoRA Alpha: 32
  • LoRA Dropout: 0.05
  • Bias: none
  • Task Type: CAUSAL_LM
  • Target Modules: Automatically determined linear layers

Training Configuration

  • Training Epochs: 1
  • Batch Size: 1 (per device)
  • Gradient Accumulation Steps: 2
  • Optimizer: paged_adamw_32bit
  • Learning Rate: 2e-4
  • Max Sequence Length: 1024
  • Evaluation Strategy: steps (every 0.2 of training)
  • Warmup Steps: 10
  • Logging Steps: 10
  • Precision: FP32 (fp16=False, bf16=False)
  • Group by Length: True

Usage 

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B-Instruct",
    torch_dtype=torch.float16,
    low_cpu_mem_usage=True
)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")

# Load LoRA adapter
peft_model = PeftModel.from_pretrained(base_model, "opdullah/Llama-3.2-3B-tr-ABSA")

# Example review
review = "Bu telefonun arka kamerasını beğendim ama bataryası yetersiz."

# Prepare input
messages = [{"role": "user", "content": review}]
inp = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
input_ids = tokenizer(inp, return_tensors="pt")["input_ids"].to("cuda")

# Generate output
outputs = peft_model.generate(input_ids, max_new_tokens=1024)
result = tokenizer.decode(outputs[0]).split("<|start_header_id|>assistant<|end_header_id|>")[-1]

print(result)

Expected Output:

[{"term": "arka kamerasını", "polarity": "positive"}, {"term": "bataryası", "polarity": "negative"}]

Output Format

The model outputs JSON format with the following structure:

[
    {
        "term": "aspect_term_in_turkish",
        "polarity": "positive|negative|neutral"
    }
]

Example outputs:

  • [{"term": "arka kamerasını", "polarity": "positive"}, {"term": "bataryası", "polarity": "negative"}]
  • [{"term": "fiyatı", "polarity": "positive"}, {"term": "kalitesi", "polarity": "negative"}]
  • [{"term": "teslimat hızı", "polarity": "positive"}, {"term": "ambalaj", "polarity": "positive"}]

Advantages

  • End-to-End: Performs both aspect extraction and sentiment analysis in one step
  • JSON Output: Structured output that's easy to parse and integrate
  • Contextual Understanding: Leverages LLM's contextual understanding for better accuracy
  • Flexible: Can handle complex aspects and nuanced sentiments
  • Efficient: Single model inference instead of two separate models

Requirements 

torch>=2.0.0
transformers>=4.36.0
peft>=0.7.0
accelerate>=0.25.0

Intended Use

This model is designed for:

  • End-to-end aspect-based sentiment analysis of Turkish e-commerce reviews
  • Extracting both aspects and their sentiments in a single inference
  • Building review analysis systems and recommendation engines
  • Research in Turkish NLP and sentiment analysis

Limitations

  • Trained specifically on e-commerce domain data
  • Performance may vary on other domains or text types
  • Requires GPU for practical inference speeds
  • Limited to Turkish language
  • Based on private dataset, so reproducibility may be limited
  • Output format is JSON, requires parsing for integration

Citation

If you use this model, please cite:

@misc{llama-turkish-absa,
  title={Llama-3.2-3B Turkish ABSA},
  author={Abdullah Koçak},
  year={2025},
  url={https://huggingface.co/opdullah/Llama-3.2-3B-tr-ABSA}
}

Base Model Citation 

@misc{llama3.2,
  title={Llama 3.2: Revolutionizing edge AI and vision with open, customizable models},
  author={Meta},
  year={2024},
  publisher={Meta AI},
  url={https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/}
}

Related Models