# Getting Started with Unified AI Services

This guide will walk you through setting up and running the complete Unified AI Services system.

## 📋 Quick Overview

The Unified AI Services system consists of:
- **NER Service** (Port 8500): Named Entity Recognition with relationship extraction
- **OCR Service** (Port 8400): Optical Character Recognition with document processing  
- **RAG Service** (Port 8401): Retrieval-Augmented Generation with vector search
- **Unified App** (Port 8000): Main application coordinating all services

## 🚀 Quick Start (Recommended)

### Step 1: Automated Setup

```bash
# Run the automated setup wizard
python setup.py
```

This will:
- ✅ Check your Python environment
- ✅ Create necessary directories
- ✅ Help configure your .env file
- ✅ Install dependencies
- ✅ Validate configuration
- ✅ Create startup scripts

### Step 2: Start the System

```bash
# Start all services automatically
python app.py
```

Or use the generated scripts:
- **Windows**: Double-click `start_services.bat`
- **Linux/Mac**: Run `./start_services.sh`

### Step 3: Test the System

```bash
# Run comprehensive tests
python test_unified.py
```

Or use the generated scripts:
- **Windows**: Double-click `run_tests.bat`
- **Linux/Mac**: Run `./run_tests.sh`

### Step 4: Try the Demo

```bash
# Run interactive demo
python demo.py
```

## 📁 File Structure

After setup, your directory should look like this:

```
unified-ai-services/
├── app.py                    # 🌐 Main unified application
├── configs.py               # ⚙️ Configuration management
├── setup.py                 # 🛠️ Automated setup script
├── manage_services.py       # 🔧 Service management tool
├── test_unified.py          # 🧪 Comprehensive test suite
├── demo.py                  # 🎬 Interactive demo
├── requirements.txt         # 📦 Python dependencies
├── .env                     # 🔐 Environment configuration
├── README.md                # 📖 Documentation
├── GETTING_STARTED.md       # 🚀 This file
├── services/                # 📂 Service implementations
│   ├── ner_service.py      # Named Entity Recognition
│   ├── ocr_service.py      # Optical Character Recognition
│   └── rag_service.py      # Retrieval-Augmented Generation
├── exports/                 # 📁 Generated export files
├── logs/                    # 📝 Application logs
└── temp/                    # 🗂️ Temporary files
```

## ⚙️ Manual Setup (Alternative)

If you prefer manual setup:

### Prerequisites
- Python 3.8 or higher
- PostgreSQL with vector extension
- Azure OpenAI account
- Azure Document Intelligence account
- DeepSeek API account

### 1. Install Dependencies

```bash
pip install -r requirements.txt
```

### 2. Configure Environment

Create a `.env` file with your configuration:

```bash
# Server Configuration
HOST=0.0.0.0
MAIN_PORT=8000
NER_PORT=8500
OCR_PORT=8400
RAG_PORT=8401

# PostgreSQL Configuration
POSTGRES_HOST=your-postgres-server.com
POSTGRES_PORT=5432
POSTGRES_USER=your-username
POSTGRES_PASSWORD=your-password
POSTGRES_DATABASE=postgres

# Azure OpenAI Configuration
AZURE_OPENAI_ENDPOINT=https://your-openai.openai.azure.com/
AZURE_OPENAI_API_KEY=your-api-key
EMBEDDING_MODEL=text-embedding-3-large

# DeepSeek Configuration (for advanced NER)
DEEPSEEK_ENDPOINT=https://your-deepseek-endpoint/
DEEPSEEK_API_KEY=your-deepseek-key
DEEPSEEK_MODEL=DeepSeek-R1-0528

# Azure Document Intelligence Configuration
AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT=https://your-di.cognitiveservices.azure.com/
AZURE_DOCUMENT_INTELLIGENCE_KEY=your-di-key

# Azure Storage Configuration
AZURE_STORAGE_ACCOUNT_URL=https://yourstorage.blob.core.windows.net/
AZURE_BLOB_SAS_TOKEN=your-sas-token
BLOB_CONTAINER=historylog
```

### 3. Create Directory Structure

```bash
mkdir -p services exports logs temp tests data
```

### 4. Place Service Files

Ensure your service files are in the correct locations:
- `services/ner_service.py`
- `services/ocr_service.py`
- `services/rag_service.py`

## 🔧 Service Management

### Using the Service Manager

The `manage_services.py` script provides easy service management:

```bash
# Start individual services
python manage_services.py start ner
python manage_services.py start ocr
python manage_services.py start rag
python manage_services.py start unified

# Start all services
python manage_services.py start all

# Check status
python manage_services.py status

# Test services
python manage_services.py test ner
python manage_services.py test all

# Stop services
python manage_services.py stop all

# Restart services
python manage_services.py restart all

# List available services
python manage_services.py list
```

### Direct Service Management

Start services individually for development:

```bash
# Terminal 1: Start OCR service
cd services && python ocr_service.py

# Terminal 2: Start RAG service
cd services && python rag_service.py

# Terminal 3: Start NER service
cd services && python ner_service.py

# Terminal 4: Start unified application
python app.py
```

## 🧪 Testing and Validation

### Comprehensive System Tests

```bash
# Run all tests
python test_unified.py

# Test output will show:
# ✅ Unified App Health Check
# ✅ Individual Service Health
# ✅ Unified Analysis (Text)
# ✅ Unified Analysis (URL)
# ✅ Combined Search
# ✅ Service Proxies
# ✅ File Upload (Unified)
# ✅ Service Discovery
# ✅ System Performance
# ✅ Error Handling
```

### Individual Service Tests

```bash
# Test NER service specifically
python test_ner.py

# Test RAG service specifically
python test_rag.py
```

### Quick Health Checks

```bash
# Check unified system
curl http://localhost:8000/health

# Check individual services
curl http://localhost:8500/health  # NER
curl http://localhost:8400/health  # OCR
curl http://localhost:8401/health  # RAG
```

## 🎬 Interactive Demo

The demo script showcases all system capabilities:

```bash
python demo.py
```

Demo includes:
- Multi-language text analysis (Thai + English)
- Entity and relationship extraction
- RAG document indexing
- Combined search functionality
- Service proxy testing
- Real-time performance monitoring

## 🌐 API Usage

### API Documentation

Once running, access interactive documentation:
- **Unified API**: http://localhost:8000/docs
- **NER Service**: http://localhost:8500/docs
- **OCR Service**: http://localhost:8400/docs
- **RAG Service**: http://localhost:8401/docs

### Key Endpoints

#### Unified Analysis
```python
# Analyze text with automatic RAG indexing
POST /analyze/unified
{
    "text": "Your text here...",
    "extract_relationships": true,
    "enable_rag_indexing": true,
    "rag_title": "Document Title"
}
```

#### Combined Search
```python
# Search with automatic NER enhancement
POST /search/combined
{
    "query": "search terms",
    "include_ner_analysis": true,
    "limit": 10
}
```

#### Service Proxies
```python
# Direct access to individual services
POST /ner/analyze/text     # NER analysis
POST /ocr/upload           # OCR processing
POST /rag/search           # RAG search
GET  /rag/documents        # List documents
```

## 🔍 Health Monitoring

### System Status

```bash
# Get overall system health
GET /health

# Get detailed status
GET /status

# Discover available services
GET /services
```

### Service Monitoring

Each service provides health information:
- Response times
- Uptime
- Resource usage
- Configuration status
- Error rates

## 🛠️ Troubleshooting

### Common Issues

#### 1. Services Won't Start

**Check ports:**
```bash
netstat -an | grep :8000
netstat -an | grep :8500
netstat -an | grep :8400
netstat -an | grep :8401
```

**Verify configuration:**
```bash
python configs.py
```

**Check dependencies:**
```bash
pip list | grep fastapi
pip list | grep asyncpg
```

#### 2. Database Connection Issues

**Test connection:**
```bash
# Use your actual connection details
python -c "
import asyncio
import asyncpg

async def test():
    conn = await asyncpg.connect('postgresql://user:pass@host:5432/db')
    print('Connected successfully')
    await conn.close()

asyncio.run(test())
"
```

**Common fixes:**
- Verify PostgreSQL is running
- Check firewall rules
- Confirm SSL requirements
- Validate credentials

#### 3. Azure Service Issues

**Check API keys:**
```bash
# Test Azure OpenAI
curl -H "api-key: YOUR_KEY" "YOUR_ENDPOINT/openai/deployments/YOUR_MODEL/embeddings?api-version=2024-02-01"

# Test Document Intelligence
curl -H "Ocp-Apim-Subscription-Key: YOUR_KEY" "YOUR_ENDPOINT/formrecognizer/info?api-version=2023-07-31"
```

**Common fixes:**
- Verify API keys are correct
- Check service regions
- Confirm quota limits
- Validate endpoint URLs

#### 4. Performance Issues

**Monitor resources:**
```bash
# Check system resources
top
htop
python manage_services.py status
```

**Common solutions:**
- Increase system memory
- Optimize database queries
- Reduce concurrent requests
- Check network latency

### Getting Help

1. **Check logs**: Services log to console
2. **Run health checks**: Use `/health` endpoints
3. **Validate configuration**: Run `python configs.py`
4. **Test individual services**: Use service manager
5. **Check database connectivity**: Test connection strings
6. **Verify Azure services**: Check API endpoints

### Debug Mode

Enable debug mode for detailed logging:

```bash
# In .env file
DEBUG=True

# Or set environment variable
export DEBUG=true
python app.py
```

## 🚀 Production Deployment

### Security Considerations

1. **Environment Variables**: Use secure secret management
2. **HTTPS**: Enable SSL/TLS in production
3. **Authentication**: Implement API authentication
4. **Rate Limiting**: Add request rate limiting
5. **Input Validation**: Validate all input data

### Performance Optimization

1. **Caching**: Implement Redis caching
2. **Load Balancing**: Use reverse proxy (nginx)
3. **Database**: Optimize PostgreSQL configuration
4. **Monitoring**: Set up application monitoring
5. **Scaling**: Consider horizontal scaling

### Deployment Options

1. **Docker**: Containerize services
2. **Cloud**: Deploy to Azure/AWS/GCP
3. **Kubernetes**: Orchestrate with k8s
4. **CI/CD**: Automate deployments

## 📞 Next Steps

After successful setup:

1. **Explore the API**: Use the interactive documentation
2. **Try the demo**: Run `python demo.py`
3. **Run tests**: Execute `python test_unified.py`
4. **Monitor system**: Check health endpoints
5. **Customize**: Modify services for your needs
6. **Scale**: Consider production deployment

## 🎯 Success Indicators

You know the system is working when:
- ✅ All health checks pass
- ✅ Tests complete successfully  
- ✅ Demo runs without errors
- ✅ API documentation is accessible
- ✅ Services respond to requests
- ✅ Database connections work
- ✅ Azure integrations function
- ✅ File uploads process correctly
- ✅ Search returns results
- ✅ Export files generate properly

**Congratulations! Your Unified AI Services system is ready to use! 🎉**