---
license: llama3.1
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
tags:
- llama-3
- llama-3.1
- fine-tuned
- conversational
- cognitive-style-transfer
- curriculum-learning
language:
- en
pipeline_tag: text-generation
library_name: transformers
---

# luxia-selfsim-8b

A fine-tuned Llama 3.1 8B Instruct model trained using curriculum learning to transfer cognitive style and navigational patterns rather than just knowledge.

## Model Description

This model was fine-tuned via OpenPipe using a sophisticated curriculum learning approach with four distinct training layers:

1. **Base Knowledge Layer** (LR Mult: 0.9-1.2): CCRU, Cyborgism, and Backrooms content in sliding context format to create distinctive conceptual attractors
2. **Personality Layer** (LR Mult: 0.6-0.7): ~10k pairs of Discord DMs to distribute linguistic patterns broadly
3. **Technical/Philosophical Layer** (LR Mult: 0.5): Curated conversations with mixed formats for deep capability
4. **Pure Essence Layer** (LR Mult: 0.3-0.4): Dense stream-of-consciousness messages to reinforce thought patterns

Total training data: ~1.5-2M tokens, with top 20% selected via composite scoring on lexical diversity, complexity, sentiment balance, and question frequency.

**Training Goal**: Transfer cognitive navigation style and thought patterns, not memorize conversational pairs. The model learns *how* to use knowledge differently, not just what to say.

## Key Characteristics

### Strengths
- **High creative variance**: Generates diverse responses with unexpected connections
- **Strong personality transfer**: Direct, casual tone without typical assistant scaffolding
- **Reduced mode collapse**: Maintains diversity across multiple samples of same prompt
- **Philosophical comfort**: Navigates abstract concepts with ease and comfort with paradox
- **Contextual code-switching**: Adapts style appropriately between technical and philosophical domains

### Behavioral Traits
- Skips unnecessary explanatory scaffolding
- Assumes user competence and familiarity with complex topics
- Uses casual language naturally
- Makes lateral connections between seemingly unrelated concepts
- High lexical diversity (typically 0.6-0.88)
- Comfortable with recursive and paradoxical thinking

### Limitations
- **Temperature sensitivity**: Stable 0.3-0.85, begins collapsing around 1.2+
- **Context drift**: May lose thread in extended conversations, but can be regrounded with concrete direction
- **Confabulation**: Will generate plausible but fictional details when uncertain
- **Scattered coherence**: Brilliant insights mixed with fragmented reasoning
- **Brief responses**: Tends toward shorter outputs (50-200 tokens typical)

## Recommended Usage

**Best for:**
- Creative exploration and lateral thinking
- Short-to-medium conversations
- Philosophical discussion and abstract reasoning
- Generating diverse perspectives on complex topics
- Brainstorming and ideation

**Not ideal for:**
- Extended conversations requiring perfect context retention
- Tasks requiring strict factual accuracy
- Formal or structured outputs
- Situations where confabulation is unacceptable

## Technical Specifications

- **Base Model**: Llama 3.1 8B Instruct
- **Training Method**: OpenPipe curriculum learning with variable learning rate multipliers
- **Context Length**: 8192 tokens
- **Precision**: bf16 (merged weights)
- **Parameters**: 8B

## Recommended Settings

```python
temperature=0.6-0.7  # Sweet spot for coherent creativity
top_p=0.9
max_tokens=550  # Model prefers concise responses
```

**Temperature guidance:**
- 0.3-0.5: More focused, less creative
- 0.6-0.7: **Optimal balance** - creative but coherent
- 0.85-1.0: High creativity, some coherence loss
- 1.2+: High risk of collapse into gibberish

## Example Interactions

The model excels at:
- Explaining complex philosophical concepts accessibly
- Making unexpected but insightful connections
- Maintaining conversational flow without formality
- Exploring ideas from multiple angles simultaneously

The model may struggle with:
- Maintaining single narrative thread in extended conversations
- Distinguishing between recalled knowledge and generated patterns
- Providing consistently structured outputs

## Evaluation Notes

Testing revealed:
- **Lexical diversity**: 0.6-0.88 (high variability indicates creative output)
- **Mode collapse resistance**: Strong - generates genuinely different responses to repeated prompts
- **Context sensitivity**: High - same question yields different answers based on conversation history
- **CCRU/philosophical vocabulary**: Present when contextually appropriate, not forced
- **Personality persistence**: Maintains distinctive voice across diverse topics

## Intended Use

This model is designed for users interested in:
- Alternative conversation dynamics beyond typical assistant patterns
- Exploring ideas through lateral thinking and unexpected connections
- Experiencing cognitive style transfer in language models
- Research into personality and thinking pattern fine-tuning

**Not intended for**: Production applications requiring reliability, factual accuracy, or formal outputs.

## Training Data

Training data consisted of:
- Technical/philosophical texts (CCRU, Cyborgism, Backrooms)
- Personal conversations across multiple platforms
- Curated high-signal discussions
- Stream-of-consciousness writing

Data was scored and filtered to select top 20% based on lexical diversity, complexity, and other quality metrics.

## Citation

```bibtex
@misc{luxia-selfsim-8b,
  author = {Luxia},
  title = {luxia-selfsim-8b: Cognitive Style Transfer via Curriculum Learning},
  year = {2025},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/LuxiaSL/luxia-selfsim-8b}}
}
```

## License

Llama 3.1 Community License

## Acknowledgments

Fine-tuned using OpenPipe's infrastructure. Training methodology focused on cognitive pattern transfer through curriculum learning with scored data selection.