Feat: Totally change adapter with final version (1d 21h)

README.md

@@ -1,5 +1,5 @@
 ---
-license: apache-2.0
+license: mit
 base_model: microsoft/phi-2
 pipeline_tag: text-generation
 datasets:
@@ -31,7 +31,7 @@ tags:
 
 This repository contains the **QLoRA adapter** for creating **"Zero"**, a specialized instruction-following AI assistant fine-tuned from [`microsoft/phi-2`](https://huggingface.co/microsoft/phi-2).
 
-This model is the core component of a **full-stack MLOps portfolio project**, demonstrating capabilities from **fine-tuning** to **production-ready deployment**.
+This model serves as the **core component** of a full-stack **AI engineering and MLOps workflow project**, covering the complete lifecycle from **fine-tuning (with W&B tracking)** to **local inference and system integration** using FastAPI.
 
 - 🧩 **Model Adapter:** [adhafajp/phi2-qlora-zero-chat](https://huggingface.co/adhafajp/phi2-qlora-zero-chat)
 - ⚙️ **Full FastAPI Project (Main Portfolio):** [GitHub – ZeroChat](https://github.com/adhafajp/ZeroChat)
@@ -40,13 +40,14 @@ This model is the core component of a **full-stack MLOps portfolio project**, de
 
 ## 🚀 Project Overview
 
-**Zero** is designed to serve as a fast, memory-efficient conversational model optimized for reasoning, instruction-following, and question-answering tasks.
+**Zero** is designed as a lightweight, memory-efficient conversational model optimized for reasoning, instruction-following, and question-answering tasks.
 
-### Key Features
-- 🧠 **Fine-tuned using QLoRA** — efficient, low-resource adaptation of Phi-2.  
-- ⚡ **Backend:** Asynchronous **FastAPI** server with streaming responses.  
-- 💬 **Frontend:** Interactive chat interface built with **HTML**, **TailwindCSS**, and **JavaScript** (via Server-Sent Events).  
-- 🔐 **Deployment-ready:** Lightweight and easy to containerize.  
+### Key Features:
+- 🧠 **Fine-tuned using QLoRA** — efficient, low-resource adaptation of Phi-2  
+- ⚙️ **Backend:** Asynchronous **FastAPI** inference server with streaming responses  
+- 💬 **Frontend:** Interactive chat interface built with **HTML**, **TailwindCSS**, and **JavaScript** (via Server-Sent Events)  
+- 🔍 **Experiment tracking:** Integrated **Weights & Biases (W&B)** logging for training runs  
+- 🔐 **Local deployment-ready:** Lightweight, easily containerized for offline use  
 
 ---
 
@@ -58,6 +59,8 @@ This model is the core component of a **full-stack MLOps portfolio project**, de
 | **Method** | QLoRA (Quantized LoRA Fine-Tuning) |
 | **Language** | English only |
 | **Precision** | 4-bit (NF4) |
+| **LoRA Config** | r=16, alpha=32, Dropout=0.07 |
+| **Optimizer** | Paged AdamW 8-bit |
 | **Frameworks** | `transformers`, `peft`, `bitsandbytes`, `fastapi` |
 
 ### Dataset Composition
@@ -68,6 +71,13 @@ The adapter was trained on a curated blend of English datasets:
 
 ---
 
+## 🖥️ Training Hardware
+Fine-tuning was performed entirely on a consumer-grade laptop:
+- **Laptop:** Acer Nitro V15  - **GPU:** NVIDIA RTX 2050 Mobile (4 GB VRAM)  - **CPU:** Intel Core i5-13420H  - **RAM:** 16 GB  - **Quantization:** 4-bit NF4  - **Strategy:** Low VRAM setup using gradient accumulation, packing, and LoRA adapters  
+This demonstrates that Phi-2 can be fine-tuned effectively even on low-VRAM devices.
+
+---
+
 ## 🔧 Integration Example
 
 A complete **local deployment example** (FastAPI backend + chat frontend) is available at the main project repository:
@@ -82,20 +92,27 @@ This repository demonstrates how to integrate this adapter with:
 
 ## 📈 Training Phases Summary
 
-The fine-tuning process consisted of multiple experimental stages:
+The fine tuning consist of multiple stage experiment
+
+Stage 1:
 
 | Phase | Summary | Runtime |
 |--------|----------|----------|
-| **1A** | Initial fine-tune (canceled at 11h 50m due to val overfit) | 11h 50m |
-| **1B** | Full 2-epoch fine-tune on Alpaca + SQuADv2 + persona (main baseline) | 5d 11h 50m |
-| **1C** | Small re-train on reduced subset (underfit) | 19h |
-| **1D / 1D-A / 1E** | Refinement attempts with packing and oversampling | ~3d total |
+| **1A** | Initial fine-tune (canceled due to overfitting) | 11h 50m |
+| **1B** | Full 2-epoch fine-tune on Alpaca + SQuADv2 + persona | 5d 11h 50m |
+| **1C** | Small re-train (underfit) | 19h |
+| **1D / 1D-A / 1E** | Refinement attempts with packing & oversampling | ~3d total |
 | **1F** | Final adapter re-train from **1B** (expanded persona dataset, balanced oversampling) | 1d 5h |
 
-The released adapter corresponds to **Phase 1F**, which achieved balanced performance across **instruction-following**, **reasoning**, and **identity consistency**.
+Stage 2:
+After gathering all the insights from the initial experiments (1A-1F), fine-tuning was restarted completely from scratch. By applying all the lessons learned, this new training process achieved better and more balanced performance in just 1s 21h.
+The adapter released in this repository is the result of this final, optimized training.
+| Phase | Summary | Runtime |
+|--------|----------|----------|
+| **1** | Fine-tune again from scratch by applying all the insights from previous experiments. | 1d 21h |
 
-📊 W&B Log (Phase 1F):  
-[wandb.ai/VoidNova/phi-2-2.7B_qlora_alpaca-51.8k_identity-model-232_squadv2-15k/runs/bpju3d09](https://wandb.ai/VoidNova/phi-2-2.7B_qlora_alpaca-51.8k_identity-model-232_squadv2-15k/runs/bpju3d09?nw=nwuseradhafajp)
+📊 **W&B Log (Phase 1F):** [wandb.ai/VoidNova/.../runs/bpju3d09](https://wandb.ai/VoidNova/phi-2-2.7B_qlora_alpaca-51.8k_identity-model-232_squadv2-15k/runs/bpju3d09?nw=nwuseradhafajp)
+📊 **W&B Log (Final):** [wandb.ai/VoidNova/.../runs/rx5fih5v](https://wandb.ai/VoidNova/phi-2_qlora_ZeroChat/runs/rx5fih5v?nw=nwuseradhafajp)
 
 ---
 
@@ -192,7 +209,7 @@ print(f"Model response:\n{final_answer}")
 ---
 
 ### 🪶 Example Prompts
-"Who are you?"
+"Who are you?" 
 "How to be success?"
 
 ---
@@ -201,11 +218,14 @@ print(f"Model response:\n{final_answer}")
 "CONTEXT:---Zinc is an essential mineral perceived by the public today as being of ''exceptional biologic and public health importance'', especially regarding prenatal and postnatal development. Zinc deficiency affects about two billion people in the developing world and is associated with many diseases. In children it causes growth retardation, delayed sexual maturation, infection susceptibility, and diarrhea. Enzymes with a zinc atom in the reactive center are widespread in biochemistry, such as alcohol dehydrogenase in humans. Consumption of excess zinc can cause ataxia, lethargy and copper deficiency.---QUESTION:How many people are affected by zinc deficiency?"
 
 
-## Acknowledgements & Licenses
+## Acknowledgements & License
 
 This project builds upon several outstanding open-source contributions:
 
-* **Base Model:** This work is a fine-tuned adapter of `microsoft/phi-2`. The `phi-2` model is licensed under the **MIT License**.
-    * `Copyright (c) 2023 Microsoft`
-* **Libraries:** This project is powered by `transformers`, `peft`, and `bitsandbytes` by Hugging Face 🤗, as well as `torch` by PyTorch. These libraries are generally available under the Apache 2.0 or similar permissive licenses.
-* **This Adapter & Code:** The original code for this repository (including the adapter weights) is licensed under the **Apache 2.0 License**.
+* **Base Model:** This adapter is fine-tuned from [`microsoft/phi-2`](https://huggingface.co/microsoft/phi-2), licensed under the **MIT License**.  
+  *Copyright (c) 2023 Microsoft.*
+
+* **Libraries:** Powered by `transformers`, `peft`, and `bitsandbytes` from Hugging Face 🤗, as well as `torch` from PyTorch — all permissively licensed (Apache 2.0 or MIT).
+
+* **This Adapter & Code:** Released under the **MIT License**.  
+  You are free to use, modify, and distribute it with proper attribution.

adapter_config.json

@@ -15,7 +15,7 @@
   "loftq_config": {},
   "lora_alpha": 32,
   "lora_bias": false,
-  "lora_dropout": 0.07,
+  "lora_dropout": 0.1,
   "megatron_config": null,
   "megatron_core": "megatron.core",
   "modules_to_save": null,
@@ -25,12 +25,12 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "k_proj",
-    "v_proj",
     "q_proj",
     "fc1",
+    "k_proj",
+    "fc2",
     "dense",
-    "fc2"
+    "v_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:96a8c75e8f69f978ebd7fecb8e78f5e70b0edccc8adb77a3f327e6fe427d346a
+oid sha256:800299e8664be4bf9d9c68eb6634579b655b31d4368bc08db5939586cf61ea8b
 size 1124708052

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4910e3cbee3e89235f7dd52c7d44cc26f899f642b3d29a6507a0e1a93d211e40
+oid sha256:71b37fb951bfd8e038bc1f7eaef715f3a4811da107951f8765b217a70438ae00
 size 6353