Update README.md

README.md

@@ -20,7 +20,7 @@ This repository contains the **Falcon3-3B-Base**. It achieves strong results on
 Falcon3-3B-Base supports 4 languages (english, french, spanish, portuguese) and a context length up to 8K.
 Falcon3-3B-Base pruned (depth + width) from Falcon3-7B-Base, was effeciently trained on only 100 GT using a knowledge distillation objective.
 
-⚠️ **This is a raw, pretrained model, which should be further finetuned for most usecases.** 
+⚠️ **This is a raw, pretrained model, which should be further finetuned using SFT, RLHF, continued pretraining, etc. for most use cases.** 
 
 ## Model Details
 - Architecture
@@ -29,8 +29,9 @@ Falcon3-3B-Base pruned (depth + width) from Falcon3-7B-Base, was effeciently tra
   - Grouped query attention (GQA) for faster inference: 12 query heads and 4 KV heads
   - Wider head dimension: 256
   - High RoPE value to support long context understanding: 1000042
-  - 8k context length
-  - 131k vocab size
+  - Uses SwiGLu and RMSNorm
+  - 8K context length
+  - 131K vocab size
 - Pruned and Healed from Falcon3-7B-Base on only 100 Gigatokens of datasets comprising of web, code, STEM, high quality and mutlilingual data using 2048 H100 GPU chips
 - Supports EN, FR, ES, PT
 - Developed by [Technology Innovation Institute](https://www.tii.ae)