codebyam commited on
Commit
a466a85
·
verified ·
1 Parent(s): c98b1e7

Update start.sh

Browse files

It will give boost upto 3x times. As docker container uses 2 cpu cores, we get best performance when threads are same as CPU count. By default it used 8 process.

Files changed (1) hide show
  1. start.sh +1 -1
start.sh CHANGED
@@ -2,7 +2,7 @@
2
 
3
  # Start llama-server in background
4
  cd /llama.cpp/build
5
- ./bin/llama-server --host 0.0.0.0 --port 8080 --model /models/model.q8_0.gguf --ctx-size 32768 &
6
 
7
  # Wait for server to initialize
8
  echo "Waiting for server to start..."
 
2
 
3
  # Start llama-server in background
4
  cd /llama.cpp/build
5
+ ./bin/llama-server --host 0.0.0.0 --port 8080 --model /models/model.q8_0.gguf --ctx-size 32768 --threads 2 &
6
 
7
  # Wait for server to initialize
8
  echo "Waiting for server to start..."