diff --git a/README.md b/README.md
index a6afc1a..edaa263 100644
--- a/README.md
+++ b/README.md
@@ -64,11 +64,21 @@
## Performance
-Tested on a Samsung Exynos phone using GPU offload (Vulkan, `BUILD_VULKAN=1`). Tune GPU offload layers (`-ngl`) in the Web UI (**Private Cloud & Services -> Hardware Tweaks**) or the browser extension.
+Tested on a Samsung Exynos phone (Xclipse 960 GPU) using GPU offload (Vulkan, packaged `llama-cpp-backend-vulkan`). Tune GPU offload layers (`-ngl`) in the Web UI (**Private Cloud & Services -> Hardware Tweaks**) or the browser extension.
-| Device | Model | tok/s | s/article | Battery / 20 articles | Temp change |
-|---|---|---|---|---|---|
-| Exynos phone, GPU (Vulkan) | Llama-3.2-3B Q4_K_M | measurements coming | measurements coming | measurements coming | measurements coming |
+Measured with `llama-bench -p 512 -n 32 -r 1` (Vulkan, `-ngl 99`) inside the Meridian app Console on a real phone:
+
+| GPU | Model | Threads | Prompt (pp512) | Generation (tg32) |
+|---|---|---|---|---|
+| Samsung Xclipse 960 (Exynos) | Qwen3 4B Q4_K_M | **2** | **232.29 t/s** | **23.23 t/s** |
+| Samsung Xclipse 960 (Exynos) | Qwen3 4B Q4_K_M | 4 | 228.05 t/s | 21.92 t/s |
+| Samsung Xclipse 960 (Exynos) | Qwen3 4B Q4_K_M | 8 | 167.78 t/s | 13.79 t/s |
+
+
+
+*Real llama-bench run inside the Meridian app Console on Samsung Xclipse 960 (Vulkan)*
+
+With full GPU offload, more CPU threads do not help: **`THREADS=2` is recommended** (fastest prompt reading and generation). `termux/start.sh` uses 2 threads by default when GPU offload is on; override with `THREADS=4 bash ~/nullcloud/termux/start.sh`.
Measure your own device: run `bash termux/bench.sh 20` in Airplane Mode with the charger unplugged. It prints real end-to-end tok/s and seconds per article. PRs with your results are welcome.
@@ -147,7 +157,15 @@ Need turn-key deployment of **on-device, no cloud AI** for privileged legal disc
- **Model download stopped halfway:** run setup again. Partial `*.gguf.part` files are resumed or replaced.
- **Share -> Termux does nothing:** install **Termux:API** from F-Droid as well, run `pkg install -y termux-api`, and check that `~/bin/termux-url-opener` exists and is executable (`chmod +x ~/bin/termux-url-opener`).
- **Generation stops when the screen turns off:** Android Settings -> Apps -> Termux -> Battery -> **Unrestricted**, and run `termux-wake-lock`.
-- **Slow or the GPU is not used:** rebuild with Vulkan: `BUILD_VULKAN=1 bash ~/nullcloud/setup.sh`. If it still fails, lower GPU layers in the Web UI hardware tweaks.
+- **GPU is not used (`llama-bench` shows backend `CPU`, or start.sh says "CPU fallback"):** install the Vulkan backend and the Android Vulkan loader:
+ ```bash
+ pkg install -y llama-cpp llama-cpp-backend-vulkan
+ pkg uninstall -y vulkan-loader-generic
+ pkg install -y vulkan-loader-android
+ llama-server --list-devices
+ ```
+ You should see a line like `Vulkan0: Samsung Xclipse 960 (...)`. If the package is not available, build from source: `BUILD_VULKAN=1 bash ~/nullcloud/setup.sh`.
+- **`llama-server` fails with a `--flash-attn` error:** newer llama.cpp needs a value (`--flash-attn on`). Update Nullcloud with `cd ~/nullcloud && git pull`; `start.sh` also falls back to GPU without Flash-Attention.
- **Web UI says "Wrong token":** your token is in `~/.nullcloud-token`. Show it with `cat ~/.nullcloud-token` and paste it once in the Web UI.
- **Port 8080 already in use:** `bash ~/nullcloud/termux/stop.sh`, then start again.
diff --git a/screenshots/bench-meridian-xclipse960.jpg b/screenshots/bench-meridian-xclipse960.jpg
new file mode 100644
index 0000000..e043d0b
Binary files /dev/null and b/screenshots/bench-meridian-xclipse960.jpg differ
diff --git a/server/ui.html b/server/ui.html
index f9c0fef..4480065 100644
--- a/server/ui.html
+++ b/server/ui.html
@@ -112,7 +112,8 @@
Need help? Setup services
CPU & GPU Hardware Acceleration Tweaks
Tune both GPU layer offload (Vulkan on Android, tested on Exynos; Metal / CUDA / ROCm on desktop) and thread count + quantized KV cache (q8_0/q4_0):
-
Detecting local CPU + GPU engine...
+
Detecting local GPU engine...
+
Measured in Meridian on Samsung Xclipse 960: prompt 232 t/s, generation 23 t/s (Qwen3 4B Q4_K_M). Recommended: 2 CPU threads with full GPU offload.
diff --git a/termux/bench.sh b/termux/bench.sh
index fd5fa37..ac53a5f 100755
--- a/termux/bench.sh
+++ b/termux/bench.sh
@@ -23,7 +23,7 @@ for i in $(seq 1 "$RUNS"); do
done
read B1 T1 < <(bat)
echo
-echo "== Nullcloud CPU + GPU benchmark =="
+echo "== Nullcloud GPU benchmark =="
echo "model: ${MODEL_NAME:-local} | gpu: ${GPU_NAME:-Mobile GPU} (-ngl ${GPU_LAYERS:-99})"
echo "runs: $RUNS (each ~1,300-token page in, <=200 tokens out)"
awk "BEGIN{printf \"end-to-end speed: %.1f tok/s (includes page reading)\n\", $TOK/$SEC}"
diff --git a/termux/setup.sh b/termux/setup.sh
index 97bc609..915d306 100755
--- a/termux/setup.sh
+++ b/termux/setup.sh
@@ -12,8 +12,8 @@ URL_1B="https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/resolve/main
MEM_KB="$(awk '/MemTotal:/ {print $2}' /proc/meminfo 2>/dev/null || echo 6000000)"
MEM_GB="$(awk "BEGIN{printf \"%.1f\", $MEM_KB / 1048576}")"
-# Auto-detect mobile GPU (Snapdragon Adreno, Arm Mali/Immortalis, Samsung Xclipse, or Vulkan ICD)
-GPU_NAME="Mobile GPU (Vulkan/OpenCL)"
+# Rough GPU guess from sysfs; replaced by llama.cpp's real Vulkan device name after install
+GPU_NAME="Mobile GPU (Vulkan)"
if [ -r /sys/class/kgsl/kgsl-3d0/gpu_model ]; then
GPU_NAME="Adreno $(tr -d '\0\r\n' < /sys/class/kgsl/kgsl-3d0/gpu_model)"
elif [ -d /sys/class/misc/mali0 ] || [ -e /dev/mali0 ]; then
@@ -48,15 +48,20 @@ fi
MODEL_URL="${MODEL_URL:-$DEFAULT_URL}"
-echo "== Nullcloud On-Device AI Setup (CPU + GPU Hybrid) =="
+echo "== Nullcloud On-Device AI Setup (GPU-first, Vulkan) =="
echo "Detected RAM: ${MEM_GB} GB | GPU: ${GPU_NAME}"
echo "Selecting ${MODEL_NAME} profile (run termux/bench.sh to measure speed)"
pkg update -y && pkg upgrade -y
-pkg install -y python curl wget jq termux-api unzip poppler vulkan-loader-android 2>/dev/null || \
- pkg install -y python curl wget jq termux-api unzip poppler
+pkg install -y python curl wget jq termux-api unzip poppler
-if [ "${BUILD_VULKAN:-0}" = "1" ]; then
+# Prefer the packaged llama.cpp with the Vulkan backend (no compiling needed).
+# vulkan-loader-android uses the phone's real GPU driver; vulkan-loader-generic does not.
+pkg uninstall -y vulkan-loader-generic 2>/dev/null || true
+pkg install -y vulkan-loader-android 2>/dev/null || true
+pkg install -y llama-cpp llama-cpp-backend-vulkan 2>/dev/null || pkg install -y llama-cpp 2>/dev/null || true
+
+if [ "${BUILD_VULKAN:-0}" = "1" ] || ! command -v llama-server >/dev/null; then
echo "Building llama.cpp from source with Vulkan GPU acceleration (-DGGML_VULKAN=ON)..."
pkg install -y git cmake clang vulkan-headers vulkan-loader-android shaderc
SRC_DIR="$HOME/.cache/llama.cpp-vulkan"
@@ -65,9 +70,19 @@ if [ "${BUILD_VULKAN:-0}" = "1" ]; then
cmake -S "$SRC_DIR" -B "$SRC_DIR/build" -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build "$SRC_DIR/build" --target llama-server -j"$(nproc 2>/dev/null || echo 4)"
cp "$SRC_DIR/build/bin/llama-server" "$PREFIX/bin/llama-server"
-elif ! command -v llama-server >/dev/null; then
- pkg install -y llama-cpp 2>/dev/null || pkg install -y llama.cpp 2>/dev/null || {
- echo "!! llama.cpp package not found in your Termux repo. Run with BUILD_VULKAN=1 bash termux/setup.sh to build from source."; exit 1; }
+fi
+
+# Ask llama.cpp for the real GPU name (e.g. "Vulkan0: Samsung Xclipse 960 (...)" -> "Samsung Xclipse 960")
+detect_llama_gpu() {
+ command -v llama-server >/dev/null 2>&1 || return 0
+ llama-server --list-devices 2>/dev/null | sed -n 's/^[[:space:]]*Vulkan0:[[:space:]]*\(.*[^[:space:]]\)[[:space:]]*(.*/\1/p' | head -n 1
+}
+DETECTED_GPU="$(detect_llama_gpu)"
+if [ -n "$DETECTED_GPU" ]; then
+ GPU_NAME="$DETECTED_GPU"
+ echo "llama.cpp Vulkan device: $GPU_NAME"
+else
+ echo "!! llama.cpp sees no Vulkan GPU. Check with: llama-server --list-devices (see README Troubleshooting)"
fi
pip install -r "$ROOT/server/requirements.txt"
@@ -101,7 +116,7 @@ echo "$ROOT" > "$HOME/.nullcloud-root"
echo
echo "== Setup complete (On-Device, No Cloud AI) =="
echo "Profile: $MODEL_NAME (CTX=$CTX_SIZE, MAX_CHARS=$MAX_CHARS)"
-echo "Compute: CPU (big.LITTLE + ARM NEON/dotprod) + GPU Offload ($GPU_NAME, -ngl $GPU_LAYERS)"
+echo "Compute: GPU offload ($GPU_NAME, -ngl $GPU_LAYERS, Vulkan), THREADS=2 recommended"
echo "Your token: $(cat "$HOME/.nullcloud-token")"
echo "Start: bash $ROOT/termux/start.sh"
echo "Entry points ready:"
diff --git a/termux/start.sh b/termux/start.sh
index bf64def..e6d79ff 100755
--- a/termux/start.sh
+++ b/termux/start.sh
@@ -1,7 +1,7 @@
#!/data/data/com.termux/files/usr/bin/bash
-# Start Nullcloud on Android (Termux): CPU + GPU hyper-optimized llama-server + local API server.
+# Start Nullcloud on Android (Termux): GPU-first (Vulkan) llama-server + local API server.
# Set TAILSCALE=1 to bind to your phone's Tailscale IPv4 for Private Cloud laptop access.
-# Set GPU_LAYERS=0 for CPU-only, or GPU_LAYERS=99 (default) for Vulkan/OpenCL mobile GPU offload.
+# GPU_LAYERS=99 (default) offloads all layers to the Vulkan GPU. THREADS=2 is recommended with full GPU offload.
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
TOKEN="$(cat "$HOME/.nullcloud-token")"
[ -f "$HOME/.nullcloud-profile.env" ] && . "$HOME/.nullcloud-profile.env"
@@ -11,11 +11,34 @@ MAX_CHARS="${MAX_CHARS:-6000}"
MAX_TOKENS="${MAX_TOKENS:-400}"
MODEL_NAME="${MODEL_NAME:-Llama-3.2-3B-Instruct-Q4_K_M}"
GPU_LAYERS="${GPU_LAYERS:-99}"
-GPU_NAME="${GPU_NAME:-Mobile GPU (Vulkan/OpenCL)}"
-# CPU tweak: detect big/performance cores, cap at 4 on 1B profile or 6 on 3B profile for optimal tok/s & thermals
+# Ask llama.cpp for the real GPU name (e.g. "Vulkan0: Samsung Xclipse 960 (...)" -> "Samsung Xclipse 960")
+detect_llama_gpu() {
+ command -v llama-server >/dev/null 2>&1 || return 0
+ llama-server --list-devices 2>/dev/null | sed -n 's/^[[:space:]]*Vulkan0:[[:space:]]*\(.*[^[:space:]]\)[[:space:]]*(.*/\1/p' | head -n 1
+}
+case "${GPU_NAME:-}" in
+ ""|"Mobile GPU"*|"Vulkan ("*|"Arm Mali / Immortalis GPU")
+ DETECTED="$(detect_llama_gpu)"
+ if [ -n "$DETECTED" ]; then
+ GPU_NAME="$DETECTED"
+ PROFILE="$HOME/.nullcloud-profile.env"
+ if [ -f "$PROFILE" ] && grep -q '^GPU_NAME=' "$PROFILE"; then
+ sed -i "s|^GPU_NAME=.*|GPU_NAME=\"$GPU_NAME\"|" "$PROFILE"
+ else
+ echo "GPU_NAME=\"$GPU_NAME\"" >> "$PROFILE"
+ fi
+ fi
+ ;;
+esac
+GPU_NAME="${GPU_NAME:-Mobile GPU (Vulkan)}"
+
+# Threads: with full GPU offload, 2 threads gave the best prompt speed in real tests
+# (Samsung Xclipse 960, Qwen3 4B Q4_K_M: 2 threads pp512 232.3 / tg32 23.2 t/s, 4 threads 228.1 / 21.9, 8 threads 167.8 / 13.8).
CORES="$(nproc 2>/dev/null || echo 4)"
-if [ "${MODEL_TIER:-3b}" = "1b" ]; then
+if [ "$GPU_LAYERS" -gt 0 ] 2>/dev/null; then
+ DEFAULT_THREADS=2
+elif [ "${MODEL_TIER:-3b}" = "1b" ]; then
DEFAULT_THREADS=$(( CORES > 4 ? 4 : CORES ))
else
DEFAULT_THREADS=$(( CORES > 6 ? 6 : CORES ))
@@ -34,22 +57,31 @@ termux-wake-lock 2>/dev/null
pkill -f "llama-server.*--port 8081" 2>/dev/null
pkill -f "server/server.py" 2>/dev/null
-# Tier 1: GPU offload (-ngl) + quantized KV cache (q8_0/q4_0) + Flash-Attention + CPU big-core threads
-ACCEL_ACTIVE="CPU ($THREADS threads) + GPU ($GPU_NAME, -ngl $GPU_LAYERS, KV q8_0/q4_0, Flash-Attn)"
+# Tier 1: GPU offload (-ngl) + quantized KV cache (q8_0/q4_0) + Flash-Attention
+ACCEL_ACTIVE="GPU ($GPU_NAME, -ngl $GPU_LAYERS, KV q8_0/q4_0, Flash-Attn, $THREADS threads)"
if [ "$GPU_LAYERS" -gt 0 ] 2>/dev/null; then
nohup llama-server -m "$HOME/nullcloud-models/model.gguf" \
-c "$CTX_SIZE" -t "$THREADS" -ngl "$GPU_LAYERS" -b 512 -ub 128 \
- --cache-type-k q8_0 --cache-type-v q4_0 --flash-attn \
+ --cache-type-k q8_0 --cache-type-v q4_0 --flash-attn on \
+ --host 127.0.0.1 --port 8081 > "$HOME/nullcloud-llm.log" 2>&1 &
+ sleep 1
+fi
+
+# Tier 1b: GPU offload without Flash-Attention / quantized KV (older or stricter llama.cpp builds)
+if [ "$GPU_LAYERS" -gt 0 ] 2>/dev/null && ! pgrep -f "llama-server.*--port 8081" >/dev/null 2>&1; then
+ ACCEL_ACTIVE="GPU ($GPU_NAME, -ngl $GPU_LAYERS, $THREADS threads)"
+ nohup llama-server -m "$HOME/nullcloud-models/model.gguf" \
+ -c "$CTX_SIZE" -t "$THREADS" -ngl "$GPU_LAYERS" \
--host 127.0.0.1 --port 8081 > "$HOME/nullcloud-llm.log" 2>&1 &
sleep 1
fi
# Tier 2 fallback: CPU-optimized quantized KV cache + Flash-Attention (if GPU offload not compiled or GPU_LAYERS=0)
if ! pgrep -f "llama-server.*--port 8081" >/dev/null 2>&1; then
- ACCEL_ACTIVE="CPU ($THREADS big-cores, NEON/dotprod, KV q8_0/q4_0, Flash-Attn)"
+ ACCEL_ACTIVE="CPU fallback ($THREADS threads, KV q8_0/q4_0, Flash-Attn) - GPU did not start, see Troubleshooting"
nohup llama-server -m "$HOME/nullcloud-models/model.gguf" \
-c "$CTX_SIZE" -t "$THREADS" -b 512 -ub 128 \
- --cache-type-k q8_0 --cache-type-v q4_0 --flash-attn \
+ --cache-type-k q8_0 --cache-type-v q4_0 --flash-attn on \
--host 127.0.0.1 --port 8081 > "$HOME/nullcloud-llm.log" 2>&1 &
sleep 1
fi