Skip to content

Real Vulkan benchmarks, --flash-attn on, GPU name detection - #4

Merged
kukirin-ui merged 4 commits into
mainfrom
real-bench
Oct 13, 2026
Merged

kukirin-ui merged 4 commits into
mainfrom
real-bench

Conversation

@kukirin-ui

Copy link
Copy Markdown
Owner
  • start.sh: --flash-attn on (newer llama.cpp needs a value; the bare flag made the GPU tier fail and fall back to CPU). Added a GPU tier without Flash-Attention/quantized KV before the CPU fallbacks. Default THREADS=2 when GPU offload is on.
  • setup.sh + start.sh: detect the real GPU name from llama-server --list-devices (first Vulkan0: <name> ( line, e.g. Samsung Xclipse 960) and save it in ~/.nullcloud-profile.env.
  • setup.sh: installs packaged llama-cpp + llama-cpp-backend-vulkan + vulkan-loader-android (removes vulkan-loader-generic); builds from source only with BUILD_VULKAN=1 or if no llama-server. GPU-first wording.
  • README: measured llama-bench table and screenshot (Samsung Xclipse 960, Qwen3 4B Q4_K_M, Vulkan, -ngl 99, 2/4/8 threads); THREADS=2 recommended. Troubleshooting for CPU backend and --flash-attn errors.
  • Web UI hardware card shows the measured numbers.
  • verify-all.sh passes.

@kukirin-ui
kukirin-ui merged commit 99eb47d into main Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant