Add 2x Intel Arc Pro B60 SYCL F16 JIT results for Swift-Qwen3.8-27B

[Nulliik]

Benchmark submission for Rig 4: Maxsun ARC Pro B60 Dual 48G Turbo (SYCL F16 JIT + Native MTP)

This PR submits measured results for Qwen3.8-27B (qwen3-8-27b) on dual Intel Arc Pro B60 using a custom JIT build of llama.cpp with the Intel SYCL backend, native FP16 compute kernels, and native embedded Q8_0 Multi-Token Prediction (MTP) draft head:

• Rig: 4 (Maxsun ARC Pro B60 Dual 48G Turbo)
• Model: Swift-Qwen3.8-27B (qwen3-8-27b)
• Quantization: q4_k_m board (Swift-Qwen3.8-27B-Q4_K_L.gguf, 17.72 GiB)
• Runtime: llamacpp upstream master build b11100 (7ab4ee7b)
• Backend / Toolchain: Intel oneAPI Toolkit 2026.1.1 (icx/icpx) + MSVC v143 + Ninja
• Build configuration: Pure JIT (-DGGML_SYCL=ON -DGGML_SYCL_F16=ON -DGGML_SYCL_DNN=ON -DGGML_SYCL_TARGET=INTEL -DCMAKE_BUILD_TYPE=Release)
• Driver / Environment: Level-Zero driver, SYCL_DEVICE_FILTER=level_zero, SYCL_CACHE_PERSISTENT=0
• OS: Windows 11 Pro

Primary Leaderboard Submission:

• results/nulliik/qwen3-8-27b-llamacpp-sycl-q4_k_m-2xb60.json
  • Decode speed (decodeTps): 24.00 tok/s (Native embedded Q8_0 MTP draft speculative decoding, --spec-type draft-mtp --spec-draft-device SYCL0 --spec-draft-n-max 3)
  • Prefill speed (promptTps): 609.47 tok/s (pp512 on Q8_0 KV cache, scaling up to 925.39 tok/s)
  • Time to first token (ttftMs): 840.1 ms (cold-input evaluation at p512)
  • Context length (contextLength): 131072 (full 128k context window supported in VRAM)
  • Stock non-MTP baseline decode: 16.13 tok/s

---

Comparative Benchmark & Analysis: Dual Intel Arc Pro B60

All tests were performed on Windows 11 Pro with dual Intel Arc Pro B60 24GB (48GB total VRAM).

• Layer distribution: -sm layer -dev SYCL0/SYCL1 -ts 1.25/0.75 (balanced across cards: ~20.8 GB GPU0 / ~21.4 GB GPU1)
• Compute kernels: Flash Attention (-fa on), batch size 4096, ubatch size 4096 (-b 4096 -ub 4096)

---

1. Token Generation / Decode Scaling (decodeTps):

Configuration / Mode	Decode Speed
Native embedded Q8_0 MTP draft head (Primary Result)	24.00 tok/s
Live llama-server Chat with Q8_0 MTP	20.00 – 30.29 tok/s
Stock layer-split decode (llama-bench, no MTP)	16.13 tok/s
Stock Vulkan decode (Dual B60)	14.99 tok/s

---

2. Q8_0 KV Cache Performance (Full 128k Context Sweep):

Configuration: -ctk q8_0 -ctv q8_0 -b 4096 -ub 4096 -ts 1.25/0.75 (matches production server profile)

Context Length	Vulkan (Dual B60)	SYCL F16 (Dual B60, Q8 KV)	Speedup vs Vulkan
512 tokens	507.93 tok/s	609.47 tok/s	+20.0%
1 024 tokens (1k)	643.16 tok/s	798.52 tok/s	+24.2%
2 048 tokens (2k)	718.37 tok/s	881.14 tok/s	+22.7%
4 096 tokens (4k)	647.53 tok/s	925.39 tok/s	+42.9% 🚀
8 192 tokens (8k)	536.92 tok/s	911.25 tok/s	+69.7% 🚀
16 384 tokens (16k)	385.28 tok/s	886.65 tok/s	+130.1% (2.30x) 🚀
32 768 tokens (32k)	256.60 tok/s	825.18 tok/s	+221.6% (3.22x) 🚀
65 536 tokens (64k)	149.30 tok/s	724.73 tok/s	+385.4% (4.85x) 🚀
131 072 tokens (128k)	OOM / crash	570.24 tok/s	Rock Solid

---

3. FP16 KV Cache Performance (Full Precision KV Sweep):

Configuration: -ctk f16 -ctv f16 -b 4096 -ub 4096 -ts 1.25/0.75 (maximum context: 65,536 tokens within 48GB VRAM)

Context Length	Vulkan (Dual B60)	SYCL F16 (Dual B60, FP16 KV)	Speedup vs Vulkan
512 tokens	507.93 tok/s	644.59 tok/s	+26.9%
1 024 tokens (1k)	643.16 tok/s	798.06 tok/s	+24.1%
2 048 tokens (2k)	718.37 tok/s	883.74 tok/s	+23.0%
4 096 tokens (4k)	647.53 tok/s	925.17 tok/s	+42.9% 🚀
8 192 tokens (8k)	536.92 tok/s	914.33 tok/s	+70.3% 🚀
16 384 tokens (16k)	385.28 tok/s	883.82 tok/s	+129.4% (2.29x) 🚀
32 768 tokens (32k)	256.60 tok/s	826.99 tok/s	+222.3% (3.22x) 🚀
65 536 tokens (64k)	149.30 tok/s	725.60 tok/s	+386.0% (4.86x) 🚀
131 072 tokens (128k)	OOM / crash	OOM (>48GB total)	—

---

4. Side-by-Side: FP16 KV vs Q8_0 KV vs Vulkan:

Context Length	Vulkan (Dual B60)	SYCL FP16 KV	SYCL Q8_0 KV	KV Memory Footprint (FP16 vs Q8)
512 tokens	507.93 tok/s	644.59 tok/s	609.47 tok/s	128 MB vs 64 MB
1 024 tokens (1k)	643.16 tok/s	798.06 tok/s	798.52 tok/s	256 MB vs 128 MB
4 096 tokens (4k)	647.53 tok/s	925.17 tok/s	925.39 tok/s	1.0 GB vs 0.5 GB
8 192 tokens (8k)	536.92 tok/s	914.33 tok/s	911.25 tok/s	2.0 GB vs 1.0 GB
16 384 tokens (16k)	385.28 tok/s	883.82 tok/s	886.65 tok/s	4.0 GB vs 2.0 GB
32 768 tokens (32k)	256.60 tok/s	826.99 tok/s	825.18 tok/s	8.0 GB vs 4.0 GB
65 536 tokens (64k)	149.30 tok/s	725.60 tok/s	724.73 tok/s	16.0 GB vs 8.0 GB
131 072 tokens (128k)	OOM	OOM	570.24 tok/s	>32 GB (OOM) vs 16.0 GB (Fits!)

---

Key Engineering Takeaways:

1. MTP Speculative Decoding: Native embedded Q8_0 MTP (draft-mtp) runs fully in VRAM with zero external model overhead, boosting token generation from 16.13 tok/s to 24.00 tok/s (+48.9%), with high acceptance rates (85%–100%) in real conversations.
2. SYCL FP16 Compute Kernels: Enabling -DGGML_SYCL_F16=ON with oneDNN eliminates the FP32 execution bottleneck, delivering over 3x prefill speedup over older FP32 SYCL builds and outpacing Vulkan by up to 4.86x on long contexts.
3. Q8_0 vs FP16 KV Cache: Both FP16 and Q8_0 KV cache deliver virtually identical prefill throughput up to 64k tokens (~725 tok/s at 64k, ~925 tok/s peak). However, Q8_0 cuts KV memory usage exactly in half, enabling full 128k context window inference (570.24 tok/s) within the 48GB dual-card boundary.
4. JIT vs AOT on Battlemage Windows: Explicit AOT (-DGGML_SYCL_DEVICE_ARCH=bmg-g21) under Windows causes access violation crashes (0xC0000005), whereas runtime JIT with Level-Zero driver is completely stable.
5. Asymmetric Tensor Split: Setting -ts 1.25/0.75 distributes memory evenly between primary and secondary cards, preventing GPU1 from exhausting memory while preserving full interconnect throughput.

---

Acknowledgments

Special thanks to @SergiioB for the Intel Arc Pro B70 inference cookbook for key JIT compilation and runtime insights.

---

Validated locally with:
npm --prefix frontend run results:validate -- --author Nulliik ../results/nulliik/qwen3-8-27b-llamacpp-sycl-q4_k_m-2xb60.json

[vercel]

@Nulliik is attempting to deploy a commit to the Community Labs Team on Vercel.

A member of the Team first needs to authorize it.

[github-actions]

- results/nulliik/qwen3-8-27b-llamacpp-sycl-q4_k_m-2xb60.json: validated
Ready for merge: the author has a linked GitHub account and the database accepted the results in a rolled-back validation transaction. Merging submits them automatically; no web form is needed.

Site / sign up · Workflow details and retry

[jackwsmth]

Hey, thinking through some stuff for this one - notably whether we want to add Swift-Qwen3.8-27B as a separate model, or submit this as is and just keep it as a note.

Will get back to you soon :)

[Nulliik]

Hello, this model is similar to Qwen3.8-27B actually, shouldn't be different in perfomance. I can rerun same test with default model. Also I tested bigger quant Q4_K_L which is missing on website currently. Can lower mine and rerun it or either it has to be added to website.