Repository navigation
[Engine] macOS / Metal: open contributions, rebased on v0.2.6 #4128
Description
Title: [Engine] macOS / Metal: open contributions, rebased on v0.2.6
This issue lists our open pull requests for Strata on Apple Silicon (Metal), all rebased on v0.2.6 and validated there. Each PR is self-contained and can be reviewed and merged on its own.
| PR | Area | What it changes | Why it matters on a Mac |
|---|---|---|---|
| #4126 | Engine | Sampling on the Metal decode path: temperature, top-k, top-p, min-p, penalties and a Philox seed (CUDA's sampler is the reference); a sampled verify window keeps a draft token only when the sample equals it. The server gives every sampled request a seed. | The Metal port decoded greedily only, so temperature had no effect. In agent use that produced word-for-word repetition loops. Greedy output is unchanged. |
| #3876 | Engine | Wrap a Gyro file without its PLE table: map only the head and tail ranges the GPU reads. | Gyro-M: wrapped 91.97 GB as 2 overlapping buffers becomes wrapped 37.57 GB as 2 separate ranges, leaving room for a second model or other work. |
| #4127 | Engine | Metal decode and prompt kernels for the pair-trellis expert types TQP5/TQP6/TQP7 (GGML types 147-149), the 256-expert router, a CPU reference and parity tests. | These are the types docs/GYRO.md names as the reason Gyro-L is not supported, and the ones the Gyro Coder files use. v0.2.6 cannot load such a file on a Mac; with this PR Gyro Coder-L runs (about 114 tok/s greedy decode here). |
| #3231 | Engine | Conversation cache on macOS: checkpoints validate and park on Metal, the memory guard reads available RAM, K/V is described for snapshots, the session fingerprint and restore accept what this engine saves. | v0.2.6 skips parking on Metal (invalid checkpoint running-state payload); returning to an 18k-token conversation took 11.3 s, with this PR 0.95 s (restored from the cache in 24 ms). Two concurrent agent conversations on one server (40-task coding set each): 692-1081 s per run without it, 238-380 s with it. |
| #3960 | Serve | Tool-call parsing: coerce parameters by declared type and recover the damaged forms Qwen3-coder-style models write (<=param> openers, </NAME> closers, a parameter name fused with its value, stray stubs). |
Fewer failed tool calls and retries in agent runs. |
| #4125 | Other | Test fix: serve/test_server.py::VisionArgs::test_min_tokens_is_passed_only_when_set fails on v0.2.6 ('Proc' object has no attribute 'pid'). |
serve/test_server.py runs without failures again, so a real regression is not hidden behind a known one. |
| #3176 | GUI | Web app layout follows the window: wide windows use the space, narrow screens stack. | Readability of the monitor and chat in wide windows and side panels. |
Closed: #3299 (Gyro on Metal). Its kernel work is upstream since v0.2.2/v0.2.3 (672fd99 and following); its remaining commits continue in #3231 and #3876.
How we run Gyro on a Mac
- Gyro-S / Gyro-M run on stock Strata (docs/GYRO.md, "On a Mac":
./setup.sh --family gyro --model Gyro-M). With [Engine] Metal: wrap a Gyro file without its PLE table (37.6 instead of 92 GB in GPU buffers) #3876 the GPU buffers shrink from 92 to 37.6 GB; with [Engine] Metal: sampling on the decode path (temperature, top-k, top-p, min-p, penalties, Philox seed) #4126 sampling works; with [Engine] macOS: conversation cache parks and restores (Metal checkpoints, K/V description, memory guard) #3231 agent conversations are parked and restored. - Gyro Coder-L (agentionai/Qwen3.8-Flash-Next-Gyro-Coder-GGUF) uses the pair-trellis types 147-149, which stock v0.2.6 does not have. It runs with [Engine] Metal: decode and prompt kernels for the pair-trellis expert types TQP5/TQP6/TQP7 #4127: build the Metal engine from that branch and pack the file the same way as Gyro-M (Hadamard-folded pack, expert profile,
--spec 2with the MTP draft,--ple-io direct). - Gyro-L uses the same types (docs/GYRO.md). A run of Gyro-L on [Engine] Metal: decode and prompt kernels for the pair-trellis expert types TQP5/TQP6/TQP7 #4127 is being verified; results will follow here.
Validation
Apple Silicon, 128 GB unified memory, Gyro-M TQ2_0 (Gyro Coder-L TQ3_0 for #4127): greedy output bit-exact against v0.2.6 (against the same stack without the PR for #4127, since v0.2.6 cannot load the file), decode speed within 2 %, and for sampling seed 1 != seed 2 / seed 1 == seed 1. Details and the exact revision are in each PR.
Activity
mnott commentedon Oct 25, 2026
Gyro-L on a Mac: verified with #4127, details in #4127 (pack, flags, a correct greedy answer, first decode run 35.5 tok/s).
mnott commentedon Oct 25, 2026
Gyro-L decode speed on Apple Silicon (128 GB unified memory), greedy, 256 tokens, 3 runs each,
--spec 2with the MTP draft, expert cache 64:wrapped 108.32 GB as 2 overlapping buffers--ple-io directwrapped 53.92 GB as 2 separate rangesThe 35.5 tok/s in my previous comment was a single cold 80-token run.