Repository navigation

[Engine] Metal: sampling on the decode path (temperature, top-k, top-p, min-p, penalties, Philox seed) - #4126

Open

mnott wants to merge 2 commits into

Niko1221:mainfrom

mnott:pr/metal-sampling

Open

[Engine] Metal: sampling on the decode path (temperature, top-k, top-p, min-p, penalties, Philox seed)#4126
mnott wants to merge 2 commits into
Niko1221:mainfrom
mnott:pr/metal-sampling

Commits on Oct 25, 2026
- feat(metal): sampling on the decode path (CPU sampler over the head's logits: temperature, top-k, top-p, min-p, penalties, Philox seed; a sampled verify window keeps a draft when the sample equals it)
  mnott
  committed
  ce59fa7
- fix(serve): sampled requests carry a Philox seed (request's, else fresh random) so temperature takes effect
  mnott
  committed
  b5660df