Skip to content

session: reject n_threads above ggml's thread limit - #229

Open
AliOsm wants to merge 1 commit into
handy-computer:mainfrom
AliOsm:session-n-threads-limit
Open

AliOsm wants to merge 1 commit into
handy-computer:mainfrom
AliOsm:session-n-threads-limit

Conversation

@AliOsm

@AliOsm AliOsm commented Oct 24, 2026

Copy link
Copy Markdown

Summary

Session init only rejects a negative n_threads today. Any positive value is accepted, but values above ggml's thread limit (GGML_MAX_N_THREADS, 512) don't work: the CPU backend barely makes progress, or thread creation fails later in transcribe_run and is reported as out of memory.

This PR adds the upper bound to the existing check, so session init returns TRANSCRIBE_ERR_INVALID_ARG right away. The four session types (ASR, VAD, langid and diarize) each had the same n_threads < 0 test, so the rule now lives in one small helper, valid_n_threads() in transcribe-abi.h, and the public headers document the range. Values from 0 to 512 behave exactly as before.

Scope

  • src/transcribe-abi.h: new valid_n_threads() helper.
  • src/transcribe-asr.cpp, transcribe-vad.cpp, transcribe-langid.cpp, transcribe-diarize.cpp: use it in session init.
  • include/transcribe.h, transcribe/vad.h, transcribe/langid.h, transcribe/diarize.h: document the limit.
  • tests: tokenizer_smoke (ASR, also checks that 512 is still accepted), langid_dispatch_unit, diarize_dispatch_unit and silero_vad_smoke check that 513 is rejected.

There is no ABI change. The struct layouts are the same, and the bindings pass the value through, so they get the new error as well.

AI Assistance

Yes. As in #228, I used an AI assistant to help build the code and the wording of this description. I reviewed the changes and ran the checks below.

Validation

transcribe-cli with parakeet-tdt-0.6b-v3 Q8_0 on samples/jfk.wav, --backend cpu, on a 6 CPU machine:

--threads     main (4d7c1cc)                            this branch
513           did not finish in 120 s                   context init: invalid argument
4096          did not finish in 120 s                   context init: invalid argument
1000000       transcribe_run: out of memory             context init: invalid argument
2147483647    transcribe_run: out of memory             context init: invalid argument

On this branch every rejected value returns within about 1.5 s, which is mostly the model load.

scripts/ci/clang-format.sh --check passes, and ctest passes (61 tests; the diarize CLI smoke is skipped without a model).

This only enforces ggml's hard limit. A count between the number of CPUs and 512 is still accepted, and it is still very slow when it oversubscribes the machine (512 on this machine did not finish in 120 s either). If you think it is useful, I can follow up with a warning when n_threads is above the number of usable CPUs.

@AliOsm
AliOsm requested a review from cjpais as a code owner October 24, 2026 19:44
@AliOsm

AliOsm commented Oct 24, 2026

Copy link
Copy Markdown
Author

Hello @cjpais, I hope you are doing well. Please feel free to stop me at anytime if my PRs are becoming annoying to you. I'm not contributing to open source to get a job, but I face issues while working on my own projects, solving them, then trying to upstream the fixes in small focused PRs. Thanks!

@cjpais

cjpais commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Thanks for this, not annoying, appreciate it and I'll take a look and pull in when I have a chance

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants