qwentts-cpp-python

Python bindings and wheel packaging for Pascal's qwentts.cpp C ABI.

This package is intentionally small:

• it loads libqwen with ctypes
• it exposes buffered and streaming synthesis
• it can bundle prebuilt libqwen/libggml binaries in platform wheels
• it does not bundle GGUF model weights

CUDA development build with an existing qwentts.cpp checkout:

python scripts/build_native.py \
  --source /path/to/qwentts.cpp \
  --backend cuda \
  --clean
QWENTTS_CPP_WHEEL_BUILD_TAG=1cu128 python -m build --wheel

CPU development build:

python scripts/build_native.py \
  --source /path/to/qwentts.cpp \
  --backend cpu \
  --clean
QWENTTS_CPP_WHEEL_BUILD_TAG=1cpu python -m build --wheel

--backend metal is the default on macOS; other platforms default to CUDA. CPU builds are still useful for development and smoke tests.

Installation

The current release is 0.5.0, with qwentts.cpp ABI v5. PyPI provides Linux CUDA 12.8 wheels and macOS 14+ Apple Silicon Metal wheels. Pip selects the appropriate wheel automatically:

python -m pip install --upgrade "qwentts-cpp-python>=0.5.0"

Additional backend-specific wheels are published to Hugging Face Hub as local-version variants. Use them when the PyPI CUDA 12.8 wheel does not match the runtime or GPU target, for example DGX Spark / GB10 with CUDA 13:

Hugging Face variants are published separately. Release 0.5.0 uses ABI v5 for all backends, matching the public PyPI release.

pip install "qwentts-cpp-python==0.5.0+cpu" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cpu

pip install "qwentts-cpp-python==0.5.0+cu124" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu124

pip install "qwentts-cpp-python==0.5.0+cu128" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu128

pip install "qwentts-cpp-python==0.5.0+cu130" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/cu130

These commands use pip's --find-links mode against the Hugging Face directory page for the selected flavor. Dependencies still resolve from PyPI normally. The wheels do not bundle CUDA runtime or cuBLAS libraries; use a base image or system installation that provides the matching CUDA runtime.

The Hugging Face wheel pages may contain multiple Linux compatibility tags for the same backend flavor. For example, the cu128 page can host both manylinux_2_35 wheels for Ubuntu 22.04+ and manylinux_2_39 wheels for Ubuntu 24.04+. Pip selects the newest compatible wheel for the current machine.

Pull requests do not build the Linux wheel matrix. The PyPI and Hugging Face publishing workflows each rebuild fresh wheels from the pinned qwentts.cpp revision; validation artifacts are not reused for publishing.

Apple Silicon (Metal)

For macOS 14 or newer, arm64 Python 3.10+, install directly from PyPI:

python -m pip install --upgrade --only-binary=qwentts-cpp-python "qwentts-cpp-python>=0.5.0"
python -c "from qwentts_cpp import QwenLibrary; print(QwenLibrary().version())"

No local CMake build, Homebrew libraries, or explicit library_path is needed. The wheel bundles libqwen and the ggml CPU, Metal, BLAS, and core libraries; Metal shader source is embedded in the Metal library. Apple frameworks come from macOS. The build applies a shader-only workaround for the pinned ggml's invalid scalar-to-BF16-vector fill cast, restoring the source checkout afterward; the C ABI is unchanged. Use a native arm64 Python, rather than an Intel Python under Rosetta. Downstream packages can declare qwentts-cpp-python>=0.5.0 as a regular dependency, without an extra index or a direct wheel URL. The macOS platform tag distinguishes Metal wheels from Linux CUDA wheels on PyPI. Intel Macs and macOS versions older than 14 are not supported by this Metal wheel.

The separately published 0.5.0+metal variant is also available on Hugging Face:

python -m pip install "qwentts-cpp-python==0.5.0+metal" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/metal

The +metal version identifies the backend on Hugging Face; PyPI wheels use the plain public version without this suffix.

Before publication, download the hf-wheel-metal-macosx-arm64 artifact from the PR's Apple Silicon Metal wheel check, unzip it, and install its .whl with python -m pip install /path/to/qwentts_cpp_python-0.5.0+metal-*.whl. This macOS check also runs for PRs; the Linux matrix remains dispatch-only. Both publishers rebuild and check their own Metal wheels.

Run a streaming smoke test using local GGUF weights (weights are not bundled):

python scripts/smoke_stream.py \
  --talker /path/to/qwen-talker-1.7b-base-Q8_0.gguf \
  --codec /path/to/qwen-tokenizer-12hz-Q8_0.gguf \
  --ref-spk /path/to/reference.spk \
  --require-metal --output metal-smoke.wav

For a CustomVoice talker, replace --ref-spk with --speaker Vivian; for a VoiceDesign talker, use --instruct "A calm, clear voice.". --require-metal forces the Metal device so the test fails if only CPU execution is available. The script checks multiple nonempty, finite audio chunks and writes a mono 24 kHz PCM WAV. Listen to it with afplay metal-smoke.wav. Hosted macOS CI checks packaging, ABI layouts, and installed loading; synthesis requires a Mac with a working Metal device and local weights.

To build the same wheel locally with the pinned source checkout:

python -m pip install build delocate twine
export MACOSX_DEPLOYMENT_TARGET=14.0
python scripts/set_local_version.py metal  # changes local version metadata
python scripts/build_native.py --backend metal --clean
python -m build --wheel --config-setting=--build-option=--plat-name=macosx_14_0_arm64
delocate-wheel --require-archs arm64 -w wheelhouse -v dist/*.whl
python -m twine check --strict wheelhouse/*.whl

AMD ROCm / HIP (Linux)

The Hugging Face publisher builds a separate +rocm724 wheel for Linux x86_64, Ubuntu 24.04, ROCm 7.2.4, and MI300X (gfx942). After a successful Hugging Face publishing run, install the ROCm flavor explicitly:

python -m pip install --upgrade --only-binary=qwentts-cpp-python \
  "qwentts-cpp-python==0.5.0+rocm724" \
  -f https://huggingface.co/datasets/andito/qwentts-cpp-python-wheels/tree/main/whl/rocm724
python -c "from qwentts_cpp import QwenLibrary; print(QwenLibrary().version())"

The wheel's linux_x86_64 tag does not let pip check the Linux distribution, ROCm version, or GPU target. Use the documented environment; other AMD architectures, Linux distributions, and ROCm versions have not been validated. The required system ROCm libraries and assets are listed below. The wheel does not claim manylinux portability and is not a PyPI backend variant.

The reusable HIP workflow builds a fresh wheel when called by the Hugging Face publisher, runs the same ABI and clean-installed checks as the PR job, and reports platform requirements with auditwheel show. The publisher includes the result in the rocm724 flavor index alongside CPU/CUDA/Metal. Dispatch Publish Hugging Face Wheels with backend=rocm724 to publish only a fresh ROCm build and preserve the other published flavor indexes and wheel files. The default backend=all rebuilds every supported backend.

The build helper supports --backend hip or QWENTTS_CPP_BACKEND=hip on Linux. It enables GGML_HIP and HIP graphs, builds shared GGML libraries, and disables CUDA, Metal, BLAS, and host-specific CPU optimizations. The pinned GGML requires ROCm/HIP 6.1 or newer at configure time; this is a build requirement, not a guarantee that every ROCm version or GPU works. Issue #35 records successful manual-build synthesis on ROCm 7.2.4, AMD Instinct MI300X VF (gfx942), in rocm/pytorch:rocm7.2.4_ubuntu24.04_py3.12_pytorch_release_2.9.1. Other versions and architectures have not been validated. The Linux HIP build and wheel validation PR check compiles the real pinned source for gfx942 on an Ubuntu GitHub worker using AMD's versioned ROCm 7.2.4 package repository and pinned compiler/BLAS development packages. It checks ABI layouts and wheel metadata, deletes the native source/build trees, checks installed ELF dependencies, and runs the wrapper tests against the installed wheel. The candidate 0.5.0+rocm724 wheel is saved as a CI artifact. These workers have no AMD GPU; synthesis and device registration still require validation on AMD hardware. A manual build with an explicit library path does not establish wheel portability.

Use a Linux ROCm development environment with a C/C++ compiler, CMake 3.21+, HIP clang++, hipBLAS, rocBLAS, and patchelf (for example, install patchelf with your system package manager). The matching AMD driver and GPU access must be available; containers need the host's ROCm device access as well. Build from this repository with the pinned native source:

python -m pip install cmake ninja build numpy huggingface-hub pytest
git clone https://github.com/ServeurpersoCom/qwentts.cpp third_party/qwentts.cpp
git -C third_party/qwentts.cpp checkout 6fae92914045cd83364d2845ceaa0f7969727319
git -C third_party/qwentts.cpp submodule update --init --recursive

python scripts/build_native.py \
  --source third_party/qwentts.cpp --build-dir build/hip --backend hip \
  --hip-compiler /opt/rocm/llvm/bin/clang++ \
  --hip-architectures gfx942 --rocm-path /opt/rocm --jobs 8 --clean
python -m pip install -e .
QWENTTS_CPP_SOURCE="$PWD/third_party/qwentts.cpp" python -m pytest -q tests
python -c "from qwentts_cpp import QwenLibrary; lib = QwenLibrary(); print(lib.path, lib.version())"

No QWENTTS_CPP_LIBRARY or build-directory LD_LIBRARY_PATH is needed: the helper copies libqwen.so, the GGML core/base/CPU libraries, and the versioned HIP backend into qwentts_cpp/lib. It resolves the HIP symlink to a regular libggml-hip.so.0 file and requires successful patchelf relocation to $ORIGIN, so these libraries resolve their packaged siblings. The Python loader preloads the HIP backend before GGML and libqwen. Building native libraries is an explicit step; pip install -e . alone does not compile them.

--hip-compiler, --hip-architectures, and --rocm-path default to CMAKE_HIP_COMPILER, CMAKE_HIP_ARCHITECTURES, and ROCM_PATH, respectively. When omitted, CMake/upstream discovers the toolkit and GPU targets. Use the target for your GPU; quote multiple targets such as "gfx942;gfx950". The ROCm prefix sets CMake's package search prefix and HIP compiler ROCm root. For additional options, use QWENTTS_CPP_CMAKE_ARGS or repeat --cmake-arg=...; these are applied last, so --cmake-arg=-DGGML_HIP_GRAPHS=OFF disables graphs. Use a clean build when changing backends or toolkits.

HIP packaging supports linked, versioned backends with GGML_BACKEND_DL=OFF (set by the helper). Dynamic module backends with GGML_BACKEND_DL=ON are rejected after configuration, before compilation. Copy-only --skip-build runs also reject an unversioned HIP module before replacing packaged libraries. Rebuild with --clean and GGML_BACKEND_DL=OFF if using an existing dynamic backend build directory.

System dependencies remain external. This helper bundles qwentts/GGML libraries only. It does not bundle libamdhip64, libhipblas, libhipblaslt, librocblas, their transitive ROCm dependencies, or rocBLAS/Tensile data files. Install a matching ROCm runtime (the tested container supplies these), along with the usual Linux C/C++ and OpenMP runtime dependencies. The dynamic loader must find the system libraries through its configured search paths; if needed, set LD_LIBRARY_PATH to the system ROCm lib/lib64 directories. Do not add the native source/build directory to that path for an installed package test. Inspect ldd on the packaged .so files and resolve any not found entries in the system environment.

Run a GPU smoke test with local weights:

python scripts/smoke_stream.py \
  --talker /path/to/qwen-talker-1.7b-voicedesign-BF16.gguf \
  --codec /path/to/qwen-tokenizer-12hz-BF16.gguf \
  --instruct "A warm, calm narrator." \
  --text "Hello. This speech was generated on an AMD GPU." \
  --require-rocm --full --output amd-smoke.wav

--require-rocm selects GGML_BACKEND=ROCm0, checks that the installed GGML registry contains that device with an AMD description, and fails before model loading if it is unavailable. It prints the library path, native revision, and device description, with native backend logs at info level. --full also runs buffered synthesis and writes amd-smoke-full.wav; streaming writes amd-smoke.wav. Both must be finite, nonsilent, mono 24 kHz audio; streaming must produce multiple chunks. Listen to both files to assess speech quality. Repeat with BF16/Q4_K_M and the Base/CustomVoice models described in the issue, using --ref-spk or --speaker for their conditioning. Device availability and GPU backend selection do not exclude CPU execution of individual unsupported operators.

For a local wheel, use a separate local version in a disposable checkout:

python scripts/set_local_version.py rocm724  # changes local version metadata
python -m build --wheel

This names the local wheel 0.5.0+rocm724; building it does not publish it. The same local version is used for the HIP PR-check artifact; that artifact targets the ROCm 7.2.4 / Ubuntu 24.04 / gfx942 environment, without a manylinux portability claim. The Hugging Face publisher calls the reusable HIP workflow to build its own fresh wheel; it does not reuse PR or validation artifacts. ROCm distributions use the separately selectable rocm724 index and local version. Do not publish ROCm and CUDA wheels with identical public package/version/platform tags to PyPI.

Before distributing a HIP wheel, copy only the wheel, smoke script, and local weights to a fresh matching ROCm environment without the repository or native build tree. Install the wheel, unset QWENTTS_CPP_LIBRARY and QWEN_LIBRARY_PATH, and keep only declared system paths in LD_LIBRARY_PATH. Verify QwenLibrary() loads the installed package and run the AMD smoke test there. Run the full wrapper tests against the installed package (without a source PYTHONPATH), inspect wheel contents/dependency paths, and record GPU memory/backend logs and audio quality as described in issue #35. The Linux packaging regression uses small ELF fixtures to test relocation and clean installed loading; it does not validate real ROCm execution or establish that a wheel is self-contained.

Native ABI compatibility

The CI wheel build defaults to qwentts.cpp 6fae92914045cd83364d2845ceaa0f7969727319 (September 28, 2026), which requires ABI v5 and includes the upstream batched compute worker, codec memory controls, language/model queries, and latest ggml fork update.

ABI v5 changes the initialization and synthesis struct layouts; libraries built from the previous ABI v2 pin must be rebuilt. Python's do_sample=False and subtalker_do_sample=False now select greedy decoding by setting the corresponding native temperature to zero. Buffered codec chunk size is set once with QwenTTS(..., codec_chunk_sec=24.0) or QwenTTS.from_pretrained(..., codec_chunk_sec=24.0), rather than on synthesize(). Zero selects the native default. The codec derives its own left context; codec_left_context_sec is no longer accepted. Streaming's codec_chunk_sec still controls Python output packet size independently. The wrapper continues to serialize requests on each context (native max_batch=1). tts.language_names() lists the model's languages (synthesis also accepts lang="auto"), and tts.model_type() reports base, custom_voice, or voice_design.

The loader verifies this native revision before calling functions that write ctypes parameter buffers. Upstream does not expose an ABI-version or struct-size query, so other revisions (including unknown builds) are rejected with an actionable error even if they may be compatible. When updating the pin, update QWENTTS_NATIVE_REVISION in the binding and run tests/test_native_abi.py with QWENTTS_CPP_SOURCE pointing to the new checkout to verify every struct size and field offset. An incompatible library fails before model loading.

QWENTTS_CPP_WHEEL_BUILD_TAG is useful for local wheelhouses. For public indexes, publish one backend flavor per package/version/platform compatibility tag; otherwise pip has no way to choose between CPU and CUDA binaries.

Local smoke test with a built library:

QWENTTS_CPP_LIBRARY=/path/to/libqwen.so python - <<'PY'
from qwentts_cpp import QwenLibrary
lib = QwenLibrary()
print(lib.version())
PY

Model files are resolved with huggingface-hub by QwenTTS.from_pretrained(...) or passed directly to QwenTTS(...) as GGUF paths.

Native logging in Python applications

Pass log_level="warning" when creating a context to hide routine qwentts.cpp and GGML/Metal messages from model loading and synthesis while keeping warnings and errors on stderr. "quiet" is an alias for "warning"; "info" and "debug" expose diagnostics, and "error" shows errors only.

from qwentts_cpp import QwenTTS

with QwenTTS.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice",
    quant="Q4_K_M",
    log_level="warning",
) as tts:
    audio, sample_rate = tts.synthesize(
        text="The sky is blue today.", speaker="Vivian"
    )

The same log_level argument works with QwenTTS(talker_path, codec_path) and QwenLibrary(...). Call set_log_level("debug") on either object to expose all diagnostics later. Configure the level before creating a QwenTTS to cover model initialization. The level and both native callbacks are process-wide: the most recent setting applies to all contexts, persists after a context closes, and should be changed only when no other thread is running native inference. This setting does not redirect the process stderr file descriptor. Existing set_log_callback() receives filtered qwentts.cpp messages; GGML messages use stderr. The package's wheel builds retain an application-installed GGML callback during first backend initialization; custom native builds must use scripts/build_native.py for the same behavior.

Streaming packet sizes

QwenTTS.stream() defaults to a 4-frame first packet (320 ms of mono 24 kHz audio). Choose first_chunk_frames=1, 2, 4, or 8 for 80, 160, 320, or 640 ms of initial audio. A larger packet gives playback more audio to start with, at the cost of waiting longer before the first yield.

for audio, sample_rate in tts.stream(
    text="The sky is blue today.",
    ref_spk_emb=spk,             # Base model; use speaker= for CustomVoice
    first_chunk_frames=4,
    codec_chunk_sec=0.64,       # Default later packets: 8 frames, independently sized
):
    play_audio(audio, sample_rate)  # Your playback/transport function

Later packets default to codec_chunk_sec=0.64 (8 frames), matching the native steady-state width without adding a larger batching delay after the first packet. Explicit values round to the nearest 80 ms frame, with a one-frame minimum (for example, 1.0 second rounds to 13 frames / 1.04 seconds). The value must be finite and positive. Successful end-of-speech or the token limit flushes any remaining audio as a short packet, including utterances shorter than the requested first packet. Cancellation or errors discard the unfinished packet; closing the iterator requests native cancellation.

Packet assembly happens in Python, using the verified ABI v5 library. The native decoder still emits its fixed 1→2→4→8-frame ramp, then 8-frame chunks. Consequently, first packets of 1, 2, 4, and 8 frames become available after native output has reached 1, 3, 7, and 15 frames respectively (or earlier at end-of-speech). Packet boundaries preserve every PCM sample but do not change native decode scheduling. Later packets may become available together when a native callback spans several packet boundaries; this is not a timed playback scheduler. Buffered synthesize() uses the native codec chunk size configured when the context is created; the native codec derives its left context.

last_stream_profile keeps first_callback_* and callback_count for raw native callbacks. first_packet_ready_ms, first_packet_audio_s, and packet_count describe the assembled Python packets; first_yield_ms measures delivery to the caller. This distinction includes the buffering cost in latency measurements instead of treating the first native callback as audible output.

Benchmark 1, 4, and 8 frames on local hardware (one warm-up, then three measured runs per setting, with rotated order):

python scripts/benchmark_first_chunk.py \
  --talker /path/to/qwen-talker-1.7b-base-Q8_0.gguf \
  --codec /path/to/qwen-tokenizer-12hz-Q8_0.gguf \
  --ref-spk /path/to/reference.spk \
  --require-metal --output /tmp/first-chunk-benchmark.json

For other model types, use --speaker or --instruct instead of --ref-spk. Omit --require-metal on other backends. The report includes native first callback time, first Python packet time/duration, and every packet's size. The benchmark checks first/later packet sizes, finite non-silent audio, and the final tail. Playback smoothness still depends on synthesis speed and the player's buffering policy.

Example measurement on an Apple M3 Pro with Metal, macOS 26.6.2, Python 3.12.13, the Q8_0 1.7B Base talker and Q8_0 codec above, and a cached speaker embedding (three runs per setting after warm-up, medians):

First frames	Native first callback	First Python packet ready	First packet audio
1	94.4 ms	94.5 ms	80 ms
4 (default)	84.2 ms	369.8 ms	320 ms
8	85.7 ms	826.4 ms	640 ms

All runs used the default 8-frame later packets (codec_chunk_sec=0.64), preserved the 115-frame utterance, and flushed the final short tail. Four frames provide a middle ground on this machine; these measurements do not guarantee gap-free playback on other hardware. Raw native callbacks remain one frame initially; the larger first packet is assembled by the binding.

Cached voice references

qwentts.cpp can skip reference WAV encoding for Base voice cloning by passing precomputed latents:

• .spk: raw float32 speaker embedding from qwen-codec --talker
• .rvq: packed 11-bit reference codec stream from qwen-codec

The wrapper can create those files in-process from decoded mono float32 audio at 24 kHz:

from qwentts_cpp import QwenTTS

tts = QwenTTS.from_pretrained("Qwen/Qwen3-TTS-12Hz-1.7B-Base", quant="Q4_K_M")

# ref_audio_24k is a 1-D numpy float32 array, already resampled to 24 kHz.
voice_ref = tts.extract_voice_ref(ref_audio_24k)
voice_ref.save("reference.spk", "reference.rvq")

from qwentts_cpp import QwenTTS, load_speaker_embedding

tts = QwenTTS.from_pretrained("Qwen/Qwen3-TTS-12Hz-1.7B-Base", quant="Q4_K_M")

spk = load_speaker_embedding("reference.spk")
audio, sr = tts.synthesize(
    text="The sky is blue today.",
    lang="english",
    ref_spk_emb=spk,
    max_new_tokens=128,
)

For ICL clone mode, load the RVQ matrix with the model's codebook count and also pass the reference transcript:

from qwentts_cpp import load_rvq_codes

rvq = load_rvq_codes("reference.rvq", tts.num_codebooks())
audio, sr = tts.synthesize(
    text="The sky is blue today.",
    lang="english",
    ref_spk_emb=spk,
    ref_codes=rvq,
    ref_text="Transcript of the reference audio.",
)