Repository navigation
Conversation
lgirdwood
changed the title
Member
Author
|
@naveen-manohar @harajend @bardliao @ujfalusi this is just for testing on the CI, it has a lot of patches not for upstream, just for debug. Will allow us to roll out WOV test alongside other WOV PR in flow. |
Contributor
|
Thanks @lgirdwood |
…overy Support building microWakeWord with external or dynamic toolchain configurations where libc.a / libm.a archive layouts differ between Xtensa toolchain versions. Specifically: - Check for XTENSA_SYSTEM libc archive path when not found in standard Zephyr paths. - Check libc archive member names to extract either Newlib (lib_a-*.o) or standard libc/libm members into the shim archive. - Run mww_shim_flatten.py to flatten .text.* and .literal.* sections into .text and .literal to prevent CALL8 out-of-range relocation overflows. - Enable -mlongcalls and -mtext-section-literals on Xtensa architecture builds. - Conditionally include ibuffer_allocator.cc when present in the TFLM tree. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
tlv_value_set_uaol_caps() set link_count to the devicetree device count unconditionally and indexed link_caps[] by device index, so a link whose uaol_get_capabilities() call failed was still advertised to the host as a present link with all-zero capabilities. That matters now that the driver rejects the query when the host has not set UAOLCTL.OFLEN: without this, a firmware that cannot read any link still reports link_count = 1 with zeroed stream counts and FIFO sizes. Count only the links that answered, pack them from index 0, and size the TLV to match. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
… on IPC4 kpb.c drives both a sel_sink (buf_id 0, the real-time feed to detectors) and a host_sink (buf_id 1, the drain that bursts pre-roll to the host). Declare two sink pins in kpb.toml so the drain path can be bound from topology. Additionally, under IPC4 pipeline state transitions and resets are synchronous; returning -EBUSY on reset breaks the host teardown state machine and causes widget free failures. Guard the deferred -EBUSY drain check under #if !CONFIG_IPC_MAJOR_4. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add NOTIFIER_ID_D0IX_STATE to notify registered DSP components when the host changes the DSP D0 substate via SOF_IPC4_MOD_SET_D0IX. Components such as microWakeWord that keep running while the host sleeps in S0iX / D0i3 can use this event to adjust telemetry reporting or power characteristics. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
When source or sink component is not a processing_module (such as a host copier or non-module comp_dev), comp_mod() returns NULL. Safely guard src_module_data and dst_module_data before accessing mpd.in_buff_size and mpd.out_buff_size to prevent NULL pointer dereferences when configuring DP ring buffers. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Hostless internal pipelines (such as background keyword detection or internal audio processing chains that feed an arbiter or buffer without a host copier) do not have a host dev. Allow ipc4_pipeline_prepare() and ipc4_pipeline_trigger() to operate on pipelines where host is NULL by falling back to source_comp or sink_comp. Furthermore, allow stopping pipelines that are currently in COMP_STATE_PREPARE. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…drain streaming Improve the WoV Arbiter component: - Host Notification: send IPC4 notification to host upon keyword detection and periodically during pre-roll buffer drain. - Active Slot Reset: reset active_slot to WOV_ARB_NO_ACTIVE and notify host/kcontrols on stream stop, reset, and free. Also emit WOV_ARB_CMD_RESUME to unpause detectors. - Drain Streaming: ensure the arbiter only produces data when an active slot is triggered, leaving the host PCM idle/blocked during low-power listening so the platform can enter D0i3. - Update wov_arbiter README documentation. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Adapt the ECNS module to support the dual-rate DMIC architecture: - Input pin 0: 16 kHz stream from dmic16k (up to 320 frames per 20ms period). Extracts left channel to mono output pin 0 feeding KPB. - Input pin 1: 48 kHz stream from dmic01 (up to 960 frames per 20ms period). Copies stereo 48 kHz output to pin 1 feeding Host Copier PCM 11. - Operates in 20ms DP mode with independent buffer tracking per pin. - Update ECNS header and README documentation. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…ter to fix heap leak
TFLite Micro defines TF_LITE_REMOVE_VIRTUAL_DELETE on MicroMutableOpResolver,
which expands to a no-op 'void operator delete(void* p) {}'. Consequently,
calling 'delete inst->op_resolver' in MWW_Free() never returned the memory
to the heap, leaking ~600 bytes per slot on every stream open/close cycle.
Across multi-slot configurations (3 detector slots), repeated stream cycles
exhausted the DSP system heap, failing on run 10 with -ENOMEM in
MWW_InitOps() and returning -22 (IPC4_INVALID_REQUEST) to snd_pcm_start().
Eliminate heap allocations from MWW model initialization by allocating
both MwwOpResolver and MicroInterpreter in aligned static storage using
placement new and explicit destructor invocation.
Verified with 10 consecutive D0i3 sleep-and-wake test cycles on Panther Lake
hardware (Aphid), achieving 10/10 passes with zero errors.
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…er controls Add power and testing enhancements to microWakeWord: - D0i3 Power State: listen for NOTIFIER_ID_D0IX_STATE and suppress score telemetry notifications while the host is in D0i3 to prevent spurious wakeups. - Test Trigger Controls: implement SOF_IPC4_SWITCH_CONTROL_PARAM_ID to expose a volatile 'wovdebug' switch kcontrol per slot. Setting this control arms a fake wake timer (MWW_FAKE_WAKE_DELAY_MS) that triggers a keyword detect event and automatically resets the control upon firing. - Add CONFIG_MICROWAKEWORD_FAKE_WAKE and app/wov-d0i3-fake-wake.conf test overlay. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Disable UAOL on Intel ADSP targets in boards/intel_adsp/Kconfig.defconfig (default n). Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…odules Functional and feature topologies (such as SoundWire RT721/RT722 and DMIC pipelines) require open audio processing components like DRC, TDFB, Multiband DRC, Aria, and MUX. Ensure these modules are enabled in both PTL and WCL board configurations and module overlays so they are built into sof-ptl-openmodules.ri and sof-wcl-openmodules.ri. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…slot WoV pipelines Implement the dual-rate DMIC topology architecture with multi-slot WoV: - Provide dual PDM DAI endpoints: - dmic01 on PDM0: 48 kHz stereo 16-bit - dmic16k on PDM1: 16 kHz stereo 16-bit - Connect dmic16k into ECNS (pin 0) and dmic01 into ECNS (pin 1). - Connect ECNS output pin 0 (16k mono clean) to KPB and multi-slot WoV detectors (slots 0-2: OK Google, Alexa, Hey Jarvis), feeding WoV Arbiter and Host Copier PCM 12 (16 kHz mono WoV capture). - Connect ECNS output pin 1 (48k stereo clean) to Host Copier PCM 11 (48 kHz stereo capture). - Remove deprecated raw PCM 10 host copier. - Add volatile wovdebug switch controls and score enum controls per slot. - Set capture_compatible_d0i3 to true on WoV PCMs. - Update 4ch and multi manifests for PTL, WCL, and generic Intel platforms. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…topologies Integrate the multi-slot WoV pipelines and feature flags into SoundWire and HDA topologies: - Wire multi-slot WoV feature inclusion into cavs-sdw.conf, cavs-rt5682.conf, and sof-hda-generic.conf. - Ensure Speaker and other primary PCMs are preserved when WoV features are instantiated. - Add CMake topology build targets for Panther Lake (PTL) ACE 3.0, Wildcat Lake (WCL), and Tiger Lake (TGL) with multi-slot WoV SoundWire and HDA targets in production and development CMake files. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…PTL, TGL, and WCL Provide prebuilt topology binary files for quick testing and hardware bring-up: - build/sof-ptl-dmic-wov-multi-4ch.tplg - build/sof-tgl-dmic-wov-multi-4ch.tplg - build/sof-wcl-dmic-wov-multi-4ch.tplg Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add test utilities for verifying Wake-on-Voice capture pipelines: - wov_blocking_read.c: alsa-lib utility that opens the WoV capture PCM in blocking read mode with SND_PCM_NO_IRQ_WAITEVENT or mmap to ensure the driver does not generate periodic IRQs during listening. - wov_blocking_read_tinyalsa.c: TinyALSA version of the blocking reader. - test_wov_no_irq.py: automated test runner validating WoV capture and wake across all slots in S0 and D0i3 (idle suspend) states. - Makefile: build targets for wov_blocking_read and wov_blocking_read_tinyalsa. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…i3 verification Comprehensive documentation updates for multi-slot Wake-on-Voice and dual-rate DMIC: - tools/topology/topology2/README.md: - Document complete pipeline architecture and DMIC routing diagrams. - Document dual-rate DMIC PDM DAI structure (dmic01 @ 48k, dmic16k @ 16k). - Document ECNS 20ms DP processing and pin mappings. - Document KPB circular buffering, pre-roll drain, and WoV Arbiter slot routing. - Document multi-slot WoV detector pipeline (OK Google, Alexa, Hey Jarvis). - Document PCM endpoints: PCM 11 (48k stereo clean), PCM 12 (16k mono WoV). - Add build steps with alsatplg and verification runbooks. - doc/developer_guides/wov_ecns_integration_guide.md: - Update integration guide with dual-rate DMIC architecture and D0i3 verification. - README.md: - Update top-level README with microWakeWord and WoV documentation references. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…ECNS, KPB, and host copier Remove redundant mixin and mixout components between ECNS output pins and downstream consumers: - ECNS Output Pin 0 (16 kHz mono clean) connects directly to KPB (kpb.116.1 / kpb.106.1). - ECNS Output Pin 1 (48 kHz stereo clean) connects directly to the Host Copier capture endpoint (host-copier.11.capture / host-copier.10.capture). - Remove mixin.115.1, mixin.115.2, mixout.116.1, and mixout.117.1 from dmic-wov-multi.conf. - Remove mixin.105.1, mixin.105.2, mixout.106.1, and mixout.107.1 from standalone 4ch manifests (PTL, WCL, TGL). - Update prebuilt topology binaries and README topology diagrams and PCM tables. - Validated on Aphid hardware across 10x S0 and 10x D0i3 WoV capture tests (20/20 PASS). Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…DMEs Synchronize Mermaid architecture diagrams to reflect direct widget routing: - Add dedicated Dual-Rate DMIC & Multi-Slot WoV flowchart to tools/topology/topology2/README.md. - Update src/audio/ecns/README.md, src/audio/wov_arbiter/README.md, and doc/developer_guides/wov_ecns_integration_guide.md to eliminate intermediate mixin.115.1/mixin.115.2 and mixout.116.1/mixout.117.1 widgets. - Update src/audio/microwakeword/README.md to eliminate intermediate mixin.105.1/mixin.105.2 and mixout.106.1/mixout.107.1 widgets. - Show direct ECNS Pin 0 connection to KPB and ECNS Pin 1 connection to Host Copier (PCM 11 / PCM 10). Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
… 2-endpoint table Align standalone DMIC multi-slot Wake-on-Voice topology manifests (PTL, WCL, and TGL) to embed the full 2-endpoint NHLT table matching production SoundWire and platform topologies: - Define Object.Dai.DMIC with both Endpoint 0 (dmic01 @ 48 kHz, id 6) and Endpoint 1 (dmic16k @ 16 kHz, id 7) with 4-channel active PDM0/PDM1 mics. - Set Pipeline 100 DMIC_DAI_INDEX=1 and DMIC_NAME="dmic16k" so the 16 kHz capture copier matches Endpoint 1 in the topology-embedded NHLT table. - Generates the exact 6,211-byte, 2-endpoint NHLT binary table on PTL and WCL matching production nhlt-sof-ptl-rt721-4ch.bin bit-for-bit (md5: e16dcb62d2b3bacc939b24a8a0a9d0f0). - Generates matching 6,211-byte NHLT binary table on TGL with CAVS 2.5 driver version 1. - Update prebuilt topology binaries for PTL, WCL, and TGL in build/. - Update tools/wov_capture/run_wov_10x.py to dynamically auto-detect DMIC Multi-WOV capture PCM device index and active slot debug kcontrol names. - Update documentation in tools/topology/topology2/README.md. - Validated on Aphid hardware across 10x S0 and 10x D0i3 WoV capture tests (20/20 PASS). Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…cation
In IPC4, the Linux kernel assigns module instance IDs sequentially as
it traverses DAPM widgets. Because widgets are stored in a LIFO linked list
(list_add), the widgets in pipelines 101, 102, 103 are instantiated in
reverse order:
- mww.103.1 -> instance 0
- mww.102.1 -> instance 1
- mww.101.1 -> instance 2
Previously, mww.c derived slot_id as (IPC4_INST_ID(dev->ipc_config.id) % 3),
which assigned slot 0 to mww.103.1 and slot 2 to mww.101.1. When the host
triggered wovdebug_101, the arbiter received slot 2 and set wov_active_slot
to 3 ('Slot 3'), inverting slots 1 and 3.
Align with detect_test.c by computing:
cd->wov_slot_id = (uint8_t)(2 - (IPC4_INST_ID(dev->ipc_config.id) % 3));
This ensures:
- wovdebug_101 (Pipeline 101) -> slot 0 -> wov_active_slot = 1 ('Slot 1')
- wovdebug_102 (Pipeline 102) -> slot 1 -> wov_active_slot = 2 ('Slot 2')
- wovdebug_103 (Pipeline 103) -> slot 2 -> wov_active_slot = 3 ('Slot 3')
Also update wov_blocking_read.c and run_wov_10x.py:
- wov_blocking_read queries wov_active_slot upon receiving audio frames
and asserts that triggered active_slot matches the expected slot.
- run_wov_10x.py logs and validates trig_slot == slot for each iteration.
- Validated on Aphid hardware across 10x S0 and 10x D0i3 runs (20/20 PASS).
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Immediate logging runs every log call synchronously in the caller's context, including the LL scheduler tick and DAI start. On ADL this stalls the DMIC capture pipeline start: SET_PIPELINE_STATE(RUNNING) times out and the DMIC GP-DMA overruns afterwards. Drop CONFIG_LOG_MODE_IMMEDIATE and go back to the default deferred mode. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
CONFIG_ZEPHYR_DP_SCHEDULER can be enabled on cAVS, but only the ACE platform_init() registered the DP scheduler. Creating a DP module on cAVS then failed with "unable to bind scheduler to task (type 3)" and module init returned -ENODEV. Call scheduler_dp_init() after the LL scheduler is set up, as ACE does. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The DP lock is taken by the LL thread on every tick in scheduler_dp_ll_tick() and by each DP thread after every run. It was a k_sem, which has no priority inheritance: when a DP thread holding it was preempted by other DP threads, the cooperative high priority LL thread blocked until the holder got CPU time again. With several busy DP modules (3 MFCC + 3 MWW on cAVS 2.5) single LL ticks took up to ~40 ms, overrunning the DMIC DMAs of unrelated streams. With a k_mutex the holder inherits the LL priority: LL max drops from ~1.5M to <30k timer cycles, with no LL overruns or DMA xruns. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
In IPC4 a buffer gets its stream params from the consuming module at bind time. Legacy comp_driver consumers don't do that, so a DP module feeding one (e.g. MWW to the WoV arbiter) saw rate and frame size 0 on its sink and module_adapter_calculate_dp_period() divided by zero. Before calculating the DP period, apply the module's own stream params to any sink buffer which has no rate yet. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The dual-rate multi-slot WoV pipelines declare 2ch DMIC DAI copiers for both dmic01 (48 kHz) and dmic16k (16 kHz). This matches the dmic-wov-feature topology, which splits PDM0/PDM1 between the two DAIs, but not dmic-generic.conf, where both DAIs enable all NUM_DMICS mics. On 4-mic platforms (e.g. sof-adl-max98357a-rt5682-wov-multi) the NHLT only carries 4ch DMIC blobs, so the kernel fails to start the ECNS capture with: no matching blob for sample rate: 16000 sample width: 16 channels: 2 Add DMIC_WOV_DAI_CHANNELS (default 2) and set it to 4 in dmic-generic when NUM_DMICS is 4. With 4 channels the DAI copiers take 4ch input and keep the 2ch output, letting the copier remap the first two mics to stereo for the ECNS, so the rest of the WoV graph is unchanged. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The ECNS high-pass filters have fixed gain, +30 dB on the Pin 0 KPB detection output and +20 dB on the Pin 1 host capture output. Add a capture boost switch control to select between these boosted responses and the same filters with 0 dB gain. The 0 dB coefficient sets are added as separate blobs, converted from the highpass_100hz_0db_16khz, highpass_40hz_0db_48khz and highpass_40hz_0db_16khz topology blobs. A change only moves the IIR coefficient pointers, keeping the filter delay lines, and it is applied in copy() and prepare() so it also takes effect during streaming. This keeps the freedom to select other responses later. Switch controls are dispatched by control index. The capture boost is the first control (index 0) and the test signal the second (index 1), since the test signal may be removed later. The firmware starts boosted, so topologies without the control keep the current behavior; with the control, the kernel sends its value (default off) when the widget is set up. In the wov-multi topologies the controls are named per ECNS instance, "DMIC ECNS boost" and "Jack In ECNS boost", and the test signal switches "DMIC ecns_kpb_test_signal" and "Jack In ecns_kpb_test_signal", so a topology with both the DMIC and the jack input ECNS has unique control names. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Some topologies are byte-for-byte identical to an existing one and only need to be reachable under a different name. Add a SYMLINKS list, processed alongside TPLGS in production/CMakeLists.txt, that creates a real symlink instead of building a duplicate topology. cmake -E copy_if_different dereferences symlinks when deploying the build output, turning them back into regular file copies. Switch to "cp -a" so the deploy step preserves symlinks as symlinks. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> (cherry picked from commit c22e996)
WCL DMIC function topologies are byte-for-byte identical to the PTL ones, only reachable under a different name. Drop the duplicated TPLGS entries and list them in SYMLINKS instead, so they are built as symlinks to the PTL topologies rather than separately compiled duplicates. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> (cherry picked from commit dd2eeae)
The WCL DMIC and RT721 multi-slot WoV topologies are byte-for-byte identical to the PTL ones, only reachable under a different name. Drop the duplicated TPLGS entries and list them in SYMLINKS instead, so they are built as symlinks to the PTL topologies rather than separately compiled duplicates. The WCL RT722 targets stay as separate TPLGS entries since they enable INCLUDE_SDW_JACK_WOV, which the PTL ones do not. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The hda-generic naming convention declares in case NHLT blob is included: hda-generoc-[DSP plaform]- PTL and WCL are ACE3, they use the same blob for NHLT, the name of the topology must be: sof-hda-generic-ace3-4ch-wov-multi.tplg Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
platform/intel/wcl.conf was a verbatim copy of ptl.conf and the WCL
manifests only differed from the PTL ones by PLATFORM="wcl". WCL is an
ACE3 platform exactly like PTL; the SOF topology layer does not need a
separate platform key for it, as the existing sof-wcl-dmic-*-id* symlinks
already show.
Remove wcl.conf and the "wcl" IncludeByKey.PLATFORM entries, delete the
duplicated WCL DMIC multi-WoV manifests and generate sof-wcl-dmic-wov-multi
and sof-wcl-dmic-wov-multi-4ch as symlinks to the PTL topologies instead.
The WCL RT722 targets were the only ones enabling INCLUDE_SDW_JACK_WOV;
PTL and WCL are identical from the topology's point of view, so enable
it on the PTL RT722 targets and turn the WCL ones into symlinks as well.
Update the README to match, including the stale
sof-{ptl,wcl}-hda-generic-4ch-wov-multi target names which were renamed
to sof-hda-generic-ace3-4ch-wov-multi earlier.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
wov_capture_app used -c for the card and -C for the channel count, while tinycap (and arecord) use -D for the card/device and -c for the channel count. The subtle difference made it easy to pass the card number as the channel count and vice versa when switching between the tools. Switch to -D CARD and -c CHANS to match tinycap, and update the usage examples and the wov_arbiter README accordingly. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
…rase Detect ID When both DMIC and SoundWire headset WoV paths are instantiated in the same topology (e.g. rt722 wov-multi), both wov-arbiter widgets previously generated an ALSA mixer enum control with the identical name 'wov_active_slot', causing a naming collision in the mixer. Disambiguate the active slot controls per input path: - Change default wov-arbiter enum name to 'DMIC Phrase Detect ID'. - Override wov-arbiter enum name in sdw-jack-wov-multi.conf to 'Jack In Phrase Detect ID'. - Update integration guides and documentation accordingly. Co-developed-by: Naveen Manohar <naveen.m@intel.com> Signed-off-by: Naveen Manohar <naveen.m@intel.com> Co-developed-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Support unique phrase detect control names across the automated test harnesses and validation utilities: - In run_wov_10x.py, test_wov_no_irq.py, and wov_multi_slot_test.py: add find_active_slot_control() to auto-detect 'DMIC Phrase Detect ID', 'Jack In Phrase Detect ID', or fallback to 'wov_active_slot'. - In wov_multi_slot_test.py, add --control parameter to allow selecting a specific phrase detect control when multiple interfaces are present. - In wov_blocking_read.c, probe 'DMIC Phrase Detect ID', 'Jack In Phrase Detect ID', and 'wov_active_slot'. Co-developed-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Co-developed-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Plotting a log with several detector slots or a raw memory dump of the log buffer gave jumps in time, hop index and probability. All slots were merged into one series, repeated dump lines were parsed again, and spliced lines, where a truncated record continues with the tail of another one, paired a timestamp with the wrong payload. Skip exact duplicate lines, sort events by time, and require each MWW payload to follow its own log header. Plot one detector component at a time, chosen with --comp or by default the one with a detection, else the busiest. Also parse the "avg probability" detection message and default the threshold line to the firmware averaged trigger of 50%. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
When capturing pure digital silence (all-zero PCM code, such as from SSP loopback mode before playback begins), the linear Mel energies are zero. PCAN maps zero energy to int8 -128, which dequantizes to float 0.0 at the TFLM model input. Under continuous 0.0 input, non-zero convolutional layer biases propagate through the internal temporal ring buffers, causing models such as 'hi_intel' to output very high probabilities (peaking at ~99%) and fire false keyword detections immediately after warmup despite vad_flag being 0 and acoustic energy being zero. Gate keyword detection on VAD speech presence with a 100-hop (1.0 second) hold window. When speech activity is absent: - Flush the rolling probability history with 0.0 to prevent stationary silence bias from triggering on speech onset. - Suppress scoring kcontrol peak updates so digital silence does not report 100% confidence. - Increment the vad_gated_inferences telemetry counter whenever model probability reaches the detection threshold during silence. When VAD reports speech, retain detection enablement across the 1.0 second hold window to cover unvoiced keyword segments and normal speech pauses. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Every control change and phrase detected notification allocated a new IPC message that was never freed, since the IPC send path does not free messages. With a prepare, a detection and a stop in each WoV session this leaked 448 bytes of heap per session. In a capture stress test the heap was fragmented after about 440 sessions, and a DAI DMA buffer allocation for a concurrent playback failed. Allocate the two notification messages once per instance on first use, update and resend them for each notification, and free them in wov_arb_free(). Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The score and wovdebug control notifications allocated a new IPC message for each change and never freed it, since the IPC send path does not free messages. This leaked heap on every WoV session. Allocate the notification messages once per instance on first use with a common helper, update and resend them for each notification, and free them in mww_free(). Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add single-detector SoundWire DMIC Wake-on-Voice topology support for Panther Lake with Realtek RT713 headset jack and RT1320 smart amplifiers. The capture pipeline downmixes SoundWire microphone audio through micsel, resamples to 16 kHz with SRC, and feeds a 2000 ms KPB ring buffer. KPB pin 0 feeds the detector copier directly, which converts the 32-bit samples to 16-bit for the DP MFCC and microWakeWord modules. KPB pin 1 feeds the WoV arbiter, which forwards the drained 32-bit history to the host capture copier on PCM 12. Also add the sof-sdw-wov feature topology build target with this pipeline, for dynamic feature loading alongside SoundWire base topologies such as sof-ptl-rt713-l3-rt1320-l12.tplg. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add a Kconfig choice COMP_MWW_MODEL to select the built-in wake word model. The default choice is COMP_MWW_MODEL_PIPELINE_ID, which preserves the default behavior of selecting models dynamically via pipeline ID (compiling all default model headers into the firmware image). Add Kconfig options to force a single built-in keyword model at build time, omitting all other model headers from the build to reduce firmware footprint: - COMP_MWW_MODEL_HI_INTEL (mww_model_data_hi_intel.h) - COMP_MWW_MODEL_HEY_JARVIS (mww_model_data_hey_jarvis.h) - COMP_MWW_MODEL_STRAWBERRY (mww_model_data_strawberry.h) - COMP_MWW_MODEL_ORANGE (mww_model_data_orange.h) - COMP_MWW_MODEL_BANANA (mww_model_data_banana.h) Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
…build Align intel_adsp_ace40_nvl.conf with the static build configuration of intel_adsp_ace30_ptl.conf. This enables the static modules needed for Wake-on-Voice and audio feature processing (WOV Arbiter, KPB, ECNS with test signal, MFCC with PCAN, microWakeWord with PCAN and debug trace, DRC, Multiband DRC, TDFB, Aria, MUX, S24_3LE converter). Disables HiFi4 nnlib acceleration for microWakeWord (CONFIG_COMP_MWW_HIFI_NNLIB=n) to fall back to portable TFLM kernels on the HiFi5 DSP core. Disables LLEXT and the dynamic library manager in favor of a static build. Expands heap and malloc arena sizes, sets early P4WQ init for core 0 IDC worker threads, sets 100ms IDC timeout, enables telemetry, and configures DMA LLI pool and L2 MMU tables while preserving Nova Lake specific settings. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Include module toml configurations for WOV arbiter, ECNS, and microWakeWord (MWW) when enabled by Kconfig or when forcing all modular builds. This allows rimage to populate the module entries in the firmware manifest for Nova Lake (NVL) and prevents firmware load failures. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add the lean SSP0 NoCodec Wake-on-Voice topology sof-ptl-nocodec-wov.tplg
(and sof-tgl-nocodec-wov.tplg) defined in cavs-nocodec-wov.conf based on
the single-slot WoV pipeline from sdw-dmic-wov-single.conf.
The topology provides:
- Playback (passthrough): PCM 0 ("Port0", 2ch, 48 kHz, S16_LE / S32_LE) ->
host-copier -> dai-copier -> SSP0 with LBM mode (quirk 64) enabled.
- Capture (Wake-on-Voice): SSP0 DAI copier (loopback, 48 kHz 2ch S16_LE) ->
micsel (stereo downmix to mono 48 kHz 16-bit) -> SRC (48 kHz to 16 kHz
mono 16-bit) -> KPB (2000 ms 16 kHz mono 16-bit).
- Detection slot: KPB pin 0 -> detector copier (16-bit pass-through) ->
MFCC (10 ms hop PCAN) -> microWakeWord (MWW) -> WOV Arbiter pin 1.
- History drain: KPB pin 1 -> WOV Arbiter pin 0 (16-bit) -> host-copier
(PCM 12, 16 kHz mono S16_LE, D0i3-compatible).
This enables automated stress tests and keyword recognition accuracy testing
by playing audio into the SSP0 playback stream and receiving the looped-back
signal on the keyword detection and WoV capture path.
Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
All builds, topologies and models use the int8 Mel features from MFCC, so the int32 Q9.23 input path with the in-component soft AGC and int8 requantization is not used anymore. Remove it together with the COMP_MWW_PCAN option that selected between the two paths. The soft AGC will be added to MFCC as an alternative int8 output mode to PCAN, so MWW only needs to copy the int8 features to the model. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
MWW accepts only int8 Mel features, so the MWW_PCAN=false choice of 32-bit Mel frames does not work anymore. Replace the true/false switch with MWW_FEATURES that names the int8 feature type from MFCC, for now only "pcan". This makes it possible to add other int8 feature types later. The generated WoV topologies are binary identical before and after the change. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add an output mode to the MFCC state that is set once in setup from the configuration blob: cepstral coefficients, int32 Mel or int8 PCAN Mel. The output preparation and the output sample size now use it instead of checking the PCAN enable flag under the Kconfig option in several places. This prepares adding more output data types. The output is bit-exact with the previous version for the cepstral, Mel and PCAN benchmark topologies in testbench. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add AGC8, an int8 Mel output mode that is a light alternative to PCAN. A single gain is added to all Q9.23 Mel log values of a hop. It attacks instantly to keep the hop peak at or below +1.0 and releases towards a target, slower when the VAD reports speech. The values are then clamped and converted to int8 Q1.7. It needs no per-channel noise estimate and no tflite-micro code, and is always built, so one MFCC build supports cepstral, int32 Mel, PCAN and AGC8 output. The algorithm is moved from the removed microwakeword int32 input path and is bit-exact with it. AGC8 is enabled with the new enable_agc8 blob flag and tuned with agc_target, agc_floor, agc_release and agc_release_speech, which take the previously reserved blob space. Zero values select the defaults, so the blob size and existing blobs are unchanged. Setup rejects a blob that enables both PCAN and AGC8, or either of them with cepstral output. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add the mel40_10ms_agc8_compress blob for microwakeword: 40 Mel bins with 10 ms hop, soft AGC int8 output, and VAD enabled for the AGC release and the MWW gating. The blob uses log10 Mel with scale 0.4, so the int8 range covers 50 dB below the AGC normalized peak, and a 3.0 gain target, so quiet input is boosted. The release in silence is 3 dB/s to recover quickly after loud sounds, and 0.1 dB/s during speech. With these settings the features of keywords in background noise are the same for input peak levels from -50 to -6 dBFS. Add the "agc8" choice for MWW_FEATURES in the WoV topologies. The default stays "pcan" and the WoV topologies are binary identical with it. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
EXPORT_SYMBOL() is defined in zephyr/llext/symbol.h. Without LLEXT nothing includes it indirectly, so a static build with the debug stream text messages fails. Include the header directly; the macro is a no-op when LLEXT is disabled. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The model output is a LOGISTIC with the fixed scale 1/256, so the probability is the raw output minus the zero point in Q0.8. Use that directly for the detection average, threshold and the live score instead of a float probability. The detection decisions and the logged values are identical to the float version. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Twister rejects the space separated tags string in the PCAN test case description, so the test could not be run. Use a YAML list like the other math unit tests. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The PCAN strength and offset were passed as floats only to be converted back to Q1.15 and Q8.7 inside the gain lookup function. The MFCC component in turn converted its Q1.15 and Q8.7 blob values to float to call the library. Change the PCAN configuration and the LUT computation API to take the fixed-point values directly. This removes the last floating point use from the MFCC and PCAN code. The output is bit-exact with the previous version. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The size field of the struct sof_abi_hdr in front of a model from the bytes control was used without checking it against the received blob size. A too large value made the model parser read beyond the blob. Reject the blob when the size exceeds the received payload. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The model blob was a bare TFLite flatbuffer with no information about the model. Add struct sof_mww_model_hdr in front of the flatbuffer, for both the built-in models and the bytes control. It has a version and header size for later additions, the model size and CRC32 that are checked before the model is used, the keyword name, the feature type and geometry that are checked against the component, and the trigger threshold and averaging length, so each model carries its own trigger tuning. The arena size field lets a model that needs more than the static tensor arena be rejected, and reserved words leave room for future use. A bare flatbuffer is still accepted and uses the previous default trigger settings. crc32() in numbers.c is exported for this, so that the check also works when the component is built as a loadable module. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Retrain the hi intel, strawberry, orange and banana models, and store all built-in models and sof-ctl blobs, including the upstream hey jarvis model, with the struct sof_mww_model_hdr model header. The models use int8 PCAN features and int8 streaming state, which removes 18 dequantize and quantize operations per inference and makes the models about 10 kB smaller. The trigger settings in the headers are threshold 0.5 and average of 4 probabilities. The hi intel model is trained with negative class weight 3.0 and about 6000 Mozilla Common Voice clips as continuous speech negatives. Keyword recall stays at 99.8 % while false triggers on the Common Voice clips drop from 59.4 % to 2.4 %. It triggers on one of the 11 held-out ODAQ dialog clips, where the previous model triggered on 10 of them. The strawberry and orange models use a new TTS speaker for every clip, the other fruit words as unknown speech, about 6000 Common Voice clips as speech negatives, and augmented recordings of a few real speakers as additional positives. On 120 held-out LibriTTS-R speakers at -40..-10 dBFS strawberry detection is 99-100 % and orange 96-100 %. Orange does not trigger on the other fruits in 960 clips and strawberry triggers on 10 of them. False triggers on the Common Voice clips are 0.3 % for strawberry and 1.4 % for orange. The banana model also uses Whisper screening of the TTS clips, which raises its detection from 82-86 % to 95-98 %. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR introduces multi-slot Wake-on-Voice (WoV) with low-power D0i3 sleep/wake capability, hardware PDM DMIC input, and modular topology feature integration across Panther Lake (PTL), Wildcat Lake (WCL), and Tiger Lake (TGL).
Highlights
Hardware PDM DMIC Wake-on-Voice:
dmic16k, DAI index 1) directly from the SoC PDM decimator, avoiding software resampling.dmic01) and external SoundWire microphones (Capture-SmartMic).Multi-Slot WoV Pipeline Architecture:
DMIC Multi-WOV).D0i3 Low-Power Sleep & Wake:
capture_compatible_d0i3 1to maintain low-power DSP SRAM retention across host S0ix sleep.COMP_STATE_PREPAREand suppress telemetry during D0i3.Topology Integration & Modularity:
dmic-wov-feature.confallowing multi-slot WoV to be included as a feature module into functional topologies (INCLUDE_WOV=multi).sof-ptl-rt721-4ch-wov-multi,sof-ptl-rt722-4ch-wov-multi, and WCL equivalents.Toolchain & Compiler Fixes:
CALL8out-of-range relocation in the libc shim by unifying.textsection placement.ibuffer_allocator.ccto support both upstream TFLM and trees carrying Xtensa ICF vtable fixes.Validation
test_startandwov_blocking_read.alsatplg.