A single-channel recording containing a target speaker s, K interfering speakers vk and ambient noise n is modeled convolutively, under two hypotheses:
where h are impulse responses. Define ys = hs∗s and yv,k = hk∗vk, the source images as received at the microphone: all scoring below (SINR, SI-SDR, the objective) is computed against ys, so a reverberant experiment scores against the reverberated target. The core benchmark below is anechoic — h = δ, hence ys = s — with K = 1 and additive recorded noise; the reverberant condition in §4 uses synthetic h ≠ δ. Recovering s from x is ill-posed (the map to x is many-to-one), so no inverse exists without side information. Difficulty on the primary grid is indexed by the signal-to-interference-plus-noise ratio
SINR = 10·log10 ( ‖ys‖² / ‖Σk hk∗vk + n‖² ) [dB]with primary evaluation at SINR ∈ {−5, 0, +5, +15} dB. −5 dB is the lowest nominal SINR on that grid — not a claim of "hardest possible": reverberation and higher K can create harder conditions at equal SINR, as §4 shows.
Sonacern conditions extraction on a reference recording r of the target speaker, via a learned speaker representation e(r):
ŷs = fθ ( x , e(r) )Signal fidelity is scale-invariant SDR. Implementation, fixed for all numbers on this page: signals truncated to the shorter length; mean-subtracted; no time-alignment compensation (sources are synchronous by construction); the numerical floor ε appears explicitly below:
q(z) = max(‖z‖², ε), ε = 10⁻⁸ (the floor breaks exact scale invariance only in the degenerate regime ‖z‖² < ε; it is inactive on all benchmark signals) · α = ⟨ŷs, ys⟩ / q(ys) · SI-SDR(ŷs, ys) = 10·log10 ( q(αys) / q(ŷs − αys) )Where a Δ appears between two systems it is a paired difference on identical mixtures, Δ = SI-SDR(ŷ_(s,A), y_s) − SI-SDR(ŷ_(s,B), y_s) [subscripts written inline to survive text extraction]; improvement over the raw mixture, SI-SDR(x, ys), is not the reported quantity unless stated.
Task usefulness is corpus word error rate. Protocol, fixed: recognizer = faster-whisper "small", float16, beam 5, language forced to English; references are the corpus's official transcripts; normalization lowercases, strips punctuation, tokenizes on whitespace (numerals as produced); Ei = word-level edit distance, Wi = reference word count:
cWER = Σ Ei / Σ Wi, Σ Wi = 1 725 per SINR condition (N = 100 items)Historical note for archive matching: early internal logs reported macro-WER (per-item mean of Ei/Wi), which differs from cWER on the same run — e.g. the easy/mid baseline at −5 dB reads 0.552 in macro form, 0.522 in corpus form. This page uses cWER throughout.
Development proceeded through generations under a one-factor rule (single training runs; caveats below). The decisive experiment compared the incumbent generations against one retrained identically except for its training corpus. Model identities are the internal codenames used in the archive: ct5 (best prior, easy/mid regime), ct7 (best prior, hard regime), ct10 (corpus-scaled). Corpus change: audio ≈ 4.6× (100 → 460 h), speaker count ≈ 4.7× (251 → 1 172). These co-varied, along with whatever else differs between corpus configurations (phonetic content, channels, speaker profiles): the observed difference is associated with replacement by the larger corpus configuration; individual contributions of duration, speaker count and composition are not identifiable from this design. Baselines ct5/ct7 were themselves selected as "best prior" on this same benchmark — a selection-on-benchmark bias acknowledged here, and one more reason the banner labels everything diagnostic. The comparison protocol (endpoints, correction, cluster unit) was frozen in the archive before the final run.
| cWER | −5 dB | 0 dB | +5 dB | +15 dB |
|---|---|---|---|---|
| ct5 | 0.522 | 0.220 | 0.085 | 0.041 |
| ct7 | 0.454 | 0.216 | 0.104 | 0.045 |
| ct10 | 0.339 | 0.164 | 0.078 | 0.038 |
Raw edit totals over ΣW = 1 725, for archive matching — −5 dB: 901 / 783 / 585; 0 dB: 380 / 372 / 283; +5 dB: 146 / 179 / 135; +15 dB: 70 / 78 / 66 (ct5 / ct7 / ct10).
| Comparison | ΔcWER | Relative | 95% cluster CI | raw p (exact) | Holm-adj p |
|---|---|---|---|---|---|
| ct10 − ct5, −5 dB | −0.183 | −35% | [−0.240, −0.130] | 8.2·10⁻⁸ | 3.3·10⁻⁷ |
| ct10 − ct7, −5 dB | −0.115 | −25% | [−0.161, −0.075] | 5.9·10⁻⁷ | 1.8·10⁻⁶ |
| ct10 − ct7, 0 dB | −0.052 | −24% | [−0.089, −0.021] | 0.00188 | 0.00376 |
| ct10 − ct5, 0 dB | −0.056 | −25% | [−0.105, −0.014] | 0.01863 | 0.01863 |
Test specification. Cluster unit = target speaker; C = 38 clusters over the 100 items. Statistic Tobs = ΔcWER = Σc dc / W, with dc the per-cluster edit-count difference and W = 1 725. Null: exchangeability of model labels within each cluster, implemented as cluster-level sign flips. Two procedures: (i) Monte-Carlo, B = 10⁵ sign draws, seed 1234, p = (1 + #{|Tperm| ≥ |Tobs|})/(B+1), resolution floor ≈ 10⁻⁵ — all four comparisons passed Holm at α = 0.05 under it; (ii) an independent recomputation by the auditing system evaluating the exact sign-flip distribution over all 2³⁸ patterns (computed distributionally rather than by explicit pattern enumeration), yielding the exact p-values above. One sign matrix was shared across the four tests as a computational-consistency choice — not a requirement of Holm, which controls FWER under arbitrary dependence. CIs are cluster-bootstrap percentile intervals, B = 10⁴, seed 1234. All inference on this page is one-way (target-speaker) clustering. Interferer speakers recur across target clusters, so cross-speaker dependence is not fully resolved: a separate one-way interferer-cluster sensitivity (which also passed 4/4) does not address both dependencies jointly. The required upgrade — treating connected components of the target–interferer graph as independent blocks, or multiway cluster resampling — is a listed obligation; until then, one-way results carry that reservation.
Signal-side numbers (same run, recognizer-independent), mean [95% cluster CI], N = 100:
| Model | SI-SDR, −5 dB | STOI, −5 dB | ESTOI, −5 dB | SI-SDR, 0 dB | STOI, 0 dB | ESTOI, 0 dB |
|---|---|---|---|---|---|---|
| ct5 | 1.92 [0.17, 3.79] | 0.723 [0.683, 0.765] | 0.540 [0.505, 0.578] | 7.06 [5.89, 8.28] | 0.831 [0.805, 0.857] | 0.684 [0.656, 0.712] |
| ct7 | 3.33 [1.68, 5.07] | 0.749 [0.713, 0.787] | 0.563 [0.531, 0.597] | 7.79 [6.61, 8.96] | 0.837 [0.813, 0.862] | 0.691 [0.666, 0.717] |
| ct10 | 6.25 [4.58, 7.80] | 0.810 [0.780, 0.839] | 0.653 [0.622, 0.685] | 10.04 [8.87, 11.13] | 0.878 [0.858, 0.897] | 0.760 [0.736, 0.783] |
The +4.3 dB figure quoted elsewhere is the −5 dB ct10 − ct5 mean difference (6.25 − 1.92 = 4.33).
Protocol: for each item the system received, in place of the target's reference, a different utterance of the interfering speaker — one not used inside the mixture, duration-matched to the correct reference. "Same-chapter" / "cross-chapter" denote the recording condition of that reference relative to the interferer's in-mixture utterance (same or different audiobook chapter; no claim about recording sessions is made). Mean SI-SDR with cluster-bootstrap 95% CIs, −5 dB:
| Condition | N | est → target | est → interferer |
|---|---|---|---|
| Correct reference | 100 | +6.25 [+4.58, +7.80] | — |
| Interferer reference, same-chapter | 100 | −23.23 [−24.02, −22.44] | +14.28 [+13.89, +14.64] |
| Interferer reference, cross-chapter | 78 | −21.44 [−22.80, −19.94] | +13.22 [+12.03, +14.18] |
Item-level failure rate: with yr the reference-indicated source and yo the strongest non-indicated source (here, the original target), define Di = SI-SDR(ŷi, yr,i) − SI-SDR(ŷi, yo,i); a steering failure is Di < mS, with the margin mS to be frozen before any confirmatory use. Under a provisional mS = 0, recomputation from the archived caches gives: same-chapter 0/100 failures (−5 dB), 1/100 (0 dB); cross-chapter 2/78 (−5 dB), 4/78 (0 dB) — development diagnostics; one-sided bounds under a frozen margin belong to the holdout.
N = 78 because 9 interferer speakers have no second chapter in the corpus — a structural exclusion, not random; the subsets are therefore not directly comparable and no paired same-vs-cross difference is reported. Interpretation on record: behavioral evidence of reference conditioning with same-speaker/different-utterance references; embedding-similarity selectivity and out-of-corpus generalization not measured.
Reproducibility, by level. (1) Metric replay: the statistical chain recomputes bit-identically from archived per-item caches. (2) Inference reproduction: a 20-point spot check after a hardware migration reproduced cached outputs to 0.000 dB; a 12-combination tally check independently re-derived the corpus edit totals. (3) Training reproduction: not attempted; training RNG state is not checkpointed, so bit-level training reproduction is not claimed. What was previously called "independent audit" is, precisely: independent recomputation of metrics from archived outputs by AI referee systems separate from development — not peer review.
Mean est→target SI-SDR with cluster-bootstrap 95% CIs (N = 100 unless noted):
| Condition | −5 dB | 0 dB | Status |
|---|---|---|---|
| Base (K = 1, full reference) | +6.25 [+4.58, +7.80] | +10.04 [+8.87, +11.13] | reference point |
| 1-second reference (first 1 s of the clip; single crop) | +5.84 [+4.33, +7.26] | +9.67 [+8.41, +10.78] | mean −0.4 dB; at −5 dB, 10/100 items lose > 3 dB |
| Common-RIR spectral-smearing stress, RT60 = 0.3 s | −4.58 [−5.09, −3.97] | +1.53 [+1.01, +2.10] | 8–12 dB drop; see design note |
| Common-RIR spectral-smearing stress, RT60 = 0.6 s | −5.56 [−5.98, −5.06] | +0.83 [+0.40, +1.32] | |
| K = 2 interferers | −0.99 [−1.89, −0.04] | +5.40 [+4.39, +6.41] | insufficient at −5 dB |
| K = 3 interferers | −1.63 [−2.40, −0.89] | +5.19 [+4.34, +5.96] |
Design notes. Multi-interferer: interferers are power-equalized and their sum is scaled against the target — total interference power held constant across K. Decompose the denominator: SIRtotal = 10·log₁₀(‖ys‖²/‖Σvk‖²) for speech interference alone; SNRnoise = 10·log₁₀(‖ys‖²/‖n‖²) for ambient noise. Per-interferer SIR ≈ SIRtotal + 10·log₁₀K when interferers are power-equalized and cross-energy terms are negligible; exact equality requires ⟨vi, vj⟩ = 0 for i ≠ j. The relation concerns the interference term only and does not follow from total SINR when n ≠ 0. Reverberation: synthetic impulse responses (exponentially decaying noise with a unit direct-path tap — a stress stimulus, not a room simulation; no geometry, distance or DRR modeled); one deterministic RIR per item per RT60; the same RIR applied to target and interferer; each source peak-normalized after convolution; scoring reference = the reverberated target; training contained no reverberation. Real measured rooms: unmeasured.
Target absence is reported under H₀, where SINR is undefined. Energy behavior:
A = 20·log10 ( max(rms(ŷs), εr) / max(rms(x), εr) ) | H₀, εr = 10⁻⁶ (mean-square floored at 10⁻¹² in the implementation) · median +28.45 (P10 +26.51, P90 +30.71) → after phase 1: −22.87 (P10 −27.69, P90 +26.19) · gate median(A) ≤ −30 not metExact tail counts after phase 1: 0/100 items reached −30 dB; 19/100 remained above 0 dB; 18/100 above +20 dB — a bimodal outcome. The recognizer produced at least one word on 100/100 items before and 99/100 after (median 17 words). A control that is missing and pending: recognizer word production on pure silence and on the raw H₀ mixture — without it, the split between residual leakage and recognizer hallucination is unknown. The target-present side simultaneously regressed: ΔSI-SDR −0.68 [−1.31, −0.21] at −5 dB, −0.53 [−0.77, −0.33] at 0 dB. Verdict on record: energy suppression alone is insufficient; phase 1b introduces an explicit per-frame presence decision dθ.
Silence must be guaranteed in three separable senses — energy, decision, semantics — because our own phase-1 result shows the first does not imply the third. End-state constraints, all under H₀:
P( A > −30 dB ) ≤ εA ∧ P( dθ = 1 ) ≤ εFA ∧ P( WASR > 0 ) ≤ εASRand under H₁, steering and deployment constraints:
P( steering failure ) ≤ εS ∧ RTF ≤ 1 ∧ L95 ≤ LmaxRTF (real-time factor) is meaningful only against a declared reference device; hardware, sample rate, audio length and batching will be frozen together with (ε, δ). L95 = 95th percentile of end-to-end wall-clock latency from audio-frame arrival to output emission; Lmax frozen in advance. Steering failure is the event defined in §3.
with the objective
maxθ E[ SI-SDR( fθ(x, e(r)), ys ) | H₁ ]Every probability constraint is to be verified on finite data through a one-sided confidence bound at level 1 − δ, with all (ε, δ) frozen in advance. Sample-size planning follows — e.g., demonstrating a failure rate below 1% at 95% confidence with zero observed failures requires ≈ 299 independent clusters; more under dependence. The −30 dB energy threshold is a pre-specified engineering choice, currently without a psychoacoustic or downstream-ASR derivation; producing that justification (or revising the threshold) is itself a listed obligation. The phase-1b criteria already frozen in the development archive are retained as a model-selection screen: median(A) ≤ −30 ∧ ASR-active items ≤ 10/100 ∧ cluster-TOST(ΔSI-SDR, ±0.5 dB, α = 0.05) ∧ UCB95%(ΔcWER) ≤ +0.02. This screen may guide development, but it is not a silence guarantee or a release gate; release claims require the probability bounds above, with one-sided confidence at 1 − δ, on a fresh holdout. The distinction prevents a median or a 100-item count from being mistaken for a tail guarantee.
Standing obligations: (1) a fresh single-use holdout for confirmatory claims; (2) a dependence-aware analysis unit (connected components of the target–interferer graph) or multiway-cluster method; (3) multi-seed replication before causal attribution; (4) controlled human listening tests before user-facing benefit claims; (5) silence and raw-H₀-mixture ASR controls; (6) a psychoacoustic or downstream-task basis for the −30 dB threshold, or a revised threshold; (7) paired uncertainty intervals for signal-side metrics; (8) item-level steering-failure bounds under a frozen margin; (9) real-room, causal, device-level RTF and latency tests; (10) training reproduction with checkpointed RNG state.
| Item | Identity |
|---|---|
| Checkpoints (full SHA-256) | ct5 fa5bd18e73535c385963f7e657b62b38a3ca3a203a18fe26ea77648b7bbfe016 ct7 8b63a516e039442f237f46ed59ec5f5ae007735eda3c6c21a78706e95a04d1b9 ct10 6ba2b57613fea6964667a1da4c9540a5957b15b9d6459ca32ee61d59739121c7 |
| Test manifest | 100 items, 2-speaker mixtures, unseen speakers; manifest SHA-16 2612dac74f0c65a7 |
| Result records | final-eval log + per-item cache (Ei, Wi, speaker ids per item); steering, stress and target-absent runs each with their own per-item caches and logs |
| Statistics code | cluster permutation + Holm script at archive commit 067eb21; bootstrap-CI diagnostics alongside |
| ASR | faster-whisper "small", float16, beam 5, language=en. Outputs are not bit-reproducible across runs (known limit); the transcripts actually produced in the corpus evaluations are archived per item. Exact faster-whisper/CTranslate2 revisions were NOT retained, so audio-to-text regeneration is not claimed to be bit-reproducible; reported cWER values remain replayable from archived per-item edit counts, and produced hypotheses are archived per item. Confirmatory work must freeze package revisions, decoding options and normalized hypotheses. |
| Metric | cWER throughout this page; macro-WER appears only in historical logs |
The archive is private; the identities above allow item-level verification under audit access. Public confirmatory artifacts will accompany the holdout study.
TSE overview: Zmolikova et al., IEEE SPM 2023 · SI-SDR: Le Roux et al., ICASSP 2019 · STOI: Taal et al., IEEE TASL 2011 · ESTOI: Jensen & Taal, IEEE/ACM TASLP 2016 · Holm: Holm, Scand. J. Statist. 1979 · Cluster bootstrap: Field & Welsh, JRSS-B 2007 · TOST: Schuirmann, 1987