Skip to content

libDF: interfering-speaker RIR is applied to the target instead of the interferer #701

Description

@svaningelgem

Scope first, so this doesn't read as alarming: the released DeepFilterNet3 config ships p_interfer_sp = 0.0, so this branch never executes with the default recipe and the published weights are unaffected. That is also the likely reason it hasn't been reported before. It only bites if you turn interfering speech on.

In the interfering-speaker branch, the sampled RIR is applied to the target speech rather than to the interferer.

https://github.com/Rikorose/DeepFilterNet/blob/main/libDF/src/dataset.rs#L1327

if self.p_interfer_sp > 0. && self.p_interfer_sp > rng.uniform(0f32, 1f32) {
    // Add an interfering speaker to noise
    for _ in 0..rng.uniform(1, 3) {
        ...
        let mut sample = self.read_max_len(&sp_name, &sp_key, Some(n_read))?;   // the interferer
        if let Some((rir_name, rir_key)) = self.rir_keys.iter().choose(&mut rng) {
            let rir = self.read(rir_name, rir_key)?;
            self.reverb.transform_single(&mut speech, rir)?;                    // <-- `speech`, not `sample`
        }
        ...
        interferers.push(sample);                                              // pushed un-reverberated
    }

sample is the interferer that ends up in interferers; speech is the target. Two consequences:

  1. The interferer is never reverberated, so an interfering speaker is always dry — which removes the cue that would distinguish it from the near speaker.
  2. The target becomes reverberant, and the input does not. speech_distorted is cloned from speech at line 1283, before this block, and it is what forms the mixture passed on as noisy. So the reverb lands only on the target. That inverts the usual arrangement and trains the model toward adding reverberation rather than removing it. It also compounds, since the loop runs 1–3 times.

Suggested fix, if I've read it right:

self.reverb.transform_single(&mut sample, rir)?;

For transparency: this comes from reading the code, not from an A/B of trained models — I hit it while building interfering-speech mixtures for a 48 kHz model and set p_interfer_sp = 0.0 to construct the interference externally instead. Happy to open a PR if the one-line change is what you want.

Observed on d375b2d, libDF 0.5.7-pre.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions