Why does removing only 4 bytes from the beginning of the audio causes more wave to be chopped off?

[gexgd0419]

Here is a weird problem I discovered when trying to fix the bug mentioned here, reported by @Danstiv, introduced by the leading silence removal feature.

The leading silence removal feature works by removing some silent audio data before sending the audio data to the output device. However, even removing just 4 bytes from the audio can sometimes make some audio segments sound like they were "chopped off" at the beginning. Also, not all segments are chopped off, only some of them. If you wait for a few seconds before making the next sound, the next audio segment will be more likely to be chopped off.

To reproduce the issue: (requires modifying some code)

1. Checkout a version before the leading silence removal feature was introduced. This is to eliminate the possible impacts from this feature, and to prove that this issue exists without this feature.
2. In file /source/tones.py, line 89, change player.feed(buf.raw) to player.feed(buf.raw[4:]), removing the first 4 bytes from all beep sounds. As beep sounds are 16-bit stereo, 4 bytes are one sample.
3. Launch NVDA, then switch the speech mode to "beeps".
4. Listen to the beeps carefully. Once in a while, you may find some beeps being "chopped off" from the beginning.

The chopping off is audible, although you may have to pay close attention. I checked the waveform of the output audio, and the waveform is also different. Regular beeps look more like a rectangle (actually sine waves when zoomed in), but the chopped off beeps look like a "fade in" effect was applied to them: the volume goes from zero gradually to the current level. Just because a single wave sample is removed from the beginning. And if the change to tones.py is reverted, no chopping off happens anymore.

This weird problem is bugging me. I'm not an expert on audio processing. I couldn't find the cause of it, and I'm not sure if this problem only exists on my system.

Could someone help me test if the problem exists on their system? Could someone give me some idea about the cause of this problem?

cc @jcsteh

[jcsteh]

I don't have any real idea, but here's a hypothesis: could this be something the Windows mixer is doing? Does trimming 1 sample from the start result in a wave form that the Windows mixer somehow thinks might produce clicks, etc., so it fades it in? Could we solve that by leaving 1 sample of silence at the start of the audio?

[gexgd0419]

Can this be reproduced on your system?

[gexgd0419]

I did another test. I changed the first sample of the beep to maximum +1.0, so that there's a spike at the beginning of the beep:

data = bytearray(buf.raw)
data[0:4] = b'\xff\x7f\xff\x7f'  # 0x7fff, 0x7fff
player.feed(bytes(data))

And sometimes the spike will be chopped off as well.

If I put the spike in the second sample, keeping the first sample zero, then the spike will not be chopped off.

[gexgd0419]

Seems that adding an extra zero sample could solve the problem of chopping off audio.

[SaschaCowley]

@gexgd0419 I experience the premature cutting of speech on my system - I had to turn the feature off, because I was struggling to tell letters apart (using eSpeak at rate boost 35). Haven't tried to repro the beeps issue