Deep Speech Inpainting of Time-frequency Masks
Table 3: Blind speech inpainting (section 3.2). STOI & PESQ were computed between the recovered and the actual speech segments from validation set and averaged. The best scores were denoted in bold. PC - informed (partial conv.), FC - blind (full conv.) inpainting.
['Intrusion', 'Size', 'Gaps STOI', 'Gaps PESQ', 'Noise STOI', 'Noise PESQ', 'PC - gaps STOI', 'PC - gaps PESQ', 'FC - gaps STOI', 'FC - gaps PESQ', 'FC - noise STOI', 'FC - noise PESQ', 'FC - additive STOI', 'FC - additive PESQ']
All of the considered framework configurations successfully recovered missing or degraded parts of the input speech resulting in improved STOI and PESQ scores. The framework for the informed inpainting (PC-gaps) yielded better results, as compared to its blind counterpart, when the masked parts of the input were set to either zeros or random noise (FC -gaps, -noise, respectively). Notably, when input speech was mixed with, not replaced by, the high-amplitude noise, the framework operating in the blind setup (FC-additive) led to the best results for larger intrusions of all the considered shapes. These results suggest that the framework for the blind speech inpainting takes advantage of the original information underlying the noisy intrusions, even when the transient SNR is very low (below -10 dB).