Gender-Specific Speech Enhancement Architecture for Improving Deep Neural Networks Learning

Deep learning techniques for speech enhancement rely on training a deep neural network to process noisy speech, regardless the gender of the speaker. However, research shows that the speech of male and female stimulates different parts in human brain, and that female speech requires more complex analysis. This implies that different processing is applied on the speech, based on the speaker gender. In this work, we argue that male and female speeches have different features that can help in the learning process of speech enhancement deep neural networks if the training is performed on male and female speech data, independently, and using two different deep neural networks, specifically implemented for enhancing the clean speech signal of the target gender. This work presents a genderspecific speech enhancement architecture, which consists of a front-end binary classifier to detect the speaker gender. Based on the classifier decision, the noisy speech is enhanced using either a male or female speech enhancement model. One-stage and twostage speech enhancement approaches are used to process male and female speeches, respectively. The results reveal that genderspecific speech enhancement has positive impact on the enhanced speech by deep neural networks. Additionally, the developed architecture achieved classifier accuracy 96.9% and 0.11 increase in Covl speech quality metric for the test data, in comparison to other best-performing networks.