通過卷積多視角注意力和SudoNet進行高效的人聲分離

ESC MA-SD Net: Effective Speaker Separation through Convolutional Multi-View Attention and SudoNet

本研究以人聲分離(speech separation)為主題，研究如何將混合的多個人聲信號成功分離。我們是利用端到端(end-to-end)的高效語音分離模型SuDoRM-RF做為基礎，並結合了MANNER模型中的殘差卷積轉換器區塊(Residual Conformer Block)以及多視角注意力區塊(Multi-view Attention block)來達到高效的語音分離模型ESCMA-SD Net。本模型中殘差卷積轉換器區塊在於移除無用資訊的同時還能保留重要語音信息，而透過多視角注意力模塊則用以關注擷取對各個面向語音特徵，如此一來，我們將可以得到相較原本SuDoRM-RF模型更加高效的語音分離模型ESC MA-SD Net。在我們的實驗中，分別從驗證資料(Validation dataset)以及時頻圖(Spectrogram)來展示提出之方法的良好的語音分離成效。

This study focuses on speaker separation, investigating how to successfully separate mixed multiple speech signals. We build upon the efficient end-to-end speech separation model SuDoRM-RF and integrate the Residual Conformer Block from the MANNER model along with the Multi-view Attention block to create the efficient speech separation model ESC MA-SD Net. The Residual Conformer Block in this model eliminates irrelevant information while preserving crucial speech details. The Multi-view Attention module is employed to capture diverse aspects of speech features. By doing so, we achieve a more efficient speech separation model, ESC MA-SD Net, compared to the original SuDoRM-RF model. In our experiments, we demonstrate the effectiveness of the proposed method using validation data and spectrograms to showcase the improved speech separation performance.