探究端對端語音辨識於發音檢測與診斷

Investigating on Computer-Assisted Pronunciation Training Leveraging End-to-End Speech Recognition Techniques

電腦輔助發音系統（Computer assisted pronunciation techniques, CAPT），任務可分為錯誤發音檢測（Mispronunciation detection）以及錯誤發音診斷（Mispronunciation diagnosis）。在過往的研究中，這兩種任務主要依賴於傳統語音辨識系統的強制對齊（Forced alignment）方法，並利用強制對齊產生的音素（Phone）段落與觀測到的全部音素或較混淆的音素計算GOP（Goodness of pronunciation）分數，並以此作為發音好壞的依據。然而傳統語音辨識系統的訓練流程既冗長且複雜。近年來，端對端語音辨識系統不僅大幅簡化此問題，且效能也有追上傳統語音辨識的趨勢。因此，本論文將基於端對端架構下，分別探討（1）基於辨識產生的信心分數（Confidence score）；（2）基於語音辨識結果，兩者對於發音檢測任務的影響。實驗結果顯示，使用端對端架構進行發音檢測與診斷，不僅相較於以往基於傳統語音辨識架構有更少的訓練流程，也大幅提升檢測與診斷的效果。

One of the primary tasks of a computer-assisted the pronunciation techniques (CAPT) system is mispronunciation detection and diagnosis. Previous research on CAPT mostly relies on a forced-alignment procedure which is usually conducted with the acoustic models adopted from a traditional speech recognition system, in conjunction with a phoneme paragraph, to calculate the goodness of pronunciation (GOP) scores for the phonemes of spoken words with respect to a text prompt. However, the training process of the traditional speech recognition system is complicated. In recent years, the end-to-end speech recognition system has not only greatly simplified this problem, but also has the trend of catching up with traditional speech recognition. In view of this, this thesis sets out to conduct mispronunciation detection and diagnosis on the strength of end-to-end speech recognition. To this end, we design and develop two mispronunciation detection methods: 1) method leveraging a recognition confidence measure; 2) method simply based speech recognition results; A series of experiments showed that leveraging end-to-end speech recognition architecture on mispronunciation detection and diagnosis not only reduced the training steps originally required for traditional speech recognition but also improve the performance of detection and diagnosis significantly.