聽障者多模中文口語訓練模型與分析

Multimodal Speech Training for the Hard of Hearing in Mandarine

本研究為協助聽障人士提升口語發聲能力和溝通品質，提出將語音的聲學模型轉換以視覺形式呈現，輔助聽障人士進行發聲訓練，並建構口語發聲練習平台，提高聽障人士發聲練習的自主性，為聽障人士實現更加明確的聽與更清晰的說的目標。
本研究將韻母、聲母與聲調等語音元素的聲音訊號，藉聲學模型轉換為可對應發聲原理的頻譜分布與動態變化。學習者可看見韻母發聲過程的軌跡，是否符合口腔與舌位的變化，聲母的發聲動作與出氣方式是否有正確的隨時間變化，聲調的掌握是最困難的，本研究採用以3個狀態過程來描述4種聲調。經過聽覺訊息視覺化以及視覺與觸覺的多模回饋，聽障者可以藉以自主練習提升發音的清晰度。

This study aims to assist the hearing-impaired in improving their vocal ability and communication quality and proposes to convert the acoustic model of speech into visual form to assist the hearing-impaired in self-vocalization training. Establish a platform for oral vocalization practice to assist the hearing-impaired to learn independently. The core goal of this platform is to enable hearing-impaired people to hear more clearly and speak more clearly.
In this study, the sound signals of speech elements such as vowel, consonant, and tone were converted into spectral distribution and dynamic changes that could correspond to the principle of sound pronunciation through the advanced model. Learners can see the trajectory of the vowel vocalization process, whether it meets the changes in the oral cavity and tongue position, whether the consonant's vocal action and exhalation mode change correctly with time, and the mastery of tone is the most difficult, this study uses 3 state processes to describe 4 tones. Through the visualization of auditory information and multimodal feedback of sight and touch, the hearing impaired can improve their pronunciation through self-practice.