基於BERT模型之多國語言機器閱讀理解研究

Multilingual Machine Reading Comprehension based on BERT Model

在網路資訊爆炸的現代，人們的生活與網路已密不可分，但受限於檢索技術的瓶頸，雖然能提供多方面的資訊來源，卻不一定是最相關有幫助的資訊。自然語言中的兩個主題：機器問答（Question Answering）與機器理解（Machine Comprehension），由於對檢索系統，以及服務轉型中重要的聊天機器人，都具有高度相關，因此成為近年熱門的研究議題。本論文使用了Google BERT的pre-trained model進行詞嵌入向量，以單詞及單字為單位，組織出一個句子的特徵。並且基於問題、答案、與文本間不同組合的答題策略，最終選擇最高餘絃相似度的選項，作為機器作答的依據。本論文分別實驗在英文TOEFL-QA資料集，以及中文開放性問答資料集，對比於雙向GRU以及A Strong Alignment IR Baseline的方法，分別取得34.87％及57.5％準確率，實驗結果顯示，雖然不同語言之間具有文法的差異，但本論文所提的方法具有一定程度多國語言的通用性。

In recent years, Internet provides more and more information for people in daily life. Due to the limitation of information retrieval techniques, information retrieved might not be related and helpful for users. Two research topics in natural language processing have attracted much attention due to the important applications of information retrieval and chatbot in the past few years: question answering and machine comprehension. In this paper, we use Google BERT pre-trained model as a word embedding model to form semantic sentence features based on single words and phrases. Based on different strategies for question answering, we use cosine similarity to calculate similarity and choose the option of highest cosine similarity score as machine inferenced answer. In our experiments on TOEFL-QA dataset for English and Formosa Grand Challenge dataset for Chinese, our proposed method was compared with Bi-directional GRU and a strong alignment IR baseline, and obtained an accuracy of 34.87% and 57.5%, respectively. With the grammar difference between difference language, our model is capable of processing multilingual questions with comparable performance to existing methods.