一個基於BERT與孿生架構的檢索模型

A BERT-based Siamese-structured Retrieval Model

由於深度學習的發展，在以Transformer為架構的雙向編碼器BERT的帶領下，自然語言處理的相關任務獲得長足的進步。資訊檢索任務是從大量的文件中，尋找出與使用者查詢最相關的結果。雖然基於BERT的檢索模型已在許多研究中展現優異的任務成效，但這些模型通常有著計算量龐大或需要大量額外儲存空間的問題。有鑑於此，本研究提出一套基於BERT的孿生架構的檢索模型，不僅擁有以預訓練語言模型為主體的優點，更具備了自動查詢擴增技術，與使用強化學習於模型訓練。因此，我們所提出的檢索模型不僅改善了現有方法的問題，也在三個公開的大型資料集中，驗證了它的檢索成效。

Due to the development of deep learning, the natural language processing tasks have made great progresses by leveraging the bidirectional encoder representations from Transformers (BERT). The goal of information retrieval is to search the most relevant results for the user's query from a large set of documents. Although BERTbased retrieval models have shown excellent results in many studies, these models usually suffer from the need for large amounts of computations and/or additional storage spaces. In view of the flaws, a BERT-based Siamese-structured retrieval model (BESS) is proposed in this paper. BESS not only inherits the merits of pre-trained language models, but also can generate extra information to compensate the original query automatically. Besides, the reinforcement learning strategy is introduced to make the model more robust. Accordingly, we evaluate BESS on three public-available corpora, and the experimental results demonstrate the efficiency of the proposed retrieval model.