資訊擷取與知識注入技術於機器閱讀理解之研究

Abstract

近年來，預訓練上下文語言模型 (Pre-trained Contextualized Language Modeling, PCLM) 的出現，使得基於 PCLM 的方法在各種機器閱讀理解 (Machine Reading Comprehension, MRC) 與對話式機器閱讀理解 (Conversational MRC, CMRC) 都有非常優秀的表現。然而，在機器閱讀理解領域仍然較少研究琢磨於開放領域知識 (Open-domain Knowledge) 與域內知識 (In-domain Knowledge) 的運用。有鑑於此，本論文提出一種針對MRC與CMRC的有效建模方法。此方法具有兩個主要的特點：首先，針對文章段落進行訊息提取 (Information Extraction, IE) 的預處理，藉此將每個文章段落聚類成一個偽類 (Pseudo-class) 以提供PCLM 進行訊息增強，進而提升後續 MRC與CMRC的任務表現；另一方面，本論文提出了一種新的知識注入 (Knowledge Injection, KI) 方法，將開放領域知識 (Open-domain Knowledge) 與域內知識 (In-domain Knowledge) 注入至 PCLM ，藉此捕捉更為精準的問題與文章段落間的相互關係。本論文將實驗結果與數個當今最佳的方法進行比較，除了在多個MRC與CMRC資料集上都有一定程度的表現外，大量的實證實驗也證明了本論文方法的有效性與可行性。In the recent past, pre-trained contextualized language modeling (PCLM) approaches have made inroads into diverse tasks of machine reading comprehension (MRC), as well as conversational MRC (CMRC), with good promise. Despite the success of these approaches, there are still not many efforts on the integration of either open-domain or in-domain knowledge into MRC and CMRC. In view of this, we propose in this thesis an effective modeling method for MRC and CMRC, which has at least two distinctive characteristics. On one hand, an information extraction (IE) preprocess is conducted to Cluster each paragraph of interest into a pseudo-class for the purpose to provide augmented information for PCLM to enhance MRC and CMRC performance. On the other hand, we also explore a novel infusion of both open-domain and in-domain knowledge into PCLM to better capture the interrelationship between a posed question and a paragraph of interest. An extensive set of empirical experiments carried out on several MRC and CMRC benchmark datasets indeed demonstrate the effectiveness and practical feasibility our proposed approach in comparison to some top-of-the-line methods.