科學文章摘要自動化概念比對計分方式的發展與應用

A Comparative Study on Automatic Scoring Methods for Summarization of Science Articles

本研究目的在發展國小六年級學生適用的科學文章摘要自動化概念比對評分系統，文中以科學文章摘要人工評分為效標，同時以科學閱讀理解和在校學科成績為相關變項，比較概念比對（CS）、潛在性語意分析（LSA）和關鍵詞比對（KWC）三種自動化計分方法與關聯變項的關連組型，作為科學文章摘要自動化概念比對評分機制的效度討論依據。研究結果顯示以人工評定為效標，CS法表現較佳。如果看摘要能力與科學閱讀理解表現的相關，LSA 評分結果與理解相關略高於CS。以在校成績為關聯變項，其相關組型顯示CS 與LSA 與在校國語與自然相關較高，呈現合理的幅合和區別效度。整體而言，CS 法呈現經濟的優勢和文章特定的限制，相反地，LSA 需要龐大的資料庫但應用的類推潛力可能較佳。

The purpose of this study is to compare the performances of three automated scoring methods on science reading summarization. Human rating is used as the criterion. The automatic scoring methods included are concept similarity (CS)， latent semantic analysis (LSA) and key-word comparison (KWC). Four expository passages on science were used in this study. Science reading comprehension and school grades on Mandarin， science and mathematics were also collected for convergent and discriminant validity discussions. Two facets of summary are rated by human raters: key concept and the structure of reading passage. On the facet of key concept， the correlation coefficients between automated scoring and human rating are around .88 for CS， .72 for LSA， and .63 for KWC. On the facet of structure， the correlation coefficients are .84 for CS， .69 for LSA and .52 for KWC. The correlation coefficients of automated scoring with science reading comprehension are .38 for CS， .45 for LSA and .21 for KWC. Generally speaking， CS and LSA demonstrate promising potential for further technical and application researches.