透過log數據探析澳門學生的問題解決行為： 以PISA 2012的公開題為例

Analysis of Log File Data to Understand Macau’s Student Problem-Solving Behavior: An Example of a Released Item from PISA 2012 Study

問題解決能力為學生應對未來生活及工作挑戰的重要能力，然而，傳統紙筆測 驗並未能探討問題解決過程的學生行為及其認知和情意特點。電子化問題解決測驗 及其log 數據則使解決上述困難成為可能。教育數據探勘是研究log 數據等教育大 數據的新興學科，該學科建基於商業數據探勘，其研究模式及方法皆有別於一般教 育研究。有鑑於此，本研究首先闡述教育數據探勘的原理及技術，再使用PISA 2012之電子化問題解決能力測驗公開題的log數據進行探究。在研究方法上，本研 究以PISA 2012 問題解決表現最佳的十個經濟體為樣本，涉及三種典型的log 數 據，即解題時間、點擊次數，以及操作─回應路徑；進行預處理後，本研究使用分 類和異常值檢測，及模式探勘等手法分析上述數據。結果顯示，澳門學生的解題時 間及點擊次數均多於其他九個經濟體；研究樣本的操作─回應路徑則可分為六個問 題解決群組，分別為：「最佳作答」、「自檢查後正確作答」、「不作答」、「誤 解題意或不小心作答」、「嬉戲」，以及「其他」。其中，澳門在首兩個群組之總 比例少於其他九個經濟體，缺乏解題動機或經過自檢查後仍然出錯者也多於其他經 濟體。最後，研究者總結學生的問題解決特點，為教育工作者改善學生的問題解決 能力提供實證依據，並為未來的教育數據探勘研究提出建議。

Problem-solving competency is essential for students to tackle challenges in future life. However， the traditional paper-pencil problem-solving test cannot reveal student behavior， nor the cognitive and motivational processes in solving problems. Fortunately， computer-based assessment and log file data analysis may be the silver bullet. Educational data mining (EDM) is an emerging discipline that investigates large-scale educational data， such as the log file data. It is developed on the basis of data mining techniques widely employed in business domains. The analytic tools and research methods of EDM are different from those in general educational studies. Under this backdrop， this research aimed to elucidate the principles and research methods of EDM， explored and analyzed the log file data of PISA 2012 computerbased problem-solving assessment. Regarding research methods， this study looked into the top ten high-performing economies in the PISA 2012 problem-solving assessment， analyzing three typical kinds of log file data， i.e. item response time， the number of mouse clicks， and operation-response paths. After pre-processing， techniques of classification and outliner detection， and pattern mining were applied to analyze the log file data. Research results are summarized as below: Macau students’ item response time was obviously longer， and their number of mouse clicks was more， than those of the other high-performing economies. The operation- response paths further illustrated six typical log file response patterns， namely: “Perfect with High Effectiveness”， “Correct Answer with Self-Checking”， “No Response”， “Misunderstanding and Carelessness”， “Playing” and “Others”. Macau’s total proportions of the first two student groups were obviously smaller than those of the other high-performing economies， and Macau had more students who lacked motivation to solve problems， or answered incorrectly even after self-checking than the other economies. Finally， this research provided empirical evidences to enhance students’ problem-solving competency and gave recommendations for further studies.