生成式AI於STEM跨領域流程圖評量之評分一致性研究

A Study of Scoring Agreement of Generative Artificial Intelligence in the Assessment of STEM Interdisciplinary Flowcharts

隨著大型語言模型具備圖像理解與語意推理能力，生成式AI是否能協助教師對學生圖像化作品進行穩定評分，已成為教育評量研究之重要議題。本研究旨在探討生成式AI於STEM跨領域問題解決流程圖評量中的評分實務可行性。本研究以81份國中生STEM問題解決流程圖作品為研究樣本，比較生成式AI與人工評分之間的一致程度。研究結果顯示，生成式AI在結構性與進階認知面向中展現一定程度之評分一致性與實務應用潛力，在高階認知面向之概念整合與跨域推理辨識上仍存在侷限。這些發現表明，在明確評分規準與結構化流程支持下，生成式AI可作為輔助評量之可行工具。本研究成果為未來STEM跨領域問題解決能力評量設計與生成式AI自動化評分應用之參考基礎。

As large language models have increasingly demonstrated multimodal comprehension and semantic reasoning capabilities, the question of whether generative AI can reliably support teachers in scoring students' visual artifacts has become an important issue in educational assessment research. This study aimed to examine the scoring agreement between generative artificial intelligence (AI) and human raters in the assessment of STEM interdisciplinary flowcharts. Using 81 STEM interdisciplinary flowcharts produced by junior high school students as the research sample, this study compared the level of agreement between generative AI and human scoring. The results indicated that generative AI achieved moderate scoring agreement in structural and intermediate cognitive dimensions. However, lower agreement was observed in the higher-order cognitive dimension involving conceptual integration and cross-disciplinary reasoning. These findings suggest that, under clearly defined scoring rubrics and structured procedural criteria, generative AI may serve as a supplementary assessment tool rather than a full replacement for human judgment. The outcomes of this study provide empirical evidence for future STEM interdisciplinary assessment design and AI-assisted scoring applications.