以「補充性表現水平描述輔助自陳式測量構念」之延伸Angoff標準設定研究

Extended Angoff Method in Setting Standards for Self-Report Measures With Supplementary Performance-Level Descriptors

隨著十二年國民基本教育課程推動，其核心重視學生知識、態度、技能與策略等全人培養。為回應這波課程革新對於學生表現影響，遂成立臺灣學生成就長期追蹤評量計畫（TASAL），目的在描述臺灣學生表現及探究影響因子，其本質為標準本位評量。然而，在回顧過去標準設定研究，多以學科認知範疇的標準設定為焦點，較少涉及情意、策略面向。據此，本研究提出補充性表現水平描述（S-PLD）概念，以輔助專家教師使用延伸Angoff（extended Angoff）標準設定法，進行自陳式測量構念通過分數設定，並透過檢視效度之過程、內部與外部證據，以支持本研究結果合理性。本研究結果顯示標準設定成員對於標準設定過程，多認同其適切性。藉由討論與反思，凝聚出可接受、適當結果，而在檢視成員於各輪次評定結果的穩定與一致性，其誤差也多能在合理範圍內。最後，本研究所設定兩個切截點，也具有一定區別不同層級策略使用者於外在效標（英語文理解）的表現。整體而言，本研究結果是具有過程、內部與外部證據支持，並於文末，提出幾點建議，供未來研究者參考。

"One vision of the 12-Year Basic Education Curriculum in Taiwan is to promote the comprehensive learning and development of all students. To ensure the quality of this curriculum reform, the Ministry of Education funded a long-term project, the Taiwan Assessment of Student Achievement: Longitudinal Study (TASAL), to evaluate the impact of the curriculum on student performance. The TASAL is a large-scale standards-based assessment, and standard setting is one of its main task. A comprehensive literature review indicated that most empirical studies related to standard setting have focused on cognitive domains and few study undertake expert-oriented standard-setting processes in affective domains because of some practical limitations. The present study suggests a new approach, employing supplementary performance-level descriptors (S-PLDs) in an extended Angoff method in setting standards for self-report measures. The purpose of this study was to uncover evidence of the procedural, internal, and external validity of implementing an extended Angoff method procedure with S-PLDs in standard setting for English comprehension strategy use among seventh grade students in Taiwan. PLDs are designed to outline the knowledge, skills, and practices that indicate the level of student performance in a target domain. In the present study, the use of comprehension strategies for learning English as a foreign language was examined. S-PLDs provide comparable but unique functions within the standard-setting process. S-PLDs offer supplementary material to subject matter experts to facilitate the formation of profiles of student performance in target domains, especially when ambiguities in conventional PLDs may prevent expert consensus during the standard-setting process."