自然科成就測驗試題難度成份分析

The Cognitive Components of Item Difficulty of a Science Achievement Test

本研究使用認知成份分析的觀點，利用線性羅吉斯潛在特質模式（LLTM）和類別迴歸分析兩種方法來探討試題的難度成份與由Rasch模式所估計得到的難度參數之間的關係，並比較由這兩個分析方法所得到試題難度有效的預測變項（難度成份）之異同。研究所用自然科試題取自於國內某一個成就測驗題庫建置計畫中五年級的部分，一共有176個試題，由三個年度的資料所組成，總共有4918名受試者作答。研究中所指稱之難度成份包括：（1）文本特徵（例如：圖表的有無…等）；（2）認知特徵，亦即包含概念理解、科學探究和實務推理三個層次的認知水準；（3）知識特徵，亦即試題所屬的內容領域。研究結果主要如下：一、類別迴歸分析與LLTM分析所得之預測變項不盡相同，類別迴歸中僅有知識特徵，而LLTM中則是所有難度成份都有顯著。二、類別迴歸分析與LLTM分析的預測難度相關在-.645～-.713之間，兩者預測難度與Rasch難度的相關分別為.370～.471與-.544～-.618之間。

The purpose of this research was to study the relationship between the cognitive components of item difficulty and the item difficulty parameter estimated by using the Rasch model based on the view of cognitive component analysis. The Latent logistic latent trait model (LLTM) and categorical regression were used to find out the efficient predictors (cognitive components) and the results given by the two methods were compared. The empirical data from the science assessment of an item pool was used. The sample consisted of 4,918 fifth graders from the Southern Taiwan and 176 items from the 2005, 2006 and 2008 administrations.The sources of processing difficulty, identified using cognitive component analysis, include (1) text attribute (e.g., presence of a figure, etc.); (2) cognitive level, included conceptual understanding, scientific investigation, and practical reasoning; (3) knowledge characteristics, i.e., content fields of items. The relationship between Rasch difficulty parameter and different sources of processing difficulty was investigated by using the categorical regression and the logistic latent trait model (LLTM).The main findings of this study were as follows:1. There are differences between the cognitive components identified by the categorical regression analysis and the LLTM. In the categorical regression analysis, item content is the only significant component, but in the LLTM analysis, all cognitive components are significant.2. The correlation between the difficulties predicted by the categorical regression analysis and the LLTM is between -.645 and -.713. The correlation between the difficulties predicted by the categorical regression analysis and the Rasch difficulty parameter is between .370 and .471, and the correlation between the difficulty predicted by the LLTM and the Rasch difficulty is between -.544 and -.618.