四個試題反應模式的整體能力與領域能力估計精確性之比較研究

Comparing the Accuracy of Examinees' Overall and Domain Ability Estimates of Four IRT Models

本研究旨在比較廣義次向度模式（Brandt, 2012）、多向度隨機係數多項logit模式（MRCMLM；Adams, Wilson, & Wang, 1997）及高階試題反應模式（HO-IRT；de la Torre & Song, 2009）在各種模擬條件下對考生在測驗上的表現所估計得到的整體能力值及領域能力值估計的精確性。模擬資料時，領域向度數（3、5個）、各領域的題數（20、40題）以及各領域能力之間的相關（0.2、0.5及0.8）等三個變項被操弄。在12個條件的組合之下，每一個皆重複模擬30次，每次模擬1000個考生的作答反應。這些資料以MCMC法分別以上述三個模式來估計考生的整體能力以及各領域的能力。以r_(ζζ)、RMSE及BIAS來評估各試題反應模式的整體能力及領域能力估計的精確性。同時也以98年第一次國中基本學力測驗五個科目1000名考生的作答反應資料進行整體能力及領域能力的估計，並比較估計能力參數差異情形。研究發現如下：（1）模擬研究結果顯示，當領域向度數增加、各領域能力之間的相關係數變大，各領域內題數增加時，三個試題反應模式的整體能力和領域能力估計精確性均相當不錯。同時，領域能力估計值之間的相關亦均高於模擬時所設定的真值。（2）以貝氏估計的DIC指標作為評估三種試題反應模式對98年第一次國中基本學力測驗資料的適配情形，結果發現廣義次向度模式的適配情形最好，其次為MRCMLM模式和HO-IRT模式。（3）利用廣義次向度、MRCMLM及HO-IRT等三種模式來估計98年第一次國中基本學力測驗的資料，發現三種模式所估計得到的領域能力之間的關介於.86與.92之間，屬於高度相關；三個模式的相關平均數分別為0.88、0.90、0.90，而根據DETECT分析的結果，推測98年第一次國中基本測驗的資料應接近本質上的單一向度。

The purpose of this study is to compare the accuracy of overall and domain ability estimates given by four multidimensional item response models, including Generalized Sub-dimensional Model (GSM; Brandt, 2012), MRCMLM (Adams, Wilson, & Wang, 1997), and Higher-Order IRT Model (de la Torre, & Song, 2009). The numbers of domain abilities, and numbers of items in each domain as well as the size of correlation coefficients were manipulated, and under each combination of conditions, 1,000 examinees' response vector were generatedto be a multidimensional data set with simple structure. Ability and item parameters were calibrated by using the MCMC algorithm. Addtionally, the empirical data given by the 1000 examinees' responses on the 2009 Basic Competence Test (BCTEST) were also analyzed. The results are as the following. 1. In the simulation study, it was found that the accuracy of estimation of overall ability will be better when the numbers of domain abilities are larger, the correlation among the domain traits are larger, or there are more items in each domain. In addition, the correlation coefficients between estimated domain abilities are higher than those between the true abilities. 2. The values of the DIC index indicated that the GSM model fit the BCTEST data best. 3. The average correlation coefficients between the domain abilities were 0.88, 0.90 and 0.90 for the GSM, MRCMLM and HO-IRT, respectively. This result suggested that the BCTEST data is nearly essentially unidimensional.