題組效果對參數估計精確度之影響   全文下載

The Influence of Testlet Effect on the Accuracy of Parameter Estimation

本研究旨在探討題組效果對參數回復性(item parameter recovery)的影響，以模擬資料探討傳統IRT模式、TRT計分模式及bi-factor模式等三種模式在不同題組(testlet)條件下，對受試者能力以及試題參數估計回復性的比較，同時也探討Yen(1984)的Q3統計數對於試題局部依賴性(local item dependence)的偵測。每一個測驗皆包含6個題組，每個題組皆為5題，而三個題組效果（0、0.6、1.2）以及兩種樣本人數（500人、1000人）被操弄來產生模擬的試題反應向量，每種條件組合重複100次，所產生的試題反應資料分別以上述的三種校準估計的模式進行分析，並計算Q3值。本研究主要發現如下：（1）在各種模擬條件下，以題組模式(testlet-based model)的試題參數回復性表現良好；以TRT模式的估計最佳，其次為bi-factor模式估計，再者為傳統IRT模式。（2）人數越大，參數估計的回復性也較好。隨機題組斜率影響越大，估計造成的RMSE也越大。（3）假定有相依關係的題組內試題配對的Q3值明顯大於假定是獨立的題組間的試題配對所得到的Q3值，顯示對於偵測存於試題之間的相依情形，Q3的表現不錯。

This study investigated the influence of testlet effect on the recovery of item and person parameters. Simulated data was used to compare the parameter recovery given by the traditional item response theory (IRT) model, the testlet response theory (TRT) model and the bi-factor model. The Q3 index was computed for all the data sets to serve as a tool for detecting the local item dependency. There are six testlets in each test with 5 items for each testlet. Three testlet slope (0.0, 0.6, & 1.2), and two sample sizes (500 & 1000 examinees) were manipulated for simulating the item response vectors. For each combination of the conditions, item response data was simulated 100 times. The simulated item response data was calibrated by using the three models described above separately, and Q3 was calculated for each data set. The main findings are as the following: (1) The accuracy of item parameter recovery was higher for the testlet-based models than that of traditional item-based model (i.e., 3PL model). Among the three models used, the TRT model performs best, followed by the bi-factor model and then the IRT model; (2) The greater sample size is, the better accuracy of item parameter recovery is gained. Similarly, the greater testlet slope is, the greater estimation error is found; (3) The Q3 indexes calculated for the paired items from the same testlet were found larger than those for the item pairs from different testlets. This indicates that the Q3 index performs well in terms of detecting the local dependence between items.