測驗類型、因素數目、各因素題數及樣本數對六種信度估計法估計誤差之交互影響分析

Effects of Classical Test Theory Assumptions, Test Length, Number of Factors, and Sample Size on Six Reliability Estimation Techniques

本研究以三種測驗類型、兩種因素數目、四種各因素題數及四種樣本數為模擬資料之操弄變項，探討此四個因子對ρglb、λ2、λ3(α)、λ4、λ6、ωt等六種信度估計誤差之交互作用效果，以瞭解不同信度估計法估計誤差受測驗類型、因素數目、各因素題數及樣本數之交互影響情形。研究結果顯示：一、四個因子對各信度估計法之估計誤差均有交互作用效果，顯示不同測量情境組合需採用不同的信度估計方法。二、無論是哪種測量情境組合，λ4及ωt之估計誤差均極微，計算ωt時需正確判斷因素數目才能精確估計信度，而計算λ4時則否，建議當測驗資料之因素結構很明確時，以ωt估計信度，若因素結構不明確時以λ4估計信度。三、ρglb雖受到不少學者之青睞，但其估計誤差受樣本數影響很大，且當樣本數較小時也會因題數較多而更明顯高估信度，因此，ρglb僅適用於大樣本情境。四、將六種信度估計法估計誤差進行集群分析，可區分為「ωt、λ4、ρglb」與「λ2、λ3、λ6」兩大類，前者會高估信度，後者會低估信度，其中以「ωt與λ4」及「λ2與λ3」最為相似。本研究之分析結果可提供予測驗使用人員依不同測驗情境選擇較適切之信度估計法參考。

This study analyzed the accuracy of six techniques for estimating reliability. Simulated data that are designed to vary and interact with each other essentially comprise three types of classical test theory assumptions, two types of factors, four types of test length, and four types of sample sizes. The results are as follows: (1) The four factors of the simulated variables have an interactive effect on the reliability of error estimation, indicating that different measurement combinations must be coupled with different reliability estimation methods. (2) λ4 and ωt are two optimal reliability estimation methods with an extremely low error in this scenario. When the factor structure of the test data is clear, reliability should be estimated using ωt or λ4. (3) Substantial research attention has been paid to ρglb, and the estimation of the reliability error is considerably affected by the sample size. Moreover, high reliability can be achieved when the sample size is small as the test length increases; furthermore, it is only suitable for a large sample size. (4) The six arguments can be categorized into (ωt, λ4, ρglb) and (λ2, λ3, λ6) groups through cluster analysis. The (ωt, λ4, ρglb) group overestimates reliability, and the (λ2, λ3, λ6) group underestimates reliability. Moreover, (λ4, ωt) and (λ2, λ3) exhibit the highest similarity. This study provides testing persons with an appropriate estimate of reliability indicators according to different test scenarios.