華語文寫作測驗信度與評分規準適切性研究──以流利精通級摘要寫作題型為例

The Reliability and Rubric Relevance of the TOCFL Writing Test–Analyzing Band C Summary Writing as an Example

寫作測驗因題數少且須仰賴人工評分，使其信度研究更形重要，然而過往華語寫作測驗信度相關文獻甚為少見。緣此，本研究針對流利精通級華語文寫作測驗摘要寫作題型的信度與評分規準的適切性進行探討，運用的分析模式包括斯皮爾曼等級相關分析、類推性理論與多元迴歸分析。研究結果顯示：（1）在採取分析式評分方式的情況下，多數評分者所評定出的整體級分與最後成績呈中度或高度正相關，評分者間信度大致良好。（2）受試者的變異成分最高，可佐證其得分能反映寫作能力。（3）文本由3人評閱，有利兼顧評分品質與經濟效益。（4）在11個變項中，複雜句型、組織、詞彙語法、詞語運用、規範性為預測力較為顯著的變項。 A writing test is a form of language test that has fewer items in each test and relies heavily on subjective rating. These features of the writing test make the issue of reliability rather significant. This research focuses on the task type of Summary Writing used in the TOCFL Writing Test, Band C. By utilizing the analysis methods associated with Spearman's rank correlation coefficient, multiple regression analysis and generalizability theory, this study aims to discuss the reliability of, and the rubrics associated with, this specific task type. The results of this study can be summarized as follows: (1) By adopting an analytic rating method, the individual scores given by the majority of the raters have high-positive or medium-positive correlations with the final score. Inter-rater reliability is moderately high. (2) The variance components of the test-takers are the highest, which indicates that the scores for this task type can clearly reflect individual test-taker's writing ability. (3) Assigning three raters to rate each test essay can assure the quality and efficiency of the rating process. (4) Among the 11 variables which we have analyzed, 5 of them - complex sentence patterns, organization, vocabulary and syntax, vocabulary switching, and response requirements - can predict test-takers' writing ability more significantly.