In each July, high school seniors in Taiwan sit for the Advanced Subjects Test (AST), of which an English subtest (AST-ES) is an integral part. Their scores on the AST-ES affect significantly which universities they are qualified to be admitted to. A question of their great concern is: what English ability does the AST-ES truly measure? This study was aimed to address this question through studying the cloze items of the AST-ES. This study adopted a simpler version of Purpura’s (2004) model in which four components of grammatical knowledge were used to classify the AST-ES cloze items. Classification was performed by five experienced tertiary- level English teachers. Then a series of confirmatory factor analyses (CFAs) were applied to determine the degree of fit between their classifications and test-takers’ responses. The findings showed that their classifications did not fit the test-takers’ responses and that a one-component model best described the data. The finding of a one-component model was consistent with the unitary competence hypothesis of Oller (1976, 1979), who advocated using cloze tests for measuring overall language ability. This study concluded with some implications and limitations of the findings.