Investigating Topic Familiarity as Source of Testlet Effect in Reading Tests: Bifactor Analysis
pp.79-109
The current study aims to examine whether topic familiarity can be associated with, and can also substantially contribute to, the testlet effect often found among items sharing a common passage in reading comprehension tests, particularly by using the Multidimensional Item Response Theory (MIRT) bifactor model as the main data analytic scheme. Data analyzed in this study were item scores collected from 830 Korean high school students who took a 8-passage, 32-item reading comprehension test (4 items per passage) and a 40-item self-reported topic familiarity questionnaire (5 items per passage). The results of analysis revealed that (a) the bifactor model best fit the test data; (b) the self-reported topic familiarity score was not correlated with the testlet factor scores at a statistically significant level; and (c) the overall topic familiarity score had a positive correlation with the general reading ability score, however. Implications of these major findings are discussed in terms of the usefulness of the bifactor model and the nature of testlet effects in reading tests, along with avenues for further investigation.