This study investigated the thematic and lexical characteristics of high-difficulty English reading items—commonly referred to as killer questions—in the Korean College Scholastic Ability Test (CSAT) between 2018 and 2025. Using text mining methods, including Latent Dirichlet Allocation (LDA) and CEFR-based lexical profiling, the analysis reveals that these items disproportionately addressed abstract, academic topics such as cultural heritage, moral dilemmas, and cognitive science. Lexical analysis further indicated that over 30% of the words used are at the C1 level or above, reflecting substantial vocabulary demands. These findings point to a misalignment with the national curriculum, which is designed to target B1–B2 proficiency with more familiar and accessible topics. The combined challenge of abstract themes and advanced vocabulary raises concerns about the construct validity and fairness of these items, suggesting that they may measure background knowledge or test-taking strategies rather than genuine English reading proficiency. The study concludes by calling for evidence-based item development and closer alignment with curricular goals to ensure valid and equitable high-stakes language assessment.

This study examines the difficulty of reading text in elementary school English textbooks. Four elementary school English textbook series published by three publishers were evaluated by Word Critical Factor (WCF). WCF considers cognitive demands for word recognition; it assesses the match of linguistic content in the text with the phonetically regular and high-frequency words that are associated with particular stages of reading development. For the analysis, all of the words that appeared in the reading and writing sections from four elementary school English textbooks were analyzed by two criteria, that is, the ratio of high frequency words and phonetically regular words among 100 running words, and the number of unique words in each textbook. The results showed that all four textbooks’ difficulty levels were very high considering the learner’s reading ability. This was due to the textbooks having a low repetition of words and also the complicated vowel patterns that were above the reading abilities of students. All of these factors combined are what have caused the difficulties presented in English textbooks. In conclusion, although the reading text introduces a variety of genres and activities for the development of reading skills, most reading text could not assist the reader’s cognitive processing.