This study examined a timed cloze test for evaluating English proficiency in second language (L2) experimental research. Forty-five Korean college students were randomly assigned to either a timed or untimed condition. In the timed condition, the participants read the sentences of the text one phrase at a time, using the self-paced, cumulative, moving-window reading paradigm, and their reading time (RT) and accuracy were measured. In an untimed condition, the participants carried out a typical pencil-and-paper cloze test. Linear mixed-effect models were used to analyze the data. Although the accuracy data did not indicate any significant results, the RT data showed that the participants responded faster when they selected accurate answers and they answered function words rather than content words; also, as the participants’ TOEIC scores increased, a marginally significant RT difference was observed. A significant correlation was also found between the cloze test and high TOEIC scores in the timed condition. The results showed that the timed cloze test used in the study can provide useful data for L2 experimental research in measuring L2 proficiency.