In light of the expanding use of technology in education, we attempted to analyze how Korean college students perceived the use of Machine Translation (MT) tools in the classroom. Specifically, this study attempted to explore students’ perceptions of their ability to use MT tools and to measure the reliability of the MT-generated output, along with measuring students’ general sense of confidence in English learning. This research analyzed 183 EFL college students’ responses to an online survey, and a one-way ANOVA was used to test for the differences in the averages of three groups. The results of data analysis revealed that 1) Among beginners, intermediate learners, and advanced learners, those self-identifying as advanced had the highest scores on all the factors measured.; 2) There was a significant mean difference in students’ perceptions of the ability to use MT tools, their beliefs regarding MT’s effectiveness as a learning tool, and affective attitudes towards the use of MT tools between beginner and advanced groups. Based on the findings, pedagogical implications for the effective use of MT tools in the Korean EFL classrooms, and suggestions for future research were presented.