This study investigated the family size effect on English word processing via visual lexical decision task with three different groups of speakers, i.e., L1 English speakers, Korean L2 English learners, and English bilinguals. For English simple nouns, verbs, and adjectives, we examined the effects of the type count of morphologically related members and the surface base-frequency on lexical processing. First, results showed that the family size effect emerged in Korean L2 learners, but it was mostly inhibitory. To be specific, words with a large family size elicited slower response latencies than those with a small family size. However, the facilitatory effect arose for bilinguals and native speakers of English. Second, it was exhibited that high-frequency base words were recognized more quickly than low-frequency counterparts, confirming that token frequency as well as type frequency codetermines their recognition latency. These findings suggest that L1, bilinguals, and L2 learners' mental lexicons are organized by morphologically related words along with surface frequency although their effect size differs depending on the amount of language exposure. Finally, building on the results in the current experimental study, we propose a formal account for the processing advantages of words with a large family size under a psycholinguistic model and processing constraints.