Work, leisure, and subjective well-being are inextricably linked. Both work and leisure affect subjective well-being to varying degrees. Thus, seeking a balance between work and leisure to improve the quality of life has always been a social issue that scholars are concerned about. In the 21st century, the development of science and technology has not shortened working hours and gave rise to the “leisure society” expected by scholars. Moreover, the increase in material wealth has not brought about an increase in happiness in some developed countries and emerging economies. These situations make people start to reexamine the relationship between working time, leisure time, and subjective well-being. The purpose of this research is to explore whether overtime work has an impact on subjective well-being and to verify whether leisure activity time can moderate the effect of overtime work on subjective well-being. Cross-sectional data from the 2017 Chinese General Social Survey were utilized and twelve types of physical activity were included in statistical testing. The results found a negative linear relationship between overtime work and subjective well-being. Reading activities in leisure time moderated the negative impact of overtime work on subjective well-being to a certain extent. Especially, as the working time became longer, the more frequently female workers participated in reading, the stronger their subjective well-being would be. The results have implications for reducing the negative impact of overtime work on physical and mental health and improving the quality of life of Chinese workers.