Determining the number of operators who set up the machines in a human-machine system is crucial for maximizing the benefits of automated production machines. A man-machine chart is an effective tool for identifying bottlenecks, improving process efficiency, and determining the optimal number of machines per operator. However, traditional man-machine charts are lacking in accounting for idle times, such as interruptions caused by other material handling equipment. We present an adjusted man-machine chart that determines the number of machines per operator, incorporating idleness as a penalty term. The adjusted man-machine chart efficiently deploys and schedules operators for the hole machining process to enhance productivity, where operators have various idle times, such as break times and waiting times by forklifts or trailers. Further, we conduct a simulation validation of traditional and proposed charts under various operational environments of operators’ fixed and flexible break times. The simulation results indicate that the adjusted man-machine chart is better suited for real-world work environments and significantly improves productivity.