The purpose of this study is to develop and implement a customized AI-based speaking diagnosis, learning, and assessment system, SpeakMaster, in order to overcome the lack of systematic evaluation and practice opportunities in school English speaking class. This system integrates automated speaking scoring to provide students with feedback on their speaking abilities across pronunciation, conversation, and presentation. This study adopts a design-based research methodology, demonstrating the development and implementation process. 1,451 students and eight teachers in elementary, middle, and high schools participated in the experiment. Data were collected through learning logs, teacher journals, interviews, and post-surveys. The findings indicate that the system design is appropriate for English class, promoting students’ flow in engaging speaking practice. Students showed motivation and satisfaction while teachers found the system valuable for monitoring student progress and facilitating speaking assessments. Despite the challenges of improving chatbot performance and enhancing scoring reliability, the results suggest that SpeakMaster shows potential to enhance English speaking education.