Efficient and safe maritime navigation in complex and congested coastal regions requires advanced route optimization methods that surpass the limitations of traditional shortest-path algorithms. This study applies Deep Q-Network (DQN) and Proximal Policy Optimization (PPO) reinforcement learning (RL) algorithms to generate and refine optimal ship routes in East Asian waters, focusing on passages from Shanghai to Busan and Ulsan to Daesan. Operating within a grid-based representation of the marine environment and considering constraints such as restricted areas and Traffic Separation Schemes (TSS), both DQN and PPO learn policies prioritizing safety and operational efficiency. Comparative analyses with actual vessel routes demonstrate that RL-based methods yield shorter and safer paths. Among these methods, PPO outperforms DQN, providing more stable and coherent routes. Post-processing with the Douglas-Peucker (DP) algorithm further simplifies the paths for practical navigational use. The findings underscore the potential of RL in enhancing navigational safety, reducing travel distance, and advancing autonomous ship navigation technologies.

Abstract
1. Introduction
2. Related Work
    2.1. Reinforcement learning
    2.2. Electronic Navigational Chart
    2.3. Route Optimization
    2.4. Douglas-Peucker Algorithm
3. Methodology
    3.1. Environment Setup and Grid System
    3.2. Data and Routes
    3.3. Reward Functions and Hyperparameters
    3.4. Post-Processing (Douglas-Peucker Algorithm)
4. Results and Discussion
    4.1. Deep Q-Network (DQN) Results
    4.2. Proximal Policy Optimization (PPO) Results
    4.3. Comparative Analysis of Results
5. Conclusion
Acknowledgments
References

• Jisoo Kim(R&D Center, mapsea Corp., Seoul, Korea)
• Byeonggong Hwang(R&D Center, mapsea Corp., Seoul, Korea)
• Gi-Hyun Kim(R&D Center, mapsea Corp., Seoul, Korea)
• Ung-Gyu Kim(R&D Center, mapsea Corp., Seoul, Korea)