This paper investigates the problem of ship course control in the presence of model uncertainties, external disturbances, and actuator saturation. A high-performance autopilot is developed based on a direct neural network adaptive dynamic surface control (DSC) framework integrated with deep reinforcement learning. To compensate for lumped uncertainties arising from unmodeled dynamics and disturbances, a radial basis function (RBF) neural network is employed to provide online approximation within the control design. Moreover, the actuator saturation constraint is explicitly incorporated into the controller, avoiding performance degradation commonly encountered in conventional DSC schemes.To alleviate the reliance on manual parameter tuning, the controller parameter adaptation is formulated as a continuous-action optimization problem and solved using a deep deterministic policy gradient (DDPG) algorithm. The DDPG agent learns an optimal tuning policy by maximizing a reward function that penalizes course tracking errors, excessive control variations, and energy consumption. Simulation results demonstrate that the proposed method achieves improved tracking accuracy, smoother control inputs, and enhanced robustness under complex operating conditions, thereby validating the effectiveness of the DDPG-based adaptive tuning strategy for autonomous ship navigation.

Abstract
1. Introduction
2. Problem Analysis
    2.1. Mathematical Model of the Ship Heading Control System
    2.2 Design of a Nonlinear Time-Varying Disturbance Observer
    2.3. Asymmetric Saturation and Compensation byan Auxiliary Dynamic System
    2.4. Phase-by-Phase Error Constraints
3. Controller Design and Stability Analysis
    3.1. Controller Design
    3.2. Nonlinear Time-Varying Disturbance ObserverDesign
    3.3. Stability Analysis
4. Simulation Verification
5. Conclusion
References

• LI Xinyi(School of Navigation and Shipping, Shandong Jiaotong University, China) Corresponding author
• LI Weihong(School of Navigation and Shipping, Shandong Jiaotong University, China)
• TAO Yuwen(School of Navigation and Shipping, Shandong Jiaotong University, China)