AI-driven automation for structural design has been actively studied in structural engineering. In particular, reinforcement learning (RL) has attracted attention as a framework in which an agent interacts with an environment to autonomously search for optimal design solutions in complex design spaces. This study proposes an automated design model for rectangular reinforced-concrete (RC) columns based on a multi-agent Double Deep Q-Network (Double DQN). Extending prior RL-based automation developed for RC beam design to column members, the proposed environment explicitly incorporates key column-specific behaviors, including axial force–bending moment (P–M) interaction and moment magnification due to column buckling. Four agents independently determine the section width (b), section depth (h), number of longitudinal bars (n), and bar size. The reward function combines (i) penalty terms for violations of ACI 318-19 design constraints and (ii) an economic reward defined relative to an approximate optimal cost predicted by a quadratic regression model. After training for approximately 10,000 episodes, the proposed multi-agent Double DQN consistently generated ACI-compliant column designs across all test load cases and produced solutions with improved cost efficiency compared with the approximate optimal baseline. These results demonstrate the feasibility and practical potential of multi-agent RL for automated RC column section design.