Fixed parameters in metaheuristics like Differential evolution (DE) limit truss optimization efficiency. This study proposes a PS-QDE(Phased strategy Q-Learning DE) algorithm that uses Q-learning to dynamically adapt parameters. A novel "Strategy switching factor" is also introduced to adjust the exploration-exploitation balance based on convergence. The PS-QDE algorithm was validated on four truss optimization problems (10-bar to 200-bar) with frequency constraints. Results show PS-QDE provides more stable convergence and superior or competitive optimal solutions compared to standard DE.