Stone pagoda structures have continued to be aged due to the combination of various damage factors. However, some studies on nonstructural damage have been carried out, but assessment studies on structural damage have not been done in various ways. Therefore, in this study, structural and nonstructural influencing factors according to the damage types are classified and the damage assessment according to the structural influencing factors affecting the behavior of the stone pagoda structure is performed. In addition, the damage rating classification criteria for each type of structural damages or damage locations are presented, and the damage index is calculated by providing the criteria for the classification of damage according to the degree of damage to which the damage is caused. Therefore, this study can evaluate quantitatively the damage status of stone pagoda structures.