This paper proposes a deep learning-based underground object classification technique incorporated with phase analysis of ground penetrating radar (GPR) for enhancing the underground object classification capability. Deep convolutional neural network (CNN) using the combination of the B- and C-scan images has recently emerged for automated underground object classification. However, it often leads to misclassification because arbitrary underground objects may have similar signal features. To overcome the drawback, the combination of B- and C-scan images as well as phase information of GPR are simultaneously used for CNN in this study, enabling to have more distinguishable signal features among various underground objects. The proposed technique is validated using in-situ GPR data obtained from urban roads in Seoul, South Korea. The validation results show that the false alarm is significantly reduced compared to the CNN results using only B- and C-scan images.