PURPOSES : This study uses deep learning image classification models and vehicle-mounted cameras to detect types of pavement distress — such as potholes, spalling, punch-outs, and patching damage — which require urgent maintenance.
METHODS : For the automatic detection of pavement distress, the optimal mount location on a vehicle for a regular action camera was first determined. Using the orthogonal projection of obliquely captured surface images, morphological operations, and multi-blob image processing, candidate distressed pavement images were extracted from road surface images of a 16,036 km in-lane distance. Next, the distressed pavement images classified by experts were trained and tested for evaluation by three deep learning convolutional neural network (CNN) models: GoogLeNet, AlexNet, and VGGNet. The CNN models were image classification tools used to identify and extract the combined features of the target images via deep layers. Here, a data augmentation technique was applied to produce big distress data for training. Third, the dimensions of the detected distressed pavement patches were computed to estimate the quantity of repair materials needed.
RESULTS : It was found that installing cameras 1.8 m above the ground on the exterior rear of the vehicle could provide clear pavement surface images with a resolution of 1 cm per pixel. The sensitivity analysis results of the trained GoogLeNet, AlexNet, and VGGNet models were 93 %, 86 %, and 72 %, respectively, compared to 62.7 % for the dimensional computation. Following readjustment of the image categories in the GoogLeNet model, distress detection sensitivity increased to 94.6 %.
CONCLUSIONS : These findings support urgent maintenance by sending the detected distressed pavement images with the dimensions of the distressed patches and GPS coordinates to local maintenance offices in real-time.

ABSTRACT
1. Introductuon
2. Literature Review
3. Data Acquisition and Image AnalysisMethods
    3.1. Pavement Surface Image Collection
    3.2. Optimal Location of Vehicle Mounted Camera
    3.3. Image Processing to Extract DistressedPavement Candidate Images
    3.4. Deep Learning Based Image ClassificationManual Distress Image Classification by Experts, and DataAugmentation.
4. DISCUSSION OF RESULTS
    4.1. Performance Results of Deep LearningImage Classification Models
    4.2. Results of Improvement to Model Performanceby Readjusting Image Categories
    4.3. Performance Results of Distress DimensionComputation
5. CONCLUSIONS
REFERENCES

• Lee Jong Sub(Korea Expressway Corporation Research Institute) | 이종섭
• Kim Jong Ho(Department of Transportation and Logistics Engineering, Hanyang Univ.) | 김종호
• Kim Jang Rak(Department of Transportation and Logistics Engineering, Hanyang Univ.) | 김장락 Corresponding Author