Last few years, many researches on deep learning-based crack detection model have been reported in order to develop an efficient structure inspection method. While developing crack detection deep learning model, many research results reported the importance of the training data. Since most of the research results only qualitatively discussed the importance of training data, this study examine the influence of the training data by experiment, especially in the case of negative samples such as construction joint, spider web and concrete blocks.