Source localization technique using acoustic emission (AE) has been widely used to track the accurate location of the damaged structure. The principle of localization is based on signal velocity and the time difference of arrival (TDOF) obtained from different signals for the specific source. However, signal velocity changes depending on the frequency domain of signals. In addition, the TDOF is dependent on the signal threshold which affects the prediction accuracy. In this study, a convolutional neural network (CNN)-based approach is used to overcome the existing problem. The concrete block corresponding to 1.3×1.3×1.3 m size is prepared according to the mixing ratio of Wolseong low-to-intermediate level radioactive waste disposal concrete materials. The source is excited using an impact hammer, and signals were acquired through eight AE sensors attached to the concrete block and a multi-channel AE measurement system. The different signals for a specific source are time-synchronized to obtain TDOF information and are transformed into a time-frequency domain using continuous wavelet transform (CWT) for consideration of various frequencies. The developed CNN model is compared with the conventional TDOF-based method using the testing dataset. The result suggests that the CNN-based method can contribute to the improvement of localization performance.