Surveillance plays a crucial role in safeguards. Reviewing surveillance data requires a significant number of inspection manpower. As the number of surveillance cameras increases, the demand for such manpower is expected to grow even more. Recently, in the field of security, there has been a development of deep learning models that automatically detect abnormal events from video images, and their usage is expanding. In this study, we used an AutoEncoder-based semi-supervised learning model, which can detect unexpected abnormal events, to detect anomalies in the UCSDped2 dataset and in simulating safeguards-related event videos taken at Dry Mockup facility of KAERI. To improve the model performance, we transformed the video images into two parts: the appearance part, which are sequences of video image frames, and the motion part, which are the pixel value differences of consecutive video frames. In addition, we added memory module to the bottle neck of the AutoEncoder model, and skip connection to enhance the model performance. To evaluate the model performance, we proposed a new evaluation index, which is adequate to the video images of safeguards surveillance in addition to the widely used AUC (Area Under the ROC Curve).