It is important to ensure worker’s safety from radiation hazard in decommissioning site. Real-time tracking of worker’s location is one of the factors necessary to detect radiation hazard in advance. In this study, the integrated algorithm for worker tracking has been developed to ensure the safety of workers. There are three essential techniques needed to track worker’s location, which are object detection, object tracking, and estimating location (stereo vision). Above all, object detection performance is most important factor in this study because the performance of tracking and estimating location is depended on worker detection level. YOLO (You Only Look Once version 5) model capable of real-time object detection was applied for worker detection. Among the various YOLO models, a model specialized for person detection was considered to maximize performance. This model showed good performance for distinguishing and detecting workers in various occlusion situations that are difficult to detect correctly. Deep SORT (Simple Online and Realtime Tracking) algorithm which uses deep learning technique has been considered for object tracking. Deep SORT is an algorithm that supplements the existing SORT method by utilizing the appearance information based on deep learning. It showed good tracking performance in the various occlusion situations. The last step is to estimate worker’s location (x-y-z coordinates). The stereo vision technique has been considered to estimate location. It predicts xyz location using two images obtained from stereo camera like human eyes. Two images are obtained from stereo camera and these images are rectified based on camera calibration information in the integrated algorithm. And then workers are detected from the two rectified images and the Deep SORT tracks workers based on worker’s position and appearance between previous frames and current frames. Two points of workers having same ID in two rectified images give xzy information by calculating depth estimation of stereo vision. The integrated algorithm developed in this study showed sufficient possibility to track workers in real time. It also showed fast speed to enable real-time application, showing about 0.08 sec per two frames to detect workers on a laptop with high-performance GPU (RTX 3080 laptop version). Therefore, it is expected that this algorithm can be sufficiently used to track workers in real decommissioning site by performing additional parameter optimization.