With the development of augmented reality technology, various related technologies and content production are rapidly increasing. The virtual space in the augmented reality is achieved through the synthesis of the virtual environment composed of 3D models with the background of the real world video. The harmonization of these two environments is an important factor for users enjoying augmented reality contents. In this paper, we introduce a video abstraction technique to solve the problem of mismatch between input video and 3D virtual objects, which is one of the points to be improved in augmented reality. In particular, we created a new virtual space by applying video-based cartoon and pencil stylization to transform the AR background.