독립형 VR기기가 활발히 개발되기 전 스마트폰을 HMD로 만들려는 움직임이 있었고 구글의 카드보드와 삼 성의 기어 VR이 그 예이다. 하지만 독립형 VR기기의 출시와 모바일 기기의 한계로 현재는 대부분 AR로 전 환되어 사용되었다. 그렇지만 지난 10년간 모바일 VR과 AR의 기술도 발달했지만, 그동안 모바일 기기의 성 능도 향상되었다. 따라서 발달한 기술들을 바탕으로 손 추적을 뛰어넘어 손동작 분류를 모바일 기기에서 기 존의 VR과 AR 콘텐츠와의 상호작용에 활용할 수 있다. 본 논문에서는 Manomotion을 이용하여 2D 또는 3D 의 골격을 추적하고 손동작 분류를 구현했으며 게임을 사용하여 검증하였다.

Before standalone mobile VR (Virtual Reality) were actively developed, there was a movement to make smartphones into HMD (Head-Mounted Display). Google's cardboard and Samsung's GearVR are examples. However, smartphone-based VR has shifted to AR (Augmented Reality) due to the release of standalone VR and the limitations of mobile devices. As examples of using the AR, we can wear accessories even without trying them on, and we can be also guided by arrows on the map when finding a place that we have never visited before. Over the past decade, mobile devices' performance has improved with the development of mobile VR and AR. As a result, the combination of hand gesture classification beyond hand tracking can be utilized for interaction with VR and AR content. In this paper, we describe how to use the hand gesture for the user interaction for the mobile VR and AR. When realizing the gesture classification, we used Manomotion to implement the gesture estimation and 2D/ 3D skeleton was tracked in real time. In this paper, we suggest the methods to classify the hand gesture and verify them through successfully recognizing the gesture and using them at a practical game. We also measure the performance, and the suggested algorithm requires only 0.425 msec to recognize the gesture. This interface can be also used to improve the convenience of users other than games for mobile VR and AR.

ABSTRACT
1. Introduction
2. Related Works
    2.1 MediaPipe
    2.2 Banuba
    2.3 Manomotion
3. Gesture Recognition
    3.1 Recognition of Gun-shaped Gesture
    3.2 Verification
    3.3 Performance
4. Conclusion
Acknowledgement
Reference
국문초록
결론 및 향후 연구

• Beom Jun JO(Department of Game Design and Development, Sangmyung University) | 조범준
• Seong Ki KIM(National Centre of Excellence in Software, Sangmyung University) | 김성기
• Seok-Kyoo KIM(Department of Game Design and Development, Sangmyung University) | 김석규 Corresponding author