한국재료학회지 제28권 제11호 (p.230-236)

SPAD과 CNN의 특성을 반영한 ToF 센서와 스테레오 카메라 융합 시스템

Fusion System of Time-of-Flight Sensor and Stereo Cameras Considering Single Photon Avalanche Diode and Convolutional Neural Network
키워드 :
Sensor Fusion,Time-of-Flight,Stereo camera,Single Photon Avalanche Diodes,Convolution Neural Network

목차

Abstract
1. 서 론
2. CNN을 사용한 ToF 해상도 증대
3. ToF 센서 - 스테레오 카메라 융합
4. 실 험
  4.1 CNN을 사용한 ToF 해상도 증대
  4.2 ToF 센서 - 스테레오 카메라 융합
5. 결 론
References

초록

3D depth perception has played an important role in robotics, and many sensory methods have also proposed for it. As a photodetector for 3D sensing, single photon avalanche diode (SPAD) is suggested due to sensitivity and accuracy. We have researched for applying a SPAD chip in our fusion system of time-of-fight (ToF) sensor and stereo camera. Our goal is to upsample of SPAD resolution using RGB stereo camera. Currently, we have 64 x 32 resolution SPAD ToF Sensor, even though there are higher resolution depth sensors such as Kinect V2 and Cube-Eye. This may be a weak point of our system, however we exploit this gap using a transition of idea. A convolution neural network (CNN) is designed to upsample our low resolution depth map using the data of the higher resolution depth as label data. Then, the upsampled depth data using CNN and stereo camera depth data are fused using semi-global matching (SGM) algorithm. We proposed simplified fusion method created for the embedded system.