Depth Map Upsampling via Multi-Modal Generative Adversarial Network
  Daniel Stanley Tan     Jun-Ming Lin     Yu-Chi Lai     Joel Ilao     Kai-Lung Hua  
Tan, D. S., Lin, J.-M., Lai, Y.-C., Ilao, J. and Hua, K.-L., "Depth Map Upsampling via Multi-Modal Generative Adversarial Network", Sensors, Volume 19, Issue 7, PP 1587, 2019, DOI:

Autonomous robots for smart homes and smart cities mostly require depth perception in order to interact with their environments. However, depth maps are usually captured in a lower resolution as compared to RGB color images due to the inherent limitations of the sensors. Naively increasing its resolution often leads to loss of sharpness and incorrect estimates, especially in the regions with depth discontinuities or depth boundaries. In this paper, we propose a novel Generative Adversarial Network (GAN)-based framework for depth map super-resolution that is able to preserve the smooth areas, as well as the sharp edges at the boundaries of the depth map. Our proposed model is trained on two different modalities, namely color images and depth maps. However, at test time, our model only requires the depth map in order to produce a higher resolution version. We evaluated our model both quantitatively and qualitatively, and our experiments show that our method performs better than existing state-of-the-art models.


author={Tan, D. S. and Lin, J.-M. and Lai, Y.-C. and Ilao, J. and Hua, K.-L.}, 
title={Depth Map Upsampling via Multi-Modal Generative Adversarial Network},