Regularized fully convolutional networks for RGB-D semantic segmentation

The prospect of semantic segmentation using depth is alluring. In most of the previous work features are only combined by using simple classification strategies. The inter-feature and inter-label relationships have been ignored. This paper proposes a novel unified framework for RGB-D semantic segmentation. We use regularized fully convolutional networks whose inputs are depth map and hand-crafted features. Relationships between those features and their labels are learnt and utilized by rigorously imposing regularization in fully connected layers. The regularized fully convolutional networks can be efficiently launched using a GPU implementation at an affordable training cost. Experiments demonstrate that our regularized fully convolutional networks taking features as inputs obtain competitive results on the PASCAL VOC 2011 dataset and NYUDv2.