Structure guided image restoration : a deep learning approach

Abstract

Image restoration aims at recovery of degraded images and estimating the original. Over the past few years, computer vision research has been dominated by deep learning techniques in part due to advances in computing infrastructure, algorithms and image capturing devices. As a result, deep neural networks currently set the state-of-the-art in image restoration problems. However, many of these techniques fail to reconstruct reasonable structures as they are commonly over-smoothed and/or blurry. In this dissertation, we develop models based on deep convolutional neural networks to address two image restoration problems: image inpainting and image super-resolution. We develop a new approach for image inpainting that does a better job of reproducing missing regions exhibiting fine details. Furthermore, we extend this method to image superresolution by reformulating the problem as an in-between pixels inpainting task. We propose a two-stage adversarial model that comprises of an edge generator followed by an image completion network. The edge generator hallucinates edges of the missing region of the image, and the image completion network fills in the missing regions using hallucinated edges as a priori. We evaluate our model over the publicly available datasets and show that it outperforms current state-of-the-art techniques quantitatively and qualitatively.