Towards efficient and scalable computer vision systems

Abstract

There is a large growth in hardware and software systems capable of producing vast amounts of image and video data. These systems are rich sources of continuous and possibly infinite image and video streams. This motivates researchers to build scalable computer vision systems that utilize data-streaming concepts for large-scale processing of visual data streams. However, several challenges still exist in building large-scale computer vision systems. The main challenge is the lack of formal and scalable mechanisms and frameworks for building and optimizing large-scale visual processing. Moreover, several fundamental computer vision tasks are computationally expensive and inefficient for scaling up for large-scale processing. This thesis presents formal methods and algorithms that aim to overcome these challenges and improve building and optimizing large-scale computer vision systems. We first describe a formal algebra framework for the mathematical description of computer vision pipelines for processing image and video streams. The algebra defines a set of abstract and concurrent operators with well-defined semantics for building scalable computer vision systems. It naturally describes feedback control and provides a formal and abstract method for data-stream manipulation, adaptive parameter selection, dynamic reconfiguration, incremental optimization, and defining common optimization and cost models. Second, we present new algorithms for efficiently processing image and video streams in two areas of computer vision: pixel-labelling problems and automatic visual surveillance. For pixel-labelling problems, we develop the sub-volume cost-filtering approach for solving both stereo-vision and optical-ow problems. The approach leverages sparse processing of the cost volume to achieve faster runtimes with comparable accuracy to the state-of-the-art algorithms. For automatic visual surveillance, we develop a new online algorithm for automatic lane and road-boundary detection. The algorithm runs in real time and is adaptive and able to handle several challenging environmental conditions. Finally, we express the road-boundary detection algorithm using our stream algebra. We use it as a case study for developing common optimization methods for parameter tuning in large-scale streaming pipelines.