Background
A critical performance requirement for embedded computer vision is energy-efficiency in order to preserve battery life for mobile and autonomous platforms. In particular, the image sensor and readout can take up a significant amount of energy in a computer vision pipeline, particularly if the sensor is capturing and processing video data in real-time. For instance, performing continuous face detection with Google Glass drains its battery in 45 minutes, with image sensing occupying 50% of the energy budget.
The primary mechanism by which image sensors can save energy is to limit their readout to portions of the array known as regions-of-interest (ROIs). This is a form of spatial subsampling, and can be achieved using windowing, column/row skipping, or binning in the image sensor. Such a content-driven approach can result in significant energy savings. However, this comes at the cost of potential loss of visual detail for objects, which may be necessary for end-task performance. This will certainly be the case if the subsampling approach is agnostic to the semantic information in the frames. Thus, there is an opportunity to design scene-dependent smart sampling approaches to save energy while preserving task performance.
Invention Description
Researchers at Arizona State University have developed an adaptive algorithm for video subsampling, which is aimed at enabling accurate object detection while saving sampling energy. The approach utilizes objectness measures, which has been shown to be accurately estimated even from sub-sampled frames, and then uses that information to determine the adaptive sampling for the subsequent frame. Energy savings of 18%–67% have been demonstrated with only a slight degradation in object detection accuracy in experiments. These results provide support for adaptive subsampling as a promising avenue for embedded computer vision in the future.
Potential Applications
• Computer vision
• Mobile sensors
• Surveillance
• Autonomous systems
Benefits and Advantages
• Dynamically balances energy usage and performance
• Combines object detection with adaptive subsampling
• May significantly extend battery life in image-sensing devices
Laboratory Homepage of Professor Suren Jayasuriya