Designing New Visions

Author(s):
Oliver Sidla - Joanneum Research

Industry:
Research

Products:
LabVIEW

The Challenge:
Designing and implementing high-level computer vision algorithms for object detection, classification and tracking applications.

The Solution:
Leveraging the power of the LabVIEW analysis functions combined with the IMAQ Vision libraries for image processing enables the development of advanced vision algorithms and systems.

With the introduction of the IMAQ Vision image processing toolset a couple of years ago, for the first time we could use LabVIEW for development of machine vision applications. But there is more: we can efficiently use the huge collection of mathematical and analytical functions for the design and prototyping of high-level computer vision algorithms. The IMAQ image processing functions build a foundation on which we build advanced object detection, pattern recognition, shape detection, and classification algorithms.

A greater part of the computer vision research undertaken at the Institute for Digital Image Processing at JOANNEUM RESEARCH deals with problems related to surveillance applications. We try to answer questions such as ‘How can pedestrians be detected and tracked in video streams?’ Worldwide academic research tries to find solutions for these problems but still is in lack of satisfying algorithms, especially in difficult imaging situations. Nevertheless the demand for smart surveillance systems is increasing – at any typical site (underground stations, terminals, public places) there are far too many cameras installed as to allow real-time observation by human operators.

Our department has a long tradition of using LabVIEW and IMAQ Vision for software development. Initially we designed industrial machine vision applications, but the immediate benefits of LabVIEW where obvious: the efficiency and elegance by which applications can be built. Slowly, we began using LabVIEW for the design of more research-oriented prototypes and functions. Because LabVIEW contains so many high-level mathematical functions and programming constructs, it is naturally suited for algorithms that deal with vector and matrix oriented algorithms which are very common in computer vision research.

Shape recognition by Fourier features was first used by us to classify objects for a waste recycling machine – wth the built-in fast Fourier functions in LabVIEW, we could efficiently code and test this algorithm. Further requirements for other projects led us then to the design of shape analysis functions based on principal component analysis (PCA) – again we had the big advantage that Eigenvector functions are already built into LabVIEW, which made the implementation of the core of the algorithm a matter of a few hours. With the Eigenshape principle it is possible to describe even complex shapes with very few parameters, making it an ideal candidate for real-time implementation. The flexible nature of Eigenshapes can be brought into full use when they are combined with an iterative search procedure that tries to match them against image gradient fields. The resulting active shape model (ASM) approach has been prototyped and implemented completely in LabVIEW and IMAQ Vision.

A further activity within our surveillance-based work deals with motion detection and tracking. Starting with well-known background subtraction methods we went on with the more advanced Gaussian Mixture Models that have been coded as formula nodes. Any form of change detection, even with very sophisticated models, is limited in complex real-world situations. When it comes to tracking of objects, it is essential to have a sense of where things are going. The algorithms for the computation of optical flow are countless and we on our own have developed several solutions in the last 15 years. To optimize the efficiency and accuracy of such an algorithm for LabVIEW implementation, we chose to implement the Kanade Lucas Tomasi (KLT) point tracking algorithm. This method uses local gradient and image differences in order to calculate the motion of small patches from image to image in an iterative process.

By calculating the optical flow of an image sequence or by using the KLT algorithm, we have a good idea of the action which takes place in an image. The last question is, ‘What is moving?’ To recognize objects, we designed and implemented two distinct strategies based on the same implementation framework: extract a sub window of proper size from the image, calculate features from it, and use a classifier to decide whether it is a person, vehicle, or background. Our implementation uses PCA for vehicle detection and Gabor filters and histograms of gradients (HOG) features for pedestrian detection. All these methods create intermediate, more abstract representations of the image patch that we currently look at, thus making the unwieldy grid of pixel intensity values within a detection sub window more easily manageable and suitable for further analysis. In our implementation the classification is always done with a support vector machine (SVM), which represents the state-of-the art in computer vision for complex high dimensional classification tasks. This algorithm has the advantage that its decision function can be extremely easily implemented with the linear algebra modules in LabVIEW.

Combining all the modules mentioned, we created a pedestrian tracker prototype for surveillance applications. In each new video frame, pedestrian candidates are extracted using ASMs and the resulting shape candidates are verified with the pedestrian classifier. To track people from frame to frame, we calculate their motion with the KLT point tracking algorithm and verify their location with a texture similarity measure which is based on co-occurrence matrices. Wrapped around all this are data structures which handle the storage of tracks, filter and recognize duplicate pedestrians and manage the (dis)appearance of objects.

We prefer LabVIEW as a development platform for high-level vision system development due to the many advantages it presents. The combination of very powerful mathematical and analytical functions combined with a rich set of modern programming paradigms makes LabVIEW the ideal tool for prototyping algorithms. LabVIEW offers debugging options not so easily available in other languages, which saved us more than once. Finally, we do not need to switch to a different development platform in case that we want to integrate our code on an embedded platform or industrial PC installation, because those options are readily available.

All these properties make LabVIEW a unique development system that can bridge the gap between high-level vision research and low-level hardware-oriented systems.

For more information, contact:

Oliver Sidla

JOANNEUM RESEARCH

Wastiangasse 6

8010 Graz, Azstria

Tel: +43 316 876 1756

E-mail: oliver.sidla@joanneum.at

 

Browse All Case Studies »