Using NI LabVIEW with NI Data Acquisition and Vision Products to Acquire Video of an Existing Data Collection and Analysis System

Author(s):
Brian Lowe - National Institute for Occupational Safety and Health
Yong-Ku Kong - Sungkyunkwan University
Jungyoup Han - Siloam Biosciences, LLC

Industry:
Imaging Equipment, Government/Defense

Products:
Vision, LabVIEW

The Challenge:
Creating an integrated, “turn-key” system to complement hand grip contact force measurements, taken and analyzed with National Instruments LabVIEW graphical programming software and National Instruments DAQPad-6105 data acquisition devices, with synchronous video capture of work activity.

The Solution:
Using LabVIEW, the NI Vision Development Module, and NI-IMAQ for USB Cameras to configure a system to complement measurement trials with a synchronized video record saved in AVI file format.

Ergonomic assessments of hand-intensive work involve characterizing the posture, repetitive motions, and hand force the worker is required to exert. Posture and repetitive motions are, at a gross level, visually observable. However, the hand force exertion is more reliably assessed with direct measurement. Our group uses thin profile resistive force sensors to measure hand grip contact force on tools and workpieces grasped by workers. We place these sensors at the primary grip contact points of the hand, namely on the finger and thumb segments. The sensors are attached to a thin leather athletic grip glove that fits snugly on the hand of the worker. These athletic gloves are available in a wide range of sizes to accommodate variations in hand dimensions.

Our new system uses NI LabVIEW and NI DAQPad-6015 devices to acquire voltage data from these sensors. We are currently using 20 force sensors on the glove, which requires two DAQPad-6015 devices and a standard USB hub. (A modified 16-sensor version of the force measurement system could be employed that would require only a single DAQPad-6015 device.) We have constructed our own power supply, excitation, and amplification for these sensors that interface through the mass termination cable of the DAQPad devices.

Previous System Limitations

Our objective was to complement the hand grip contact force measurements with synchronous video capture of the work activity in an integrated system that would not require an expensive machine vision camera. We needed a method whereby a consumer camcorder or USB video camera could be easily integrated to acquire the video. Our previous method for synchronizing recorded video with sampled data was a crude approach in which a visual LED was illuminated by a switch in a simple voltage circuit sampled in an auxiliary analog channel. The video frame showing the first appearance of the LED in the illuminated state was indexed to the corresponding voltage step in the auxiliary channel. This method was tedious and required a great deal of post-processing to manually identify the LED illumination in the video sequence and temporally align this frame with the auxiliary channel in the data file.

Measurement and Video Recording

Using LabVIEW, the NI Vision Development Module, and NI-IMAQ for USB Cameras, we configured a system to complement measurement trials with a synchronized video record that is saved in an AVI video file format. NI-IMAQ for USB Cameras is a free software driver for acquiring images from DirectShow imaging devices into LabVIEW. With NI-IMAQ for USB Cameras, we control the frame grab from our video camcorder within the execution loop that samples from the DAQPad devices. In each iteration of the execution loop, a frame is grabbed from the camera and appended to an AVI file that is created for the sequence.

SubVIs display the individual sensor forces in a spatial overlay on a graphical image of the hand – including a scaled color intensity function that gives a real-time color intensity indication of the force level on each sensor. This subVI is called from within the execution loop. The sum of all individual sensor forces is also displayed on a vertical fill slide indicator showing the total hand force. Other statistical summaries of the measurement trial are generated and displayed at the cessation of the recording period.

Video Playback and Measurement Display

We developed a VI to play back the recorded trial and view the hand force level synchronously with the video frame images. The video sequence can be played back at selectable speed with a cursor automatically scrolling across the force graph and individual sensor color intensities continuously being updated. We also designed a second mode of playback in which the viewer can drag a cursor across the force graph and have the NI-IMAQ image window update and display the corresponding video frame from the AVI file. In this latter mode, the viewer can drag the cursor to peaks in the hand force profile and see exactly what the worker was doing at that instant and the working posture associated with that force level. Similarly, the viewer can scroll through the video sequence to find a task element or posture of interest and then refer to the graphical display of the force distribution and total hand force associated with the video frame of interest. We also added a subVI that the viewer can use to select any number of “key frames” (such as those corresponding to high force levels or awkward postures) and to instantly generate an HTML report displaying a JPEG still image of each key frame with an image of the force sensor color intensities.

In addition, we exploited NI-IMAQ functions to append images to create a composite AVI video containing the original video sequence and VI front panel graphs. Front panel graph updates are captured as pictures in each execution loop iteration and are appended to each video frame image to create a composite video sequence showing the worker’s activities with the corresponding hand force distribution displayed beneath. We can then import these video sequences into a video production software program (such as Adobe Premiere) to convert the file into any of several commonly shared compressed video file formats, including MPEG and Windows Media Video.

Field Testing

We conducted preliminary testing at an assembly plant where jobs included a variety of hand-gripping manipulations. Some of these jobs were associated with longer cycle times in which the worker was in continuous contact with the workpiece for more than five minutes. Peaks in the hand grip force profile were identified in the graph and the corresponding video frames were easily examined to identify grip types and arm postures. For work cycles of more than five-minute duration, it would have been almost impossible to associate multiple localized peaks in the force profile with specific events without a synchronized video recording. This technology has strong potential in ergonomics and biomechanical applications in which the analyst desires a video record of the work to accompany measurements of hand force exertion.

For more information, contact:

Brian D. Lowe, Ph.D., CPE

National Institute for Occupational Safety and Health 4

676 Columbia Parkway, MS C-24

Cincinnati, OH 45226 USA

Tel: (513) 533-8161

Fax: (513) 533-8596

E-mail: blowe@cdc.gov

Browse All Case Studies »