Using LabVIEW to Create a Revolutionary Thermal Imaging Device for Noncontact Respiration Monitoring

  Read in   |   Print Print

"The 4FA medical device, entirely powered by LabVIEW, eliminates patient discomfort, improves analysis accuracy and reduces set-up complexity, thereby increasing respiratory monitoring in Child patients by 50%."

- Abdulkadir Hamidu Alkali, Materials and Engineering Research Institute. Sheffield Hallam University

The Challenge:
Designing and developing a real-time noncontact medical device that uses infrared imaging to monitor patient respiratory rates.

The Solution:
Using NI LabVIEW software and the NI Vision Development Module to acquire thermal images from an infrared camera, then perform complex image processing to track patient’s face, eyes and nose—details which are then used to extract pertinent thermal information from the region of interest beneath each of the subject’s nostrils.

Author(s):
Abdulkadir Hamidu Alkali - Materials and Engineering Research Institute. Sheffield Hallam University
Reza Saatchi - Materials and Engineering Research Institute. Sheffield Hallam University
Heather Elphick - Sheffield Children's Hospital, National Health Service Trust, Sheffield
Derek Burke - Sheffield Children's Hospital, National Health Service Trust, Sheffield

Who We Are

Dr. Saatchi heads the Medical Electronics Engineering Research Group (MEERG) at Sheffield Hallam University. The group participates in a number of medical electronics research activities, such as non x-ray system to screen for bone fracture and a novel system to assess bone density. The group also leads research into contactless respiration monitors.

The Problem

Respiratory rate (RR) is a key indicator of a person’s well-being and thus it needs accurate measurement, according to an article in The Medical Journal of Australia. It complements other physiological measures such as temperature and heart rate to determine health status and deterioration.

Existing devices rely on patient contact to monitor respiration. This leads to various shortcomings, including complex set-up, patient discomfort, disruption of recording when device is dislodged, and erroneous measurements due to subject movement. These shortcomings were observed during a hospital audit of feverish children against the standards defined by the National Institute for Health and Care Excellence. The audit showed that 40 percent of the children at the hospital did not have their RR monitored due to unsettlement, according to an article in Pediatric Pulmonology. Realising the urgent need to find an alternative to the traditional contact-based devices, medics at Sheffield Children’s Hospital approached Dr. Saatchi at Sheffield Hallam University to research a new solution.

The RR of an unwell infant subject can be greater than 80 cycles per minute. Therefore, any image-based monitoring system must be able to process enough images per second to obtain an acceptable respiration signal. Therefore, we aimed to develop a noncontact monitoring device that processes infrared, thermal images at a stable rate of 25 frames per second. As with all electronic medical devices, our solution also needed to be user friendly, intelligent, reliable, and safe. Additionally, a contactless RR monitoring device must mitigate the effects of subject movement, including the involuntary movements caused by the human heartbeat. 

To meet these requirements, we created the 4FA system. We quickly obtained ethics approval and have since evaluated the 4FA device in the laboratory. The device is now undergoing clinical trials at the Sheffield Children’s Hospital.

4FA Device Background

A thermal camera picks the infrared energy radiated by an object, which does what a thermometer would do—measure temperature.

The human body has a mean temperature of 36.8 °C, which can be easily determined with thermal imaging. Air exhaled from the body is always warmer than inhaled air, with a mean temperature of 34.5 °C. Inhalation and exhalation constantly affect the skin temperature around the nose. We can observe these temperature fluctuations using a thermal camera. We can also carefully and intelligently analyse this area of skin to see the respiration pattern for the subject.

How 4FA Monitors Respiration

We acquired the thermal images at 25 frames per second. We used LabVIEW and products from the NI Vision platform to develop an application to control the camera operation. This meant we could perform data acquisition, image processing, feature extraction, and the display of thermal images and RR metrics with a single application. 4FA utilises four discrete stages to determine the respiration rate:

 

http:/vcm.natinst.com:27110/cms/images/casestudies/sbzguxdr8467664051650181052.jpg

Figure 1. The subject on a bed about one meter away from the camera

Stage 1—Image acquisition and subject segmentation

A thermal camera acquires images using the NI Vision Development Module and FLIR ThermoVision LabVIEW Toolkit. LabVIEW uses a high pass filter to remove impulse noise from each image. We then normalized the processed image from 16-bit greyscale to 8 bits in order to minimise memory usage. Then we segmented the normalised image to remove all background, leaving just the subject.

Stage 2—Face detection and tracking

We further segmented the subject image to extract the face from the rest of the visible body. We mapped the face location on the original acquired image to indicate detection.

 

http:/vcm.natinst.com:27110/cms/images/casestudies/tyhsfbnn1574542068581351573.png http:/vcm.natinst.com:27110/cms/images/casestudies/nqoeqxeh5141541024164905252.png 

Figure 2. Face detection in thermal images

Stage 3—Eye corner detection

The corners of the eyes are the warmest points on the face. We use that fact to locate one eye corner, and then use its value and position to locate the second eye corner using an intelligent learning algorithm developed using the NI Vision Development Module. The learning algorithm accurately detects eyes' corners even during head inclination.

 

http:/vcm.natinst.com:27110/cms/images/casestudies/slivxxsq3888913144490922766.png http:/vcm.natinst.com:27110/cms/images/casestudies/iinhwcue3124563500263849556.png http:/vcm.natinst.com:27110/cms/images/casestudies/reseckrg7673303196772484062.png

Figure 3. Detection and tracking eye corners

Stage 4—Nose detection and extraction of respiration feature

The nose is the coldest part on the face and the nostrils are located below the two corners of the eyes. We used the coordinates of the two corners of the eyes to search for the coldest point beneath them. Upon locating the tip of the nose, which is the face’s coldest point, we used its coordinates to define a region of interest within the thermal image.

 

http:/vcm.natinst.com:27110/cms/images/casestudies/qyvxahya1344224543848797838.png http:/vcm.natinst.com:27110/cms/images/casestudies/elryvscr3780915369320505030.png http:/vcm.natinst.com:27110/cms/images/casestudies/brxqvkhc6058081099832479316.png

Figure 4. Nose detection and tracking

 

We extracted and processed the pixel values within this region of interest as a respiration feature for the frame. We repeated the operation with each incoming frame, and then used various signal processing techniques to obtain the RR.

 

http:/vcm.natinst.com:27110/cms/images/casestudies/moxxobdn6702454292861376335.png

Figure 5. Respiration signals determined from the thermal images

 

http:/vcm.natinst.com:27110/cms/images/casestudies/muoldpxj860683933515099466.png

Figure 6. RR over time

 

Benefits of NI technologies

We used LabVIEW and NI vision software to build the whole software side of our application, which significantly reduced the development time and cost. The concept of data flow programming is easy to understand, and the wide range of functions in the associated NI vision library offer quick development of algorithms to accurately control the thermal camera and overall speed of execution for real-time operation. The ability to “see” signals and images in real time made the development process simple and enjoyable. In addition, troubleshooting is easy due to the data flow concept and is further enhanced by the probe and highlight execution tool. LabVIEW is amazing software that has the potential to influence a large group of programmers in the future.

Conclusions

By developing the 4FA system, we have provided a noncontact respiration rate monitoring approach that uses thermal image processing of a respiration region in the face. This region is dynamically tracked to allow for patient movements.

The 2010 Resuscitation Guidelines of the Resuscitation Council (RC) recommended that monitoring of the vital signs (including RR) of patients as one of the, “strategies for the prevention of avoidable in-hospital cardiac arrests.” A similar recommendation can be found in the National Confidentiality Enquiry into Patient Outcome and Death (NCEPOD) as well. Our device can improve RR monitoring and help achieve the recommendations of both NCEPOD and RC.

4FA is entirely powered by LabVIEW, eliminates patient discomfort, improves analysis accuracy, and reduces set-up complexity, thereby increasing respiratory monitoring in child patients by about 50 percent.

Author Information:
Abdulkadir Hamidu Alkali
Materials and Engineering Research Institute. Sheffield Hallam University
Sheffield
United Kingdom
Tel: 07427613931
abdulkadir.h.alkali2@student.shu.ac.uk

Bookmark and Share


Explore the NI Developer Community

Discover and collaborate on the latest example code and tutorials with a worldwide community of engineers and scientists.

‌Check‌ out‌ the‌ NI‌ Community


Who is National Instruments?

National Instruments provides a graphical system design platform for test, control, and embedded design applications that is transforming the way engineers and scientists design, prototype, and deploy systems.

‌Learn‌ more‌ about‌ NI