NOAA Uses NI LabVIEW to Measure Deep Sea Scallops

Author(s):
Steve Tomanovich - Compass Technical Consulting, LLC.

Industry:
Industrial Controls/ Devices/ Systems

Products:
Vision, LabVIEW

The Challenge:
Developing a method for automatically measuring scallops for improved accuracy and speed over the traditional manual process.

The Solution:
Building a camera-based system with a conveyor transport and backlight illumination built on LabVIEW software.

The National Oceanic and Atmospheric Administration (NOAA) Fisheries Service conducts annual surveys between July and August to determine the abundance and size distribution of deep sea scallops (placopecten magelanicus) in areas between Cape Hatteras, NC and Georges Bank, NC. Measurements of scallop shell height constitute the primary observation on the catch with approximately 125,000 taken throughout an entire survey. The NOAA samples between 450 and 550 randomly selected stations using an 8 foot scallop dredge towed at 3.5 knots for 15 minutes. The methodology of obtaining shell height has advanced from taking manual observations to the most recent system of using magnetic measuring boards. The NOAA required improvements in accuracy, speed, and manpower to ensure that the data is efficiently collected and has high value.

William Kramer, an IT specialist at NOAA’s Woods Hole laboratory, obtained a Pioneer Funding grant to investigate using machine vision technology to improve the system. Compass Technical Consulting, LLC. of Rochester, New York constructed the prototype system described in this paper. We designed the conveyor-based system using National Instruments LabVIEW graphical development software to capture images from an ISG Firewire camera, run image algorithms, and output data. The system logs readings to the integrated computer in addition to the ship’s onboard computer system, which logs additional data about the catch.

A primary challenge to the vision system was the effect that a wide range of scallop sizes and geometries had on system timing and triggering. The system needed to inspect three species: sea scallops, Icelandic scallops, and calico. Each of these can range from a few millimeters to 200 mm in height. Scallop thickness was also variable, ranging from a few millimeters to more than 50 mm. The scallops could be loaded in any orientation and located anywhere across the belt. The size, orientation, and thickness variations along with the presence of debris on the belt made the use of typical camera triggering techniques, such as breaking a beam of light or contact switches, nearly impossible. We developed a “soft triggering” method to address this problem.

Another significant challenge stemmed from variation in color and texture of the scallop shells. Depending on their age and environment, they may be covered in “biogrowth,” barnacles, or other organic material, making the scallop appear dark in reflected light. The scallops are typically much lighter on the bottom side than the top and the scallops came through the system with either side up. After exploring our options, we opted to use a backlight system that provides a high-contrast silhouette of each scallop regardless of its shell color and texture (they all block light).

Finally, the geometric measurement of shell height presented a challenge due to the difficulty in consistently locating the correct flat edge of a scallop. The height is measured from the bottom of the shell “hinge” to the furthest point perpendicular from the hinge. To design an algorithm that would replicate this measurement method exactly, the system needs to detect the flat hinged edge in a way that is repeatable and accurate. Because shells could be cracked or have broken tabs in addition to normal shape variations, we decided not to try to detect the flat edge and developed a method, describe below, that would more reliably yield the same measurement.

We designed a standalone “Phase 1” system to meet these challenges and to undergo testing on the ship for the 2006 summer survey. We plan to evaluate robustness, repeatability, speed, and ease-of-use to determine additional requirements and upgrades for Phase 2. We developed the custom user interface shown in Figure 3 to allow both development and production run modes including an audible signal indicating a successful read. The system runs on 120 volts and has connections for wash water to spray debris from the belt and a serial output connector to send data to the ship’s onboard data collection system.

Our first problem was determining how to trigger the camera considering that scallops vary considerably in thickness, are placed randomly across the belt, and are confounded by debris on the belt. An additional concern was a decrease in performance due to exposure of any physical triggering device to salt water and salt air. As a solution, we decided to use the camera itself as a trigger by continuously capturing images (at 30 frames per second), testing each image for the presence of a scallop and processing the image if a scallop was detected. To effectively trigger, each scallop can only trigger the system once and the process must be fast to avoid a bottleneck in the overall processing time.

We created a series of overlapping regions of interest (ROIs) that would act as separate “soft triggers.” A simple (and fast) calculation of average code value taken at each trigger determines if a scallop is in the camera frame. If the value of any trigger drops below a specified threshold, the system saves the image for further processing. If no trigger value is below the threshold, the system discards the image and checks the next image. To avoid double-counting, a flag is set for each ROI when it is triggered and all subsequent images are ignored until the average value in that ROI is above the threshold and the flag is reset.

When a scallop successfully triggers the system, the image is passed to the main processing algorithm that is designed to (1) filter out unwanted debris, (2) select the correct scallop to process (there may be more than one in the image), and (3) calculate the scallop height in millimeters.

The method of calculating the final height from the scallop area maximizes accuracy and speed. As Figure 2 shows, the flat edge is not always obvious. Broken shells, bio-fouled shells or shells with particularly flat sides present a problem of detecting the correct signal (hinged flat edge) from the noise (all other flats). For these reasons, shell area is a starting point in calculating height. Shell area is a simpler and more stable measurement because it does not rely on the presence of specific physical characteristics in the shell geometry. Measuring area is a simple task using the NI Vision Development Module for LabVIEW, which offers functions for detecting, filtering, and analyzing particles. From area, the system uses the analysis library to calculate the Waddel Disk Diameter (WDD) of the scallop, which is the diameter of a circle with the same area. Effectively, the shell height was derived by calculating the diameter of a circle with an area equal to the scallop area.

We tested this calculation on a sample set of scallops to see how well it correlated to measurements made by hand. The data showed that smaller scallops were generally underestimated while larger scallops were overestimated. One possible explanation for this is the tendency of scallops to grow wider as they get larger. We implemented a correction equation to adjust the final scallop height measurement as a function of the equivalent diameter (WDD). The equation was a quadratic of the form: scallop height = a (WDD)2 + b (WDD) + c

The roots a, b, and c were derived empirically and implemented into the height calculation. We reran samples through the system and plotted them against the known heights, as shown in Figure 6. Accuracy at the prototype stage was determined acceptable, though it requires further testing on board ship under more rigorous conditions.

System specifications required measurement speeds of 1,800 scallops per hour (2 seconds per scallop). Scallops ranged from 25 mm to 200mm. Eventually, the conditions for the system would include direct salt water spray, wide temperature range, and lighting conditions from bright daylight to complete darkness. For the prototype, the conditions were somewhat relaxed, although the system meets the requirement to work in all lighting conditions. We modified a standard Dorner 2200 series conveyor to include a stainless steel idler roller (the drive roller is stainless), to accept the electroluminescent backlight and include a translucent belt. The conveyor was mounted on industrial swivel pads for operation on laboratory benches as well as on board ships.

We fabricated an aluminum shroud to house all the electronic components, the camera and the computer. We also created a wash system including a spray bar and squeegee and installed it beneath the conveyor to remove debris. The imaging system consists of an IEEE 1394 monochrome camera (ISG Lightwise 1.3 megapixel camera) with an 8 mm focal length lens. We chose the camera for three reasons: speed, resolution, and seamless software compatibility with LabVIEW. Minimizing lens distortion was critical to the application to ensure that a scallop would read consistently regardless of its position on the belt. The system required a 30 cm field of view and a working distance of about 35 cm. The 8 mm lens provided the required focal length, but resulted in distortion that caused the calculated shell height measurement to vary considerably when placed in the center of the belt versus either edge. Correcting the entire image using point-by-point remapping resulted in accurate measurements but was too time consuming. After spending more time than expected to find a “zero” distortion lens, we implemented a simple mathematical correction in the analysis algorithm to correct for scallop height based on the scallop’s location on the belt (a parameter easily extracted from the particle analysis step). No correction was made in the center of the belt with maximum correction occurring at the edges. This correction algorithm reduced the effect of lens distortion on scallop heights to an insignificant level.

We selected the electroluminescent backlight for its simplicity, resistance to the elements, and flexibility for integration into the existing system. The low-power application made it safe to use in a wet environment and its lack of delicate parts made it hold up against vibration and rough handling. Light output and uniformity were more than adequate, making the task of image analysis pleasurable compared to the reflected-light approach. The panel did require lamination to protect it from water, but this was fairly straightforward. Expected lifetime for the panel is approximately 3,500 hours.

Additional prototype testing is underway to finalize the changes required for the full production system. The most important change we are considering is replacing the laptop computer that the system runs on with a National Instruments Compact Vision System (CVS) to ensure robust performance in a difficult physical environment.

 

For more information, contact:

Steve Tomanovich

Compass Technical Consulting, LLC.

Tel: (585) 944-8425

E-mail: sjt@compasstechconsulting.com

 

 

Browse All Case Studies »