The phrase “finding a needle in a haystack” hits close to home when you’re tasked with preventing foreign materials from contaminating product on your production line.
Imagine a processing establishment is running hundreds of thousands of pounds of chicken parts through their lines each day. Somewhere among those chicken parts lies a tiny tip of plastic glove or a chunk of wood pallet. What’s the best way to control and prevent these types of foreign materials in meat? The answer may be more complex than you think.
This article explores the various technical factors that complicate foreign material detection. You’ll learn why not every physical contamination event registered by a detector is valid. And you’ll discover how detection curves can help you understand the probability of actually finding the contaminants you’re looking for. Finally, you’ll walk away with plenty of proven ideas for evaluating your own foreign material control program.
5 Technical Issues Impacting Detection Effectiveness
The effectiveness of any detection system is influenced by some important technical factors:
1. Resolution
Resolution is important no matter which detection technology you choose. There’s a limit to how far you can zoom into an object and still see an appropriate level of detail. The image below demonstrates this effect on a camera. Notice how the zoomed in image of the flower on the right-hand side makes the image appear more blurry.
When you look at the pixel size in a detector it’s natural to assume that it will find an object as small as that pixel. In reality, a pixel resolution of two millimeters means that the detector can only find objects that are four millimeters in size.
2. Motion
Objects in motion are more blurry and less defined than those at rest. Motion is unavoidable on a processing line, but as line speeds continue to increase, the ability for detectors to accurately identify foreign materials will be impacted.
3. Lighting
Detection would be simpler if your product was completely uniform. In reality, products vary in thickness, and that impacts how much light your detection systems need to do their jobs. Take x-ray detection for example. If your product is thick, the detector needs to crank up the photons to shoot more x-rays through it. If a thinner product subsequently goes through the same detector the light will be oversaturated making it more difficult to detect foreign materials.
4. Background Complexity
X-ray and vision systems can be confused just like human eyes. That’s why the complexity of your background (also called “noise”) matters. Unsurprisingly, it’s much easier to find foreign materials when they’re on a uniform background.
Imagine detecting a 0.8 mm metal ball on a cardboard box that’s the perfect distance from your detection technology, vs trying to find that same metal resting on mutli-colored product that’s all different shapes and sizes. A more complex background makes it that much more difficult to detect the same object, even under identical conditions.
5. Signal-to-noise ratio
It’s much easier to hear a pin drop in your living room than in a busy airport. Noise similarly impacts the ability for detection technologies to do their jobs. The signal-to-noise ratio compares the “signal” (information which comes from the material of interest, in this case the product being reviewed), with the “noise” (which comes from various sources and can include anything present which isn’t the material of interest).
For example, if you have strong electromagnetic interference close to an x-ray machine, it will not work as well as it would in an empty room. When it comes to vision systems, an external light that interferes with the light you’re shining on an object can impact detection efficacy.
Foreign Material Detection = Seeing + Decision Making
Detection isn’t simply the act of identifying foreign materials on your product – it’s about the ability to make appropriate decisions about their impact. You need to determine whether the event actually occurred and is impacting the quality of your product.
As we’ve just learned, there are multiple factors that can muddy the detection waters. This raises an important question for those running a production line: How do you know whether the products you rejected actually contained foreign materials? This is where the Confusion Matrix comes into play.
Meet the Confusion Matrix
The Confusion Matrix compares the expected outcome of an event with the actual outcome of an event. There are four possible outcomes, as shown in the chart below.
As you can see from the matrix, some detection or non-detection events registered by a detector are incorrect. So, for example, if we are testing for a disease (like cancer), we need to know how often a particular test will detect cancer when it *IS* present (True Positive), and how often the test will detect cancer when it *IS NOT* present (False Positive).
In the case of foreign materials detection, we have to anticipate that a detector will sometimes find a foreign material that *isn’t* there (False Positive), or miss a foreign material that *is* there (False Negative). When these False Positives or False Negatives happen too frequently then the system isn’t working well for its chosen application. There are a few additional things to note about True Positives and False Positives:
- As the True Positive rate goes down, more foreign materials are missed and may end up in the hands of consumers.
- This can be mitigated by “turning up the sensitivity” of the detection system. However, doing so may increase the incidence of False Positives.
- Higher False Positives may lead to throwing away more good product.
The secret to making a detection system work well for a specific application lies in carefully balancing the rates of True Positives and False Positives to meet your needs. Therefore, understanding the Confusion Matrix results of any detection system is a crucial place to start. As you can imagine, a balance between all the components of the Confusion Matrix is necessary in the selection of a best detection system.
But there’s more! We also need to look at the Detection Curve, for a further understanding of how effective a particular detector will be.
Assessing Detection Curves
A lot of the problems with foreign material detection can be solved by looking at detection curves. These curves don’t just tell you the smallest or largest object you can find, but also the probability with which you are about to find a given object.
Most detection methods get better at detection as the size of the foreign material increases. We can see this illustrated on the chart below. It shows the probability of detection (True Positive from our Confusion Matrix) at different sizes of foreign material, across three different detectors.
- You can see that Detector 1 doesn’t start detecting any foreign objects until they are medium sized and detects just under 75% of even large objects.
- In contrast, Detector 3 sees a high percentage of small objects and 100% percent of large objects.
Detection capability can be improved on most detection systems, but that improvement will come at a significant cost: increasing False Positives and therefore increasing rejection of good product.
Hopefully by now it’s becoming clear why it’s so important to understand both detection curves and the False-Positive/False-Negative rates for any detection system, especially in a food processing plant.
Every detection method (x-ray, metal detection, vision systems, manual inspection) will present some trade-off between actual (correct) detection, rejection of good product (False Positive) and missed detections (False Negative).
Comparing both actual detection and False Positive/False Negative rates is the only way to truly understand the capabilities of any detection system – including manual inspection.
Reimagining Foreign Material Detection
When you understand the nuances and trade-offs associated with detection, it’s easier to mitigate issues and ensure your foreign material control program provides the highest value.
When you’re evaluating different approaches, it’s important to know that any vendor promising 100% detection isn’t being honest. There are no guarantees. There’s only high confidence in the statistical probability of finding the foreign materials that matter to your operation. Each automated inspection technology offers its own strengths and weaknesses:
- Vision systems are great at finding objects when there’s strong contrast with the background.
- X-ray is proven to find contaminants embedded in product, but only if they are high-density materials. It will miss a lot of common low-density contaminants, including plastic, wood and cardboard.
- Metal detectors can only find ferrous or iron-containing metals.
- Hyperspectral imaging is amazing at finding even the tiniest low-density foreign materials on the surface of product.
The detection system you choose will ultimately depend on the problems you’re trying to address. It’s not just about the sizes of the objects you want to find, but also the types of foreign materials that you’re struggling with or worried about. Check out how the various technologies stack up in a real-world test.
Now is also a good time to start paying attention to the software that’s running detection systems. Some systems are sophisticated enough to capture rich data that paves the way for a more holistic approach to optimizing performance. Smart algorithms and machine learning tame complexity and identify trends. You can use this rich information to start implementing new and better processes to protect your business from unnecessary waste or the fallout of foreign material incidents.
It should be clear by now that foreign material detection is a complex issue. It’s no surprise that many operations are now choosing a multi-hurdle approach that counts on several different technologies to tighten their detection net.
Regardless of which detection technologies you’re considering, any vendor you’re talking to should be willing and able to show you their own detection curve and results from the Confusion Matrix. Our PPO team is always ready to share and chat about ours! Get in touch to start the discussion.