The July 2022 issue of IEEE Spectrum is here!

Close bar

Deep Learning Has Reinvented Quality Control in Manufacturing—but It Hasn’t Gone Far Enough

AI systems that make use of “lifelong learning” techniques are more flexible and faster to train

5 min read
Photograph of bottles on a manufacturing line with a square over a bottle that a system would note as defective
Photo-illustration: Neurala

This is a guest post. The views expressed here are solely those of the author and do not represent positions of IEEE Spectrum or the IEEE.

In 2020, we've seen the accelerated adoption of deep learning as a part of the so-called Industry 4.0 revolution, in which digitization is remaking the manufacturing industry. This latest wave of initiatives is marked by the introduction of smart and autonomous systems, fueled by data and deep learning—a powerful breed of artificial intelligence (AI) that can improve quality inspection on the factory floor.

The benefit? By adding smart cameras to software on the production line, manufacturers are seeing improved quality inspection at high speeds and low costs that human inspectors can't match. And given the mandated restrictions on human labor as a result of COVID-19, such as social distancing on the factory floor, these benefits are even more critical to keeping production lines running.

While manufacturers have used machine vision for decades, deep learning-enabled quality control software represents a new frontier. So, how do these approaches differ from traditional machine vision systems? And what happens when you press the “RUN" button for one of these AI-powered quality control systems?

Before and After the Introduction of Deep Learning in Manufacturing

To understand what happens in a deep learning software package that's running quality control, let's take a look at the previous standard. The traditional machine vision approach to quality control relies on a simple but powerful two-step process:

Step 1: An expert decides which features (such as edges, curves, corners, color patches, etc.) in the images collected by each camera are important for a given problem.

Step 2: The expert creates a hand-tuned rule-based system, with several branching points—for example, how much "yellow" and "curvature" classify an object as a "ripe banana" in a packaging line. That system then automatically decides if the product is what it's supposed to be.

This method was simple and effective enough. But manufacturers' needs for quality control have rapidly evolved over the years, pushing demand to the next level. There aren't enough human experts to support manufacturers' increased appetite for automation. And while traditional machine vision works well in some cases, it is often ineffective in situations where the difference between good and bad products is hard to detect. Take bottle caps, for example—there are many variations depending on the beverage, and if one has even the slightest of defect, you run the risk of having the whole drink spill out during the manufacturing process.

The new breed of deep learning-powered software for quality inspections is based on a key feature: learning from the data. Unlike their older machine vision cousins, these models learn which features are important by themselves, rather than relying on the experts' rules. In the process of this learning, they create their own implicit rules that determine the combinations of features that define quality products. No human expert is required, and the burden is shifted to the machine itself! Users simply collect the data and use it to train the deep learning model—there's no need to manually configure a machine vision model for every production scenario.

Using a Conventional Deep Learning Model for Quality Control

Data is the key in deep learning's effectiveness. Systems such as deep neural networks (DNNs) are trained in a supervised fashion to recognize specific classes of things. In a typical inspection task, a DNN might be trained to visually recognize a certain number of classes, say pictures of good or bad ventilator valves. Assuming it was fed a good amount of quality data, the DNN will come up with precise, low error, confident classifications.

Let's look at the example of spotting good and bad ventilator valves. As long as the valve stays the same, all manufacturers have to do is hit the “RUN" button and inspection of the production line can begin. But if the line switches to a new type of valve, the data collection, training, and deployment must be performed anew.

For conventional deep learning to be successful, the data used for training must be “balanced." A balanced data set has as many images of good valves as it has images of defective valves, including every possible type of imperfection. While collecting the images of good valves is easy, modern day manufacturing has very low defect rates. This situation makes collecting defective images time consuming, especially when you need to collect hundreds of images of each type of defect. To make things more complex, it's entirely possible that a new type of defect will pop up after the system is trained and deployed—which would require that the system be taken down, retrained, and redeployed. With wildly fluctuating consumer demands for products brought on by the pandemic, manufacturers risk being crippled by this production downtime.

A Different Kind of “RUN" Button

There may yet be a lesson to be learned from the traditional machine vision process for quality control that we described earlier. Its two-step process had an advantage: The product features change much more slowly than the rules. This setup meshes well with the realities of manufacturing, as the features of ventilator valves persist across different production types, but new rules must be introduced with each new defect discovered.

Conventionally, a deep learning model has to be retrained every time a new rule must be included. And to do that retraining, the new defect must be represented by the same number of images as all the previous defects. And all the images must be put together in a database to retrain the system, so that it learns all the old rules plus the new one.

To solve this conundrum, a different category of DNNs is gaining traction. These new DNNs learn rules in a much more flexible way, to the point that new rules can be learned without even stopping the operating system and taking it off the floor.

These so-called continual or lifelong learning systems, and in particular lifelong deep neural networks (L-DNN), were inspired by brain neurophysiology. These deep learning algorithms separate feature training and rule training and are able to add new rule information on the fly.

While they still learn features slowly using a large and balanced data set, L-DDNs don't learn rules at this stage. And they don't need images of all known valve defects—the dataset can be relatively generic as long as the objects possess similar features (such as curves, edges, surface properties). With L-DNNs, this part of model creation can be done once, and without the help of the manufacturers.

What our hypothetical valve manufacturer needs to know is this: After the first step of feature learning is completed, they need only provide a small set of images of good valves for the system to learn a set of rules that define a good valve. There's no need to provide any images of defective valves. L-DNNs will learn on a single presentation of a small dataset using only “good" data (in other words, data about good ventilator valves), and then advise the user when an atypical product is encountered. This method is akin to the process humans use to spot differences in objects they encounter every day—an effortless task for us, but a very hard one for deep learning models until L-DNN systems came along.

Rather than needing thousands of varied images, L-DNNs only require a handful of images to train and build a prototypical understanding of the object. The system can be deployed in seconds, and the handful of images can even be collected after the L-DNN has been deployed and the “RUN" button has been pressed, as long as an operator ensures none of these images actually shows a product with defects. Changes to the rules that define a prototypical object can also be made in real time, to keep up with any changes in the production line.

In today's manufacturing environment, machines are able to produce extremely variable products at rates that can easily surpass 60 items per minute. New items are constantly introduced, and previously unseen defects show up on the line. Traditional machine vision could not tackle this task—there are too many specialized features and thresholds for each product.

When pressing the “RUN" button on quality control software that's powered by L-DNN systems, machine operators can bring down the cost and time of optimizing quality inspection, giving the manufacturing industry a fighting chance of keeping up with the pace of innovation. Today, global manufacturers such as IMA Group and Antares Vision have already begun implementing such technologies to help with quality control, and I expect that we'll see many others begin to follow suit in order to stay competitive on the global stage.

The Conversation (0)

AI-Guided Robots Are Ready to Sort Your Recyclables

Computer-vision systems use shapes, colors, and even labels to identify materials at superhuman speeds

11 min read
Vertical
An image of different elements of trash with different markings overlaying it.
AMP robotics
Green

It’s Tuesday night. In front of your house sits a large blue bin, full of newspaper, cardboard, bottles, cans, foil take-out trays, and empty yogurt containers. You may feel virtuous, thinking you’re doing your part to reduce waste. But after you rinse out that yogurt container and toss it into the bin, you probably don’t think much about it ever again.

The truth about recycling in many parts of the United States and much of Europe is sobering. Tomorrow morning, the contents of the recycling bin will be dumped into a truck and taken to the recycling facility to be sorted. Most of the material will head off for processing and eventual use in new products. But a lot of it will end up in a landfill.

Keep Reading ↓Show less
{"imageShortcodeIds":[]}