What Is Visual Inspection?

Visual inspection is a task intended to find surface failures or defects. It’s commonly applied in sectors such as manufacturing, construction, and aerospace.

Automated Visual Inspection Systems

In the manufacturing industry, automated visual inspection systems with high-resolution cameras efficiently detect microscale or even nanoscale defects that are difficult for human eyes to pick up. However, false detection sometimes happens when an unknown or a variety of defects are present, which is a major challenge. Hence, software with deep learning technology such as MATLAB^® and the Computer Vision Toolbox™ Automated Visual Inspection Library is playing a more important role.

Developing an Automated Visual Inspection System with MATLAB and Computer Vision Toolbox

With MATLAB and the Computer Vision Toolbox Automated Visual Inspection Library, you can develop visual inspection systems. MATLAB supports the full workflow, from image acquisition to algorithm development and then deployment. Interactive apps in MATLAB help engineers explore, iterate, and automate algorithms to improve productivity. Engineers apply these techniques for many industrial applications.

Success Stories: Automated Visual Inspection in Automotive and Aerospace Industries

For example, Musashi Seimitsu Industry, an automotive parts manufacturer, was inspecting approximately 1.3 million parts per month with manual visual inspection. Using MATLAB to develop deep learning–based approaches to detect and localize different types of anomalies, it built an automated visual inspection system for inspecting bevel gears. The updated approach is expected to considerably reduce the company’s workload as well as its costs.

Similarly, Airbus built a robust visual inspection artificial intelligence (AI) model for automatically detecting any defects in multiple aircraft components to ensure its airplanes have no defects before entering service. Using MATLAB simplified the process by enabling Airbus engineers to interactively prototype algorithms and test for defects in a short amount of time.

The Defect Detection Process: Data Preparation, AI Modeling, and Deployment

The defect detection process can be broken down into three main stages: data preparation, AI modeling, and deployment.

For visual inspection, anomalous images are sometimes rare or diverse. In that case, you can train an anomaly detector, unsupervised learning that only needs normal images on training. If there are a sufficient number of abnormal images, supervised learning should work.

In the following section, you can learn the necessary steps and practical MATLAB functionalities for anomaly detection methods as unsupervised learning and object detection as supervised learning.

Data Preparation for Visual Inspection

Data comes from multiple sources and is usually unstructured and noisy, making data preparation and management difficult and time-consuming. Preprocessing images in the data set will result in higher accuracy in detecting anomalies.

Image Preprocessing with MATLAB

MATLAB has several apps to support various preprocessing techniques. For example, the Registration Estimator app in MATLAB lets you explore various algorithms to register misaligned images, making it easier for AI models to detect defects.

MATLAB provides automation capabilities to accelerate the labeling process. For example, the Image and Video Labeler app can apply custom semantic segmentation or object detection algorithms to label regions or objects in an image or video frames. 

AI Modeling for Visual Inspection

Characteristics of Anomaly Detection Methods Used in Visual Inspection

Available anomaly detectors for visual inspection include:

• Fully convolutional data description (FCDD)
• FastFlow
• PatchCore

The following table distinguishes the characteristics and performance of these anomaly detection methods available through the Computer Vision Toolbox Automated Visual Inspection Library during training and inference.

Anomaly Threshold and Explainable AI in Visual Inspection

For anomaly detection methods, the automated calculation of the anomaly threshold helps to avoid human-dependent variations in judgment. Evaluation with the explainable AI view enables to explore and interpret the classification results.

Pretrained Networks Used in Supervised Learning in Visual Inspection

When applying deep learning for supervised learning, there are two approaches. One approach is to build and train a deep network from scratch. The other is to adjust and fine-tune a pretrained neural network, also known as transfer learning. Both approaches are easy to implement in MATLAB.

Using the Deep Network Designer App to Build, Edit, and Train Deep Learning Networks

MATLAB provides the Deep Network Designer app, which lets you build, visualize, edit, and train deep learning networks. You can also analyze the network to ensure that the network architecture is defined correctly and detect problems before training.

Using pretrained networks from external platforms in MATLAB, you can import networks and network architectures from TensorFlow™, PyTorch^®, and from and to the ONNX™ model format. You can use these pretrained and edit them for transfer learning.

Object Detection to Detect and Localize Defects

Object detection like YOLOX is used to detect, localize, and classify defects in an image. The YOLOX object detection model is a single-stage, anchor-free technique that helps with small object detection, significantly reduces the model size, and improves computation speed compared to previous YOLO models. The exported data labeled with rectangular regions of interest (ROIs) is used to train object detection models.

Deployment of the Visual Inspection System

Code Generation and Deployment Framework

Deep learning models must be incorporated into a larger system to be useful. MATLAB offers a code generation framework that allows models developed in MATLAB to be deployed anywhere, without having to rewrite the original model. This gives you the ability to test and deploy the model within an entire system.

Deployment to Embedded Hardware Platforms First

MATLAB enables you to deploy your deep learning networks to various embedded hardware platforms, such as NVIDIA^® GPUs, Intel^® and ARM^® CPUs, and Xilinx^® and Intel SoCs and FPGAs. With the help of MATLAB, you can explore and target embedded hardware easily.