AI defect detection in manufacturing: a case study guide

An AI defect detection project in manufacturing puts cameras and a trained vision model on the line, so every part is checked against a learned picture of "good" and flagged when it deviates. The useful case studies show the method, the data and the metrics, and they are honest that the pilot, not the model, decides success.

Key takeaways

• Human visual inspection is less reliable than most plants assume: in a Sandia study, trained inspectors caught 85% of defective parts and wrongly rejected 35% of good ones.
• A working system has five parts: camera and lighting, image preprocessing, a detection model, a reject or alert action, and a feedback loop for retraining.
• Judge a case study by its numbers before and after, the defect types covered, the false-reject rate and how long the pilot ran. Treat any figure without a baseline as marketing.
• Involve the inspectors from day one. They know how defects really look and where images mislead, and the model is only as good as their labels.
• Plan for platform change: AWS retired Amazon Lookout for Vision on 31 October 2025, so keep your images and labels portable.

What does an AI defect detection case study actually prove?

A good case study proves that a specific model caught specific defects on a specific line, compared with a measured baseline. It does not prove that the same result transfers to your parts, lighting or line speed.

Many published "case studies" skip the parts that matter. They quote a detection rate without saying how many defective samples were in the test set, or report ROI without the cost base. Before you trust a figure, ask four questions:

• What was the baseline? Manual inspection, sampling, or a rule-based camera system. Each gives a different starting point.
• Which defect classes were in scope? Scratches and dents are easier than hairline cracks or contamination.
• What happened to good parts? A model that rejects too many good parts moves the cost from escapes to scrap and rework.
• How long did it run in production? A two-week pilot under controlled lighting says little about month six, when a new supplier changes the surface finish.

Why manual inspection leaves room for improvement

Manual inspection is flexible but inconsistent, and the inconsistency shows up as both missed defects and needless scrap.

The clearest public evidence comes from a Sandia National Laboratories study of visual inspection reliability. Eighty-two inspectors checked 140 parts for eight defect types. They correctly rejected 85% of defective items, but also rejected 35% of acceptable parts. The authors note the hit rate was not far above an industry average of about 80%, and that the task was mentally demanding.

Those numbers come from precision parts in a high-stakes setting, so they are not a universal benchmark. They do show the two failure modes any automated system has to beat: escapes and false rejects. Fatigue, shift changes and different personal thresholds between inspectors all push those numbers around.

Statistical process control (SPC) helps with trends, but it works on sampled measurements after the fact. It tells you a process is drifting, not that part number 4,312 has a crack. Camera-based inspection fills that gap by checking every unit at line speed.

A public example: BMW's AIQX platform

BMW Group describes its in-house platform AIQX ("Artificial Intelligence Quality Next") as a core part of its production system. According to BMW's own article on AIQX, cameras and sensors along the conveyor feed AI models that analyse data in real time and send feedback to line staff on smart devices.

The examples BMW gives are practical rather than headline-grabbing:

• Confirming that the right variant was built and that an assembly is complete.
• Spotting a missing item, such as a warning triangle, and alerting staff and production systems.
• Acoustic Analytics at Plant Dingolfing: microphones record driving noise and an AI model classifies it as a final check before handover.

BMW does not publish defect-rate or ROI figures for AIQX in that article, so we do not quote any. The lesson is in the design: many narrow, well-defined checks wired into the line and the workforce, rather than one model that "sees everything".

Worked example: inspecting solar panel alignment

To make the method concrete, here is how a camera-based inspection pipeline for solar panel assembly is typically built. Panels need sub-millimetre alignment of cells and ribbons, and small cracks or offsets reduce output later in the field.

Notice that steps 2 to 4 are classic computer vision, not deep learning. Geometry checks with clear tolerances are often more reliable and easier to explain with rules. Deep learning earns its place in step 5, where defects vary too much to describe with rules.

Which models are used for defect detection?

Convolutional neural networks (CNNs) are the default for classifying and locating visible defects. When you have few defective samples, anomaly-detection models such as autoencoders learn what normal parts look like and flag anything they reconstruct poorly. Synthetic images, for example from generative models, can add examples of rare defects, but they should never be your only test data.

How to run a pilot that produces a real case study

A pilot should answer one question: does this model, on this line, beat the current inspection on escapes and false rejects at an acceptable cost?

1. Pick one station and two or three defect classes. Choose ones that cost real money in scrap, rework or returns.
2. Measure the baseline first. Record current escape rate, false-reject rate and inspection time per unit for at least a few weeks.
3. Fix the optics before the model. Lighting, lens and mounting cause more failures than model choice. Test with the real conveyor speed.
4. Build a labelled dataset with inspectors. Agree on a defect catalogue with photos of each class and borderline examples.
5. Run in shadow mode. Let the system flag parts while humans keep deciding. Compare both on the same units.
6. Decide on thresholds with quality and production together. Every threshold trades escapes against scrap.
7. Write down the results with the baseline. That document is your internal case study and the basis for scaling.

Why the inspectors matter most

A team that builds the system without the inspectors is likely to miss how defects are actually recognised and where images are often misread. Inspectors know which marks are harmless, which angles hide cracks and which "defects" are just reflections. Bring them into labelling and threshold reviews, and give them a simple way to dispute a model decision. Those disputes become your best retraining data.

Cloud, edge and platform choices

Inference usually runs at the edge, next to the line, because a reject decision often has to happen in milliseconds and the line must keep running if the network drops. The cloud is the right place for image storage, labelling, training and fleet-wide monitoring of model accuracy.

Keep your data portable. AWS closed Amazon Lookout for Vision to new customers in October 2024 and discontinued it on 31 October 2025, pointing users to Amazon SageMaker, Amazon Bedrock and partner solutions, as explained in the AWS migration guidance. Teams that held their own images and labels in open formats could move. Teams that only had a model inside the service had to start again.

Common reasons projects stall

• Too few defect images. Good lines produce few defects. Start collecting rejected parts and images months before training.
• Lighting drift. A replaced lamp or sunlight through a window can break a model that worked in the pilot.
• Product changes. New variants need new labels. Budget for retraining as normal operations, not as a project.
• No owner after go-live. Someone must watch false-reject rates weekly and own the retraining loop, the same way someone owns a machine.
• Integration left late. The PLC signal, reject mechanism and MES logging take as long as the model.

How Opsio helps

Opsio builds and runs automated visual inspection systems and offers computer vision consulting from teams in Karlstad and Bangalore. We start with a scoped pilot on one station, measure the baseline with your quality team, and set up the cloud side for images, training and model monitoring as part of our wider data and AI services. For a side-by-side view of tools, see our guide to comparing defect detection systems for production lines.

Frequently asked questions

How accurate is AI defect detection compared with human inspectors?

It depends on the defect and the setup, so ask for the test set behind any accuracy claim. The useful comparison is your own baseline: the Sandia study found trained inspectors caught 85% of defective parts while rejecting 35% of good ones, which shows how much room a well-built system can have.

What types of defects can computer vision detect?

Surface defects such as scratches, dents, chips and stains, geometric faults such as misalignment or wrong dimensions, and assembly faults such as missing or wrong components. Internal defects need other sensors, for example X-ray, ultrasound or acoustic checks.

How many images do you need to train a defect detection model?

There is no fixed number. Anomaly-detection models can start from images of good parts only, while classifiers need labelled examples of each defect class. Plan to collect images for weeks before training and keep adding disputed cases afterwards.

How long does an AI visual inspection pilot take?

Most of the time goes into baseline measurement, optics and labelling rather than model training. A shadow-mode run long enough to see normal variation in shifts, batches and suppliers is what makes the result credible.

What did AWS replace Lookout for Vision with?

AWS did not launch a direct successor. Its guidance points to Amazon SageMaker, including pre-trained defect detection models in SageMaker JumpStart, Amazon Bedrock for use cases without strict latency needs, and partner solutions.