Dimensional measurement vision systems: how they work and how accurate they are

How does it work?

Every measurement follows the same chain, and a weak link anywhere limits the result.

1. Presentation: the part arrives in a known position, on a fixture, conveyor or robot.
2. Lighting: backlights give sharp silhouettes for outer profiles and holes; front, ring, coaxial or structured light shows surface features.
3. Optics: the lens forms the image. Telecentric lenses keep magnification constant as the part moves slightly closer or further away, which matters for accurate 2D gauging.
4. Camera: an area-scan camera for discrete parts, a line-scan camera for continuous material or large parts, or a 3D sensor (laser profile, structured light, stereo) for heights.
5. Software: locates the part, finds edges, fits lines, circles and arcs, calculates dimensions and compares them with tolerances.
6. Output: sends pass/fail to the PLC to reject parts, and stores values and images for SPC and traceability.

How accurate is a vision measurement system?

Accuracy depends on pixel resolution, sub-pixel processing, lighting, optics and calibration. A practical rule from vision supplier Keyence is that the tightest tolerance you can judge reliably is about ±5 pixels.

Pixel resolution

Pixel resolution is the real length one pixel represents: the field of view divided by the number of pixels in that direction. Keyence's guide to dimension inspection works through a 100 mm field of view:

The ±5 pixel figure comes from the same guide. Typical machine vision repeats to about 0.1 pixel under ideal conditions, 0.5 pixel is assumed in practice, and the tolerance is set at about ten times that.

Two ways to get tighter tolerances follow directly: a smaller field of view (zoom in on the feature) or more pixels (a higher-resolution camera). Multiple cameras, each looking at one critical feature, are often cheaper than one very large sensor.

Sub-pixel edge detection

Software does not have to place an edge on a pixel boundary. It scans across the edge, builds an intensity profile, differentiates it and finds the peak of the change. Interpolating between the peak pixel and its neighbours puts the edge at a fraction of a pixel. Keyence describes measuring edge positions in units of 0.001 pixel this way.

Sub-pixel resolution is not the same as accuracy. It only pays off when the image is sharp, the lighting is stable and the part sits consistently, which is why the practical tolerance rule stays at ±5 pixels.

Calibration and uncertainty

The system must be calibrated against a traceable artefact, such as a calibration plate or gauge blocks, to convert pixels to millimetres and correct lens distortion. Then estimate the measurement uncertainty in the usual way. The international reference is the Guide to the Expression of Uncertainty in Measurement (JCGM 100:2008), and NIST summarises the method in Technical Note 1297.

Vision measurement vs other methods

Many plants combine them: inline vision checks every part, and a CMM verifies a sample and confirms the vision system stays in calibration.

How do you implement a vision measurement system?

Start from the drawing and the tolerances, not the camera catalogue. The steps below are the ones that decide whether a project works on the line.

1. List the features and tolerances to measure, plus the cycle time per part.
2. Check feasibility with the ±5 pixel rule: required pixel resolution = tolerance / 5. Choose field of view and camera accordingly.
3. Design lighting and optics with real parts, including worst-case surface finishes and colours.
4. Fix part presentation: fixtures, conveyor guides or robot handling so the part sits consistently.
5. Control the environment: vibration, ambient light, temperature and dust on lenses.
6. Calibrate with a traceable artefact and document the procedure.
7. Run a gauge study (repeatability and reproducibility) and compare against CMM results on the same parts.
8. Integrate with the PLC for rejects and with MES, SQL or a data platform for values and images.
9. Monitor with SPC charts and capability indices (Cp, Cpk) so drift is caught before parts go out of tolerance.

Common pitfalls

• Choosing a camera first and discovering the lighting cannot show the edge.
• Ignoring part position and perspective error; a standard lens can change the measured size as height varies.
• Treating sub-pixel resolution as accuracy.
• Storing only pass/fail, which throws away the data needed to fix the process.

Where are these systems used?

• Automotive: gaskets, stamped parts, fasteners, connector pins, gap and flush.
• Electronics: component leads, capacitor diameters and lead alignment, connector pitch.
• Medical devices and pharma: syringes, catheters, vials and closures, where records support regulatory traceability.
• Metal and plastics: machined, turned and moulded parts with hole and diameter tolerances.
• Packaging: container dimensions, cap height and label position at line speed.

Where AI fits

Classic rule-based vision is still the right tool for dimensions, because edges and geometry are deterministic and auditable. Deep learning helps with what rules handle badly: locating parts with variable appearance, and spotting cosmetic defects next to the measurement. A typical modern station uses rule-based gauging for dimensions and an AI model for defects.

How Opsio helps

Opsio builds and runs visual inspection and computer vision solutions, from feasibility tests to inline deployment, plus the cloud side for storing images and inspection data. See our machine vision inspection systems, automated visual inspection and computer vision consulting. For an industry example, read our guide to automotive component visual inspection.

Frequently asked questions

How accurate can a machine vision measurement system be?

It depends on field of view and pixel count. With a 100 mm field of view, a 2-megapixel camera gives about 0.083 mm per pixel and a practical tolerance of roughly ±0.42 mm. Smaller fields of view, higher-resolution cameras and telecentric optics reach much tighter tolerances.

What is sub-pixel measurement?

Sub-pixel measurement finds an edge position between pixel centres by analysing how brightness changes across neighbouring pixels. It improves resolution well beyond the pixel size, but real accuracy still depends on lighting, optics and calibration.

Can a vision system replace a CMM?

For many 2D features checked in line, yes. For tight 3D tolerances, first-article inspection and audits, a CMM is still more accurate. Most plants use vision for 100% inline checks and keep the CMM for sampling and verification.

Why are telecentric lenses used for measurement?

A telecentric lens keeps magnification constant across a range of distances, so a part that sits slightly higher or lower measures the same size. Standard lenses change apparent size with distance, which adds error.

What data should a vision measurement system store?

Store the measured values per feature, the pass/fail result, a time stamp, part or batch ID and, ideally, the image. That supports SPC, traceability, customer complaints and recall investigations.