Overcoming High Precision Optical Challenges Using High-Speed 3D Inspection Technology

Interview conducted by Will SoutterJan 27 2016

In this interview, Tim Skunes from CyberOptics Corporation talks to AZoM about their 3D optical inspection technology, and how it can be used to solve challenges in SMT electronics manufacturing.

Could you provide our readers with an overview of the history of CyberOptics and explain how the company has grown since it was founded in 1984?

CyberOptics was founded by Late Dr. Steve Case in 1984 with its first office located in Minneapolis. The initial products of the company were laser-based, non-contact 3D sensors that could be interfaced directly with a computer.

CyberOptics’ first product was a Point Range Sensor enabling 3D micro-measurement capability which was used in a wide variety of applications where speed, accuracy, and the ability to measure deformable materials was important. The Point Range Sensor was the first commercial “digital” range sensor and employed an innovative detector technology.

CyberOptics’ first generation multiple reflection suppression (MRS) technology was enabled by this new digital detector technology and the ability to process images on a personal computer. Non-contact measurement of challenging metrology applications such as threaded fasteners, contact lenses, hybrid ceramic electronic circuits, and reverse CAD were now possible in a commercial system.

In the late 1980’s, companies such as Motorola, IBM, Hewlett Packard, and Digital Equipment Corporation were developing the next generation of printed circuit board technology called surface mount technology (SMT). Steve noted that CyberOptics’ sensors and automated scanning systems were being used more and more for SMT process development such as solder paste printing and lead coplanarity measurement of SMT packages.

Download the CyberOptics white paper on automated high-speed 3D optical inspection.

As SMT assembly technology was becoming mainstream, CyberOptics developed a number of application specific 3D measurement systems throughout the 1990’s for SMT such as a benchtop solder paste measurement system called the Laser Section Microscope (LSM) and the Laser Lead Locator for lead coplanarity measurement.

During this time, CyberOptics also developed a line of embedded sensors, called LaserAlign, that are still widely employed today for high speed, vision-guided robotic assembly applications in SMT pick-and-place systems.

In the late 1990’s CyberOptics introduced a number of high speed 3D solder paste inspection systems that could be installed directly into automated SMT production lines for defect detection and process control. In 1999, the company acquired a powerful machine vision technology that was originally developed at the University of Manchester. This new technology enabled CyberOptics to create an automated optical inspection (AOI) system for use at yet another step in the SMT production line.

In addition to the surface-mount technology (SMT) market, the company also developed and marketed proprietary measurement devices for the semiconductor markets, which now have a world-wide presence at most major fabs and equipment OEMs.

Since then CyberOptics has continually invested in R&D and focused on providing quality inspection solutions and measurement devices through technology leadership, vast industry experience and worldwide support network.

In 2014, Dr. Subodh Kulkarni joined as the President and CEO driving the company’s technology leadership efforts. After an acquisition of LDI that year, CyberOptics also entered the 3D scanning and metrology markets, in order to leverage its high-precision 3D sensor technology into the company’s third key market.

Today, CyberOptics is a global leader in high precision sensing solutions leveraging technology into SMT, 3D scanning and semiconductor markets.

Could you explain the basic principle which underpins how most 3D inspection technology systems work?

There are a number of important 3D optical inspection technologies including white light interferometry, confocal scanning, and structured light triangulation.

CyberOptics industrial inspection systems typically use structured light triangulation where a series of structured light patterns are projected onto a measurement area and then imaged by high speed cameras that view the measurement area from a different direction. The imaged patterns are distorted by the objects being measured and the principles of phase shift profilometry can be used to accurately calculate the three dimensional shape of the objects from the distorted patterns.

For example, a fringe pattern is obliquely projected onto a measurement area in Figure 1 containing a rectangular block. The projected light is diffusely reflected and the pattern is then imaged by a camera from a different direction than the projector. The fringe pattern image is distorted by the height of the rectangular block as shown in Figure 2.

Phase shift profilometry has an unparalleled combination of speed and accuracy and, when architected properly, permits sub-micron measurement accuracy and inspection rates in excess of 50 million three dimensional measurement points per second.

Figure 1. Fringe pattern projected onto measurement area

Figure 2. Fringe image distortion

Could you outline the main inspection challenges which exist when inspecting SMT assemblies?

Many inspection challenges exist when inspecting SMT assemblies due to the varied nature of SMT applications as well as inspection processes and quality requirements. Some of the most common obstacles to high quality 3D inspection are tall components, highly reflective surfaces, modeling complex three-dimensional shapes and inspection speed requirements.

How do tall components cause shadowing effects and why is this a problem for 3D sensing systems? What are the other challenges caused by tall components?

Many SMT assemblies, especially in automotive and industrial environments, contain capacitors and connectors which can be over 30mm tall. Triangulation sensing angles needed to provide adequate height accuracy will have blind areas on the circuit board near these tall components. In such scenarios, the shadow effect is a real concern as illustrated in the figure below.

Figure 3. Shadowed component

As it is sometimes required to inspect near the base of tall components, the ability to overcome “blinds spots” becomes critical.

The projection of high frequency patterns is required to meet inspection accuracy requirements for small components and features such as solder joints. However, the high frequency patterns go out of focus quickly and are completely blurred out on tall components. So several additional, coarser frequency patterns are frequently projected to measure the wide range of feature sizes.

Why do reflective surfaces pose a challenge when inspecting SMT assemblies?

Highly reflective surfaces present several challenges for SMT inspection. For example, the mirror-like surfaces of solder joints can generate glints which in turn cause camera saturation. Other mirror-like surfaces may direct light predominantly away from the camera view which in turn produces an extremely poor signal-to-noise ratio. Highly reflective surfaces also increase the severity and probability of multiple reflections. An example is shown in Figure 4.

Figure 4. Multiple reflections between highly reflective solder joints

Could you explain how sensor architecture is crucial in overcoming inspection challenges related to multiple reflections?

Multiple reflections disturb the fringe patterns and cause erroneous height measurements unless their effects can be suppressed. The fringe pattern observed at measurement point A, for example, shown in Figure 5 receives direct illumination from the projected blue pattern and indirect illumination from light scattered at point B. The indirect illumination arriving at point A disturbs the fringe pattern image.

Figure 5. Multiple reflections

A 3D phase shift profilometry system is able to suppress potential height measurement errors induced by multiple reflections by careful analysis of the image data sets from multiple cameras and multiple fringe pattern frequencies and directions.

Both low and high frequency patterns are required for tall components and connectors (e.g. 25 mm tall) in order to unwrap the fringe pattern phase and determine absolute height. However, the multi-path reflections from lower frequency patterns can severely corrupt the reported phase as shown in Figure 6.

Here, a multiple reflection adds coherently to the direct reflection. The potential height error is shown and is influenced by the relative strengths of the direct reflection and the multi-path reflection. These effects are also more severe for closely spaced components and highly reflective features.

Figure 6. Multi-path reflections of a lower frequency pattern

In general, data from locations of high fringe contrast are most reliable when there are no multi-path reflections. However, multi-path effects from lower frequency patterns also have high fringe contrast, but can severely corrupt the reported phase as seen in Figure 6.

Multi-path reflections of a high frequency fringe pattern is shown in Figure 7. The multiple reflections from many locations add incoherently to reduce the fringe contrast, but disturb the phase very little.

Figure 7. Multi-path reflections of a higher frequency pattern

Since each oblique view camera will be affected by a multiple reflection differently, the individually reported heights from different oblique cameras will differ for the location of a multiple reflection. This fact can be used during analysis to identify areas of multiple reflections.

To suppress the effects of multiple reflections, a rich data set of the fringe contrast, the effective surface reflectance, and calculated phase are analyzed for each frequency pattern from each of the oblique view cameras. This data is combined, or fused together, to suppress the disturbances of severe multi-path reflections.

How does parallel 3D sensing differ from single 3D sensing? How can parallel 3D sensing be used to increase image acquisition speed?

The two factors that typically limit inspection speed with the 3D phase shift profilometry method are image acquisition time and the amount of time to process the acquired images. For example, one typical 3D phase shift profilometry architecture includes a single camera and multiple projectors to address shadows from tall components.

A common approach with this architecture is to sequentially acquire 32 or more images of different fringe patterns with multiple frequencies and from four projection directions.

To address the requirement for improved acquisition speed, the 3D phase shift profilometry system can be architected with a single fringe projector and multiple oblique viewing cameras (as shown in Figure 8) enabling image acquisition of a particular fringe pattern simultaneously. This results in a degree of parallelization in the image acquisition with a typical 4X reduction in image acquisition time.

Figure 8. Sensor architecture

Also, the entire 3D field of view is completely illuminated by the vertical projector with no areas of shadow.The projected fringes are then imaged simultaneously by the oblique view cameras as illustrated in Figure 9.

Figure 9. Vertical projection with oblique view cameras

Although there is a tremendous speed advantage to the multiple camera architecture, it can be seen from Figure 9 that the X,Y coordinates for a given camera pixel are a function of the Z height.

To overcome this challenge, the 3D phase shift profilometry system must accurately calibrate the X,Y locations for every pixel in every camera throughout the entire Z measurement range. This essentially amounts to calibrating the ray slope for every camera pixel. If the calibration is performed accurately, then the X, Y locations may be unambiguously decoded to micron level accuracies.

Could you explain the basic concept behind how Principal Component Analysis works? Why is this important for the accurate modelling of height information?

Download the CyberOptics white paper on automated high-speed 3D optical inspection.

Complex, three-dimensional shapes are difficult to describe with geometric models as the number of parameters tends to expand exponentially as complexity grows. Furthermore, in order to properly inspect complex shapes using a parametric model requires assigning nominal values and tolerances to each of these many parameters which quickly becomes impractical due to the large number of parameters.

More appropriate techniques exist to model complex shapes such as Principal Component Analysis (PCA) which is a mathematical technique used to provide a compact representation of a set of observations. Using a small training set of example three-dimensional images of a complex shape, Principal Component Analysis will automatically discover and model the important attributes of the complex shape as well as the allowable tolerances of those attributes.

A more popular application for PCA is facial recognition software, which automatically learns key attributes of a face and can “recognize” the face even when making different expressions. The PCA shown in Figure 10 is modelling the relationship between the glabella (flat area between eyebrows) with nose length. For this particular face, exceeding the norms of this relationship would fail the recognition.

Figure 10. PCA of a face

Principal Component Analysis can also be applied to height images from a 3D AOI inspection system to “learn” the 3D shapes of the various features of an electronic assembly such as package bodies, leads, polarity dimples, and solder joint fillets.

A PCA model can then be applied to each feature during inspection to validate the 3D shape of that feature and flag the feature when it does not match the allowable tolerances established by the PCA model.

This model training occurs without explicitly specifying these parameters. The ability to characterize a complex shaped feature with a minimum amount of programming is critical to reducing the amount of overhead to operate the 3D AOI inspection system.

Where can our readers find out more information about high precision sensing technology solutions from CyberOptics?

CyberOptics high precision accuracy at production speed is a combination of architecturally superior multi-view 3D sensor, proprietary Multi-Reflection Suppression (MRS) technology and AI² modeling technology. Watch the video to know more about our 3D AOI solution.

Information about our full range of inspection solutions are available on the CyberOptics website – www.cyberoptics.com.

We also encourage readers to sign up for our monthly newsletter, INtroSPECT, to receive latest news about our products and upcoming events.

About Tim Skunes

Tim Skunes has been the Vice President of Technology and Business Development at CyberOptics Corporation since May 2010, and has been at the company since 1987.

He has an M.EE in Optics and Electrical Engineering from the University of Minnesota, and holds 26 US patents for optical measurement systems, optical manufacturing, and fiber optic devices.