The Fast Evolving World of High-Speed CMOS Imaging

High-speed CMOS cameras are used in many applications and take on several common forms depending on their function. High frame-rate machine vision cameras are relatively simple in design and are used to ensure consistent quality in high-speed production lines.

Machine vision cameras are produced in high numbers, which drive both sensor and lens development. The cameras are simple, composed only of optics, sensor, and interface. Image data is delivered from the camera over a standard interface to a frame grabber. Image content is analyzed for decision-making rather than for human slow-motion viewing. Global shutters and short exposures minimize motion blur and distortion to facilitate analysis.

Stand-alone high-speed cameras, such as those manufactured by Fastec, are used for a variety of applications. In industry, they are a vital tool for design, set up, and troubleshooting of all sorts of machinery. They are also widely used in laboratories applications from human kinetics to microfluidics.

High-speed cameras may use industrial, SLR, or cinema lenses, or may be fitted to microscopes or other specialized optics. Some require a PC or tethered processor for control functions, although most have complex embedded systems and can be operated autonomously.

High-speed cameras such as these, commonly produce raw image data that may be transferred as sequences of discreet images or processed and converted to video formats. This is important because precise frame-by-frame analysis is crucial for many applications. While recording, raw images are written to memory buffers within the camera, to be transferred to nonvolatile storage post-capture. Some newer cameras stream raw high-speed image data directly to nonvolatile storage. Advanced triggering functions are critical for capturing fleeting high-speed events and for triggering groups of frame-synchronized cameras for 3D modeling and analysis.

Slow-motion shots are de rigueur in action movies, commercials, and sports broadcasts. Even most of our cell phones can capture high-speed footage with built- in CMOS cameras. Standard digital cinema and broadcast cameras are capable of producing most of the slow-motion shots seen in sports broadcasts and commercials. Special shots requiring higher frame rates utilize variants of high-speed cameras designed for testing and troubleshooting. For more artistic purposes, the video artist often will maximize exposure times to create a smooth video effect.

Consumer digital cameras and cell phone cameras enjoy economy of scale, benefitting from advanced technologies such as back-illuminated, stacked sensors that manage impressive light sensitivity despite their diminutive size. The “slo-mo” output of these cameras is generally highly-processed, interpolated compressed video for viewing on the device and sharing on the internet. Although capable of compelling video, motion distortion caused by rolling shutters make these cameras inappropriate for most troubleshooting and motion analysis applications. Automatic control of focus and exposure, as well as built-in optics, limit their photographic flexibility.

Both images: same lighting with the fan rotating at the same speed.

Image on the left is from a digital camera using "slo-mo" mode. Vertical fan blades exhibit motion distortion. Motion blur also is evident.

Image on the right is from Fastec IL5. Taken at 1100fps with a 30s shutter.

The high-speed world was very different in 2000. Cell phone cameras were primitive. The Vision Research Phantom 4 (1Mp @ 1Kfps), based on a proprietary FillFactory CMOS sensor, was a high-speed digital wonder, but it could not match the speed of rotary prism cameras (>10Kfps on 16mm film) or the image quality of pin-registered cameras that had been in use for decades. In 1988, Photo-Sonics won an Oscar for a 35mm pin-registered film camera that recorded 360fps!

High-speed film cameras were routinely found on military ranges for ballistics testing and at automotive test facilities for crash testing—often bolted onto vehicles smashed into barriers or other cars. But there was pressure for both the military and the auto makers to go digital:

• High-speed film cameras were complicated mechanical devices subject to wear and routinely had to be rebuilt. Camera failure could easily compromise critical tests.
• Film had to be developed off-site and digitized before the footage could be analyzed.
• Tests had to be carefully choreographed since a 400’ reel of film would only yield a second of footage once the camera spooled up to speed.
• Kodak announced it would discontinue the film favored for these cameras.
• The military was pressured to “go digital” for environmental and logistical reasons.
• Rotating prism cameras, much like rolling shutter sensors today, produced images with motion distortion: the top of each frame was exposed earlier in time than the bottom, so an object moving horizontally through the frame would appear to be bending backward.

Through the 1990s, several high-speed digital and analog video cameras were available, including the Kodak 4540, built by Photron (256 x 256 @ 4500fps), and the nac HSV500 and HSV500C3 (500pps to video tape, and a 3-CCD color camera with a great image). The 4540 was great for auto airbag testing and in labs where very high frame rates were needed. The HSV500 family was popular in automotive testing, on military tracking mounts, and in broadcast. Both Photron 4540 and nac HSV500C3 cameras sold for well over $100K.

Less expensive high-speed cameras became available in the late 1990s. The Kodak Motioncorder by Photron and the Redlake Motionscope, both based on overclocked TI CCD sensors, recorded 512 x 480 @ 250fps and 256 x 240 @1Kfps. Selling for $30,000, they were used for production line troubleshooting and university research.

By the year 2000, machine vision was advancing rapidly but high speed was not yet an option. Two things were lacking: high-speed catalog sensors and a suitable high bandwidth standard interface. Plenty of industrial CCD cameras featured excellent image quality, high quality frame grabbers were available, and applications engineers were getting the two to talk to each other. But the end of 2000 saw a big change.
In October 2000, the Camera Link Interface Standard for Digital Cameras and Frame Grabbers was released. Participating companies included Basler, Cognex, Coreco, DALSA, EPIX, Euresys, Matrox, National Instruments, and PULNiX. The interface could be configured according to bandwidth: 255MB/s for “Base”; 510MB/s “Medium”; and 680MB/s “Full.” High-speed, high-resolution imaging to a standard interface was now possible.

In November 2000, Photobit released the MV13 CMOS sensor: “MV” for Machine Vision, “13” for 1.3Mpixel. Because the sensor was fully digital, its analog to digital converters built right on the chip, it was relatively easy to integrate. The sensor output 10-bit 1280 x 1024 images at 500fps and faster frame rates in proportion to vertical resolution, e.g., 1280 x 512 at 1Kfps. The MV13 was quickly integrated into an airborne camera built by DRS, then a PCI-based high-speed camera system marketed by Roper MASD. Before long Fastec, AOS, and IDT, among others, built MV13-based high-speed cameras, which were excellent for many applications from industrial troubleshooting to all sorts of laboratory science.

The data rate of the MV13 at 8 bits was 625MB/s—well within the range of the Camera Link specification. While it did not fit the mold of the CCD sensors commonly used for machine vision, the sensor eventually found its way into several machine vision Camera Link cameras including the Basler A500.

From 2000 onward, the high-end high-speed camera companies, including Vision

Research, Photron, Roper MASD, and nac all competed in the automotive crash-test and military markets, each investing separately in proprietary CMOS sensors. The newest cameras easily surpass the 10Kfps 16mm film cameras in both speed and imaging performance. The fastest can produce over a million frames per second.

In the meantime, the machine vision world has caught up with and passed MV13 performance. Camera Link has added the Deca configuration, 850 MB/s, and other fast interfaces such as CoaXPress and PCIe have emerged. New high-speed and high- resolution sensors from On Semiconductor, Luxima, CMOSIS, FORZA, Sony, and others are now available in machine vision cameras. These same sensors are finding their way into high-speed laboratory and troubleshooting cameras, giving them higher frame rates and resolutions, and opening more applications for high-speed cameras with catalog sensors.

Technologies such as back-illumination and stacked design are just beginning to become available in industrial sensors. As time goes on, machine vision cameras will continue driving demand for high-bandwidth interfaces and higher resolution, higher speed sensors and will increasingly leverage technologies developed for high- production products such as consumer cameras and cell phones.