Building better cameras

Bad news: that $3,000 camera you just bought isn’t perfect. But it’s not just your camera. No camera is really perfect.

As every good mechanical engineer will tell you, tolerances and variation in manufacturing mean that the sensor, (that little square of photon-capturing silicon,) is not going to sit exactly on the optical axis of the lens, and neither will it be perfectly perpendicular.

Depending on just how the sensor is mounted, this means that the image will be slightly out of focus in some areas. Most likely, I’m surmising, across diagonal corners. In most machine vision applications this might not matter, but I suspect that as resolutions increase and pixels get smaller it will become more of an issue.

Good news: Kasalis, who build machines that mount CCD and CMOS sensors, are working on ways to improve sensor positioning. I learnt this from “Adaptive software eases camera lens-to-sensor alignment” published in Laser Focus World, March 2013, but you can find out more by visiting the “Active Alignment” page on the Kasalis website.

If you’re wondering why you should be interested, let me briefly explain. If you use cameras you should (a) know how they work, and (b) understand what differentiates the inferior from the superior. Clearly, one such factor will be the precision of the sensor alignment, for which we will no doubt be charged a premium.

Noise and CMOS sensors

“Old school” machine vision people are often dismissive of cameras with CMOS sensors. “Too noisy,” they say, and indeed that was once true. But “CMOS Sensors Increase Inspection Speed and Accuracy,” published in the December 2012 Photonics Spectra sets out to explain why the future will be CMOS.

Read the article for details, but the bottom line is this: CMOS technology has been steadily improving, to the point where it looks like it will be used in all new machine vision cameras. The Photonics Specttra article also includes a handy little table summarizing “The merits of CMOS sensors, at a glance.”

Well worth a look.

But, top of the list of merits is “Good full well capacity”. Now I ask you, is that always a desirable characteristic?

If you plow through the very technical “Balancing sensor parameters optimizes imaging device performance,” published in Laser Focus World, December 1^st, 2012, you’ll gain a better appreciation of the complexities of sensor noise. For there is not one source, but several. And driving down one tends to increase the others.

It all boils down to what you want the sensor to do. Scientific, low-light applications place very different demands on the sensor than do most machine vision applications where you can flood the target with photons.

And the takeaway for us machine vision craftsmen? It’s this: noise is a complicated issue, but it pays to get a better appreciation of the nature of the sources. That way, you’ll know which camera parameters matter most to your application. And yes, a CMOS sensor may be in your future.

 

Camera pricing

Point Grey sent me an email – you probably got it too – saying that the latest additions to their Flea series are priced at $995.

I haven’t purchased a machine camera for a while, but that strikes me as a competitive price for a 2.8Mp GigeE unit with a Sony CCD sensor. Time was I used to calculate the price per megapixel – call it the Grey Index – and just last year I was paying $500 per Mp. This camera, the FL3-GE-28S4C or S4M if you want the monochrome version, works out at $335 per Mp.

Cameras keep getting cheaper.

Is CMOS as good as CCD?

Camera purists will tell you that the image from a CMOS sensor is inferior to that from a CCD. I’m not going to argue, because I suspect that, in absolute terms, it’s correct. However, I’ll take a different approach and suggest that the difference is so small it doesn’t matter. And if it does matter, compensation can be applied right in the CMOS camera.

A good place to learn about the differences and possible methods of compensation is on the Adimec blog, specifically, “CCD vs. CMOS: Image Artifacts to Consider with CMOS Image Sensors” posted August 24^th, 2012. (Be sure to click the links provided.)

And why doesn’t the inferiority of CMOS matter? Because you should be engineering your systems with plenty of robustness. If a few random defective pixels are going to alter the result I would suggest you’re right on the edge rather than in the stable, predictable zone. Apply a filter to smooth out the noise, optimize your lighting; those are the kinds of things you could do.

And I think you should want to, because CMOS has two big advantages: lower price and higher frame rate. But if you really need the best, go with CCD.

CMOS sensor machine vision sales growing or declining?

The machine vision blogosphere has lit up with debate over the trend in CMOS-based camera sales. Well not exactly lit up, but there’s been some discussion.

Machine vision people tend not to like CMOS sensor technology. They perceive it as producing inferior quality images, mainly as a result of higher noise levels. I would say though that in my experience CMOS sensors work well in many applications – if yours is that noise sensitive perhaps there are other problems to address.

However, for years there have been predictions that sales of CMOS-based cameras would grow, taking share from CCD’s. Yet as reported on the Image Sensors World blog, the AIA’s recently published camera study shows the opposite. This shows the share taken by CMOS as declining from around 26% to 21% over the last 6 years.

Writing in the Adimec blog, Gretchen Alper postulates that the decline in CMOS share is because the early adopters were disappointed and gave up with the technology. She then suggests that as expectations become more realistic, market share growth will pick up.

I’d like to offer two different perspectives. The first is that it’s market share we’re talking about, not actual units. Thus, since the market is growing it’s probable that CMOS unit sales are up, but just not as much as those of CCD sales. Perhaps another conclusion to be drawn from this is that buyers are gravitating towards higher-end cameras.

Second, I’d be curious as to what was included in the AIA’s definition of “camera”. How about all those datamatrix code readers for instance? Were they counted? I suspect a look at the source data might clear some of the fog.

Unfortunately, my pockets aren’t deep enough to spring for the survey, so I suppose I will have to remain forever ignorant. But like all good bloggers, that won’t stop me from having an opinion!

Beer snobs and camera snobs

Some beer drinkers are very particular. Their beverage must have been brewed in exactly the right way, with the appropriate hops content, and be served at exactly the right temperature. Drinkers like these refuse to accept that something poured from the bottle can ever be as good as something hand-pumped from a cellar.

Some machine vision professionals get the same over cameras. For them a CMOS sensor can never be as good as a CCD.

They may be right – I’m no physicist – but I believe the difference is narrowing to the point where it really doesn’t matter any more. And what leads me to this conclusion?

Two recent articles on the state-of-play in CMOS sensor design. First up is “CMOS Imaging Technology Advances” by Eric Fox of Teledyne Dalsa and published in the April 2012 Quality Magazine. And second, a white paper by sensor manufacturer Aptina: “Global Shutter Pixel Technologies and CMOS Image

Sensors – A Powerful Combination” (it opens up as a pdf.)

Between them, these two articles provide some good detail on what’s being done to address the perceived weaknesses of CMOS sensors, and also point out some of the advantages.

There are times when a bottled beer is to be preferred over draught, and the same goes for camera sensors. Read the articles and keep an open mind.

CMOS sensors versus CCD’s

The question of whether CMOS-based cameras are suitable for machine vision is one of those debates that can arouse a surprising amount of passion. I’m somewhat agnostic since I’ve found modern CMOS cameras to be good enough. There aren’t many applications where a little noise makes the difference between success and failure, and if you are operating on a knife edge perhaps this means you just haven’t engineered a robust application.

But opinions aside, there’s not much question that CCDs still provide lower noise than do CMOS sensors, and that’s a pity because CMOS cameras can run at much higher frame rates. So, you might be asking, is it possible to get the best of both worlds?

Well according to “CCDs lose ground to new CMOS sensors,” (Laser Focus World, March 1st, 2011,) the gap is certainly closing. This fascinating article, written by a pair of camera gurus from Andor Technology, describes a new “scientific” CMOS sensor that provides both high frame rate and low noise. Currently it’s aimed at molecular biologists rather than machine vision folks, but if it works as well as they claim I would expect to see other camera companies start to pick it up.

Does sensor technology matter?

Many of the new cameras coming on to the market, and especially those with high resolutions, like the 10Mp UI-1495LE from IDS, make use of CMOS rather than CCD sensors. In addition to their pixel count, these cameras have several things going for them. They tend to be less expensive than CCD-based cameras, (I think the IDS camera referenced above sells for under $1,000,) and they generally have a higher dynamic range than their CCD equivalents. (This makes it possible to get good images in situations with extremes of contrast.) In addition, CMOS sensors can be coerced into delivering a high frame rate simply by reducing the image size. But they also have at least one drawback. Prudent engineers should be sure to understand this before using CMOS-based cameras in their machine vision applications.

The issue is the rolling shutter. Now not all CMOS cameras use rolling shutter technology, but it’s important to check the specs for this, especially if you intend using the camera in an application where the target is in motion. The reason for this is that the rolling shutter exposes pixels row by row. That means the image of an object that moves past the camera horizontally during the exposure period will be distorted. (There will be distortion if the part moves vertically too, but it won’t be so evident.)

The preferred alternative would be to look for a sensor with a global shutter. In a global shutter all the pixels are exposed simultaneously, so there’s no distortion of the image.

If you’d like to see how these two exposure techniques differ in practice, hope on over to “Sensor Artifacts and CMOS Rolling Shutter” by Barry Green and published on the dvxuser website. There you’ll find some great animations that really illustrate the difference.

While you’re there, make sure to read about the other sensor artifacts, smear, wobble and partial exposure because they can all play havoc with your vision application too.

Lastly, no discussion of CMOS versus CCD sensors would be complete without talking about image quality. The conventional wisdom is that CMOS sensors produce lower quality images than do CCD’s. My view is that while this may have been true in the past, I’m not sure it’s really an issue any more. Now I say this, not by analyzing performance specs but based on what my eyes tell me. I recently purchased a beautiful Nikon camera for home use, and it came with a CMOS sensor. Admittedly, it’s a large format sensor, but it produces gorgeous images. (Notice how I’m crediting the hardware and not the photographer?!) So my feeling is that if CMOS is good enough for the “pro-sumer” camera buyer, then it’s probably good enough for 95% of machine vision applications.

So does sensor technology matter? Yes, but because of artifacts like smear, wobble and partial exposure rather than “image quality.” As always, caveat emptor.

Learning about cameras

Most users of machine vision don’t care too much about how a camera works, just so long as it works, and does so day-in, day-out, without fail, so classes on CCD and CMOS camera technology may not be of much interest. But those of my readers working at the sharp end of camera design and development may want to follow these links.

Fundamentals of CCD and CMOS Imagers and Camera Systems

Applications, Design, and Testing of CMOS and CCD Sensors and Camera Systems

These classes, each of two days, are being put on by the UCLA Extension program, and take place over the period February 28th to March 3rd 2011 in Los Angeles, California. I’ve found it difficult to track down programs that give more than a superficial overview of aspects of imaging technology, so if you could benefit from a more thorough understanding, talk to your boss about these classes.

Don’t forget, the weather’s pretty nice in LA at the end of February too, certainly sunnier than northern Europe or the frozen Midwest, so send me a postcard!

CMOS or CCD?

My colleagues, being a conservative bunch, are adamant that CCD sensors are far superior to CMOS. I however try to keep an open mind. I think CMOS-based machine vision cameras have a few things in their favor, like high speed, dynamic range and resistance to blooming. However, I am forced to admit that the rolling shutter can be a significant drawback.

(A rolling shutter exposes a single line of pixels at a time, which mean that when acquiring an image of something in motion it will appear to slant. The alternative, a global shutter, exposes all the pixels simultaneously.)

So all this means that I was interested in a new product announcement from camera-maker IDS Imaging. They’ve recently unveiled a line of cameras that incorporate a new CMOS sensor from e2v. This, it is claimed, gives the best of both CCD and CMOS sensors. Hopefully that includes lower cost!

Maximizing machine vision camera performance

As an engineer I’ve always found that if I understand how something works I can usually squeeze a little extra performance from it. I’m not saying that I reengineer my car or PC, I just mean that by knowing how it functions I can get the best from it.

So if you’re working with machine vision cameras I would encourage you to get a grasp of how imagining sensors – CMOS and CCD – do their job. A good place to start would be “CCD and CMOS sensors become more finely tuned” by Ann R Thryft and published on the website of Test & Measurement World December 1st, 2009. This isn’t about the physics of photon capture but will will help you follow what the chip actually does, as well as providing s sense of where the technology is headed.

Learn a little about how the sensors work and I guarantee you’ll be able to get a little more speed, or a better image, from your camera.

Peering in to my crystal ball

Here’s a glimpse of what Gypsy Rose Brian foresees for 2010…

Whether for defect detection, gauging, part identification or 3D accuracy, buyers want higher resolution, but they also want reasonable frame rates. This creates a problem. Using CCD sensors, which most camera buyers prefer for image quality, there are some constraints on how fast the data can be shuffled off the silicon. However, CMOS sensors don’t have quite the same limitations.

This is why I’ll go out on a limb with my first prediction and forecast that CMOS-based cameras will see more growth in the higher resolution sensor segment of the market, (say 2 to 11 Mp) than will CCD-based cameras. Here’s my reasoning: frame rates of 3 to 5 fps are unacceptable in many applications, but we know we can achieve higher speeds by windowing or binning the image. CMOS sensors provide more opportunities to extract data from just a select region of the image, with commensurate speed increases. Thus the only way we users are going to get both the resolution and speed our applications demand is by switching to CMOS.

Small fish in a big pond?

The big Vision 2009 show has prompted many companies to email me reminders of their product offerings. I’m not complaining; machine vision is a highly fragmented industry and it’s good to have my memory jogged from time to time.

One such jog came from Matrix Vision. They’re a German manufacturer of USB and GigE cameras, as well as framegrabbers and a smart camera, (the Linux-based mvBlueLYNX.) Not a lot of vision hardware companies use USB, but for some people, particularly university researchers, it has advantages over FireWire. Matrix Vision list a wide range of both CCD and CMOS cameras on their site, so if you need to put together a simple lab system in a hurry, they might be worth talking to.

They’re not a huge company, but they’re trying to carve out a niche, so take a look.

Pixel pitch and pixel spacing

In “MTF and high resolution sensors” I discussed the impact of pixel pitch on resolution. Having received a question on this, I’d like to explain briefly why its pixel pitch that matters and not pixel size (although they are of course related,) and tell you where to find the information.

The size of a pixel gives you an indication of its light-receiving area. However, in discussing resolution it’s the spacing between the pixels that matters. The equation is:

Fmax-practical = 1/(4 x pixel pitch)

Where Fmax-practical is the maximum line-pair frequency that the sensor can resolve.

I gave the example of the 5Mp Sony ICX625 CCD which has pixels of 3.45 microns. However, the pixel pitch is around 4 microns. Why is the pitch greater than the pixel size? Well not all of the silicon collects light. In effect, there’s a border around each pixel, or to put it another way, there’s a space between neighboring pixels.

To determine the pitch you need the dimensions of the active region of the CCD, and for that you need to sensor’s spec sheet. Strangely, I couldn’t find it on Sony’s web site – they seem to want to keep it secret – but it's out there if you look hard enough. Once you've tracked it down you'll learn that the ICX625 measures 9.93 mm in the horizontal direction, and there are 2448 pixels in that length, which is how we get the pixel pitch of 4 microns. As a sanity check, you could do the following: given that the sensor is a 2/3” format with a 4:3 aspect ratio and a diagonal of 11.016mm, work out the length of the sides and calculate the pitch from there.

Hope that makes things clear. And by the way, I do welcome comments, so if you’ve got questions, complaints or anything you want to share, please make use of the “Comment” function.

Multispectral camera

Yesterday I mentioned that Key Technology build their own 3 CCD camera for imaging in the UV, visible, and IR parts of the spectrum. Today I discover that Fluxdata, out of Rochester, New York, offer such a thing.

This looks to be a rather high-end imaging device, so you’d need a very solid reason for not using three separate cameras. In fact I’m rather puzzled as to the market for a 3 CCD camera. Can anyone provide a little enlightenment?

More pixels are on the way!

I picked up an announcement from Kodak regarding a “new” 8.1 megapixel CCD sensor. (The KAI-08050 – datasheet opens as pdf.) With an array of 3296 x 2472 pixels, and a claimed speed of up to 16fps, it will fill the gap between widely available 5 Mp cameras and the hugely expensive 11 and 16 Mp units. The downside is that a pixel size of 5.5 microns makes this a pretty large slice of silicon: with an optical format of 4/3” I don’t expect to see it in too many ‘C-mount’ format camera bodies.

I anticipate that the first machine vision cameras with this sensor will appear before the end of 2009. They’ll use the CameraLink standard and will probably take an ‘F-mount’ lens. Expect pricing to be in the region of $9,000.

And yes, there is a color version.

What’s with all the red lights?

Have you ever wondered why most modern machine vision systems use red light?

Yes, manufacturing engineers love the soft, romantic glow that comes from rows of red LEDS, but there’s more than aesthetics involved. I know of two reasons for ‘going red’:

Red LEDs are the least expensive, (I’m guessing this is driven by volume,) and no one likes to spend more than they have to.
In the early days of silicone-based CCD and CMOS sensors, or so I’m told, the material was most sensitive to photons with energies in the red part of the spectrum. So it made sense to match the wavelength of the light to the peak sensitivity of the detector.

But things change. CCD and CMOS sensor manufacturing is driven by consumer electronics, and consumers want digital cameras to produce images that agree closely with their eyes. So as the human eye has its’ peak sensitivity in the green part of the spectrum, (to better see the predator lurking in the bushes?) camera sensors are now engineered with the same response.

What does this mean for the Joe the machine vision engineer? Well, he should check the spectral response graph for the sensor in the camera(s) he’s going to use – most manufacturers put this on their web site – and consider using light of a wavelength the camera is best able to detect. (It’s a question of efficiency really – aim to ‘harvest’ the greatest amount of energy from the input.)

What this suggests, if you haven’t beaten me to the punchline, is that it may make sense to use green light rather than red.

Is 5 Megapixels the new black?

Two or three years ago everybody was bringing out a 2 Mp camera based on the Sony 1 1/8 chip (1600 x 1200 pixels.) When incorporated into a Firewire A camera, it’s a good combination of resolution, framerate and price.

But machine vision people always want more resolution, so the camera vendors are responding with 5 Mp products. In fairness, CMOS-based cameras of 5 Mp and more have been around a while, but now we’re starting to see CCD cameras with this resolution.

Eager to be in the vanguard of this latest fashion, Basler and JAI each have new products based around the Sony ICX625 2/3” sensor. Both are available in GigE and CameraLink flavors, and deliver 15fps of 2,448 x 2.050 pixel resolution. No word on pricing, but I’m predicting a list of $2,999.

Allied Vision Tech (AVT) has also just announced 5 Mp versions of their diminutive Guppy and the Stingray. The Guppy (not available until January 2009) is CMOS-based and will have a Firewire A interface, while the Stingray will be a B.

Read about the JAI camera here, the Basler offering here, and use this link for the AVT cameras.

Machine vision primer

For anyone new to machine vision, this is a handy little introduction to the subject. Beware the popup and popunder ads though. They’ll drive you mad!

But if you can put up with them, you’ll find the ‘seminar4u’ blog to be a treasure trove of info on subjects ranging from a comparison of CCD and CMOS image sensors to an intriguing set of internal combustion engine-related videos.

Most bloggers have some kind of theme, but ‘seminars4u’ just has an eclectic collection of ‘stuff’

Have fun browsing!

CMOS vs. CCD (yet again)

CMOS sensors are cheap, have great dynamic range, and will let you ‘window’ to increase frame rate. CCD sensors are less noisy and produce a higher quality image.

Need we say more?

Well Toshiba Teli have something to say on the subject. Their new 12 megapixel camera (link courtesy of Test & Measurement World,) utilizes a CMOS sensor to deliver high frame rates. 12 Mpixel images at 25fps is very impressive. No word on pricing, but I’m estimating something in the $5,000 - $7,000 range.