A new price point for machine vision cameras

Every time a new camera is announced I calculate it’s price per megapixel. As you might expect, the trend has been generally downwards, and when my data set is sufficiently rich I will share the results with you. Today though I’ve plotted a new low on my chart.

The Imaging Source have on their website a 13 Megapixel CMOS USB3 camera priced at $599.I have to wonder if there’s a typo there somewhere because at $46/Mp that seems extraordinarily cheap. Now it is a rolling shutter and it comes with a built-in auto focus lens, but even so, that’s a lot of pixels for the dollar.

Resolution up, prices down

It’s a familiar trend but the SP-20000 “Spark Series” from JAI adds another data point. This is a 20Mp camera (monochrome or color,) that spits out 30 full frames per second (each measuring 5,120 by 3,840,) and costs under $9,000. Yes it’s CMOS, which I know upsets some traditionalists, but the pixels are 6.4 microns and it takes an F-mount lens, so it’s photo-capturing credentials are good.

If you take the time to plow through the spec sheet you’ll see this is a pretty interesting camera. My interest of course is industrial inspection, but I can see this being used in many other applications, like surveillance and aerial mapping. Just not traffic.

Renewed R&D effort at Dalsa?

Now that camera-maker Dalsa is part of Teledyne it’s hard to get a clear view of what’s happening there, but the recently published Q2 2013 results and associated conference call offered a few clues. (www.seekingalpha.com is a great resource if you’re interested in conference call transcripts.)

The results showed that while revenues in the Digital Imaging group were off 6% in the second quarter income was up 5.3%. That says to me there’s been some cost-cutting going on, and indeed, as much was admitted during the conference call. It was however also mentioned that industrial machine vision sales were up slightly so the shortfall is elsewhere.

Also in the conference call was some discussion of Teledyne’s acquisition of a company called Axiom IC. Axiom designs CMOS chips, and I’d like to share with you a couple of quotes from the conference call.

First off, CEO Robert Mehrabian said, “This quarter, we made a small but important acquisition for Teledyne DALSA. We acquired Axiom IC, a designer of high-performance CMOS mixed signal integrated circuits. The technologies at Axiom will help us continue developing highly differentiated CMOS imaging sensors and cameras for our machine vision market.”

Then, answering a question about the integration of Axiom, he commented, “…right now, our focus is to get our CMOS imaging development programs improved.”

What can we take away from this?

First, it says to me that Teledyne is serious about investing in the Dalsa camera business. And second, perhaps Teledyne’s management thought Dalsa’s R&D effort needed a bit of a kick in the pants.

And the implications for machine vision cameras?

I think Teledyne and Dalsa are seeing opportunities for differentiated, and presumably higher performance imaging sensors. My guess is that will mean higher frame rates, lower noise levels, and perhaps more development of TDI technology. All this has to be good for us end-users, (and perhaps for Basler too, since there might be less overlap with their business.) The only downside is that these new products will likely command premium prices, but then you get what you pay for.

CMOS and GigE

I’ve been reading about Basler’s racer linescan cameras. They’re pretty interesting. I hadn’t appreciated that they are both CMOS-based and GigeE. That’s a combination that yields some interesting benefits: less heat generated and no high-end framegrabber or expensive CameraLink cable required.

And where did I learn this? From “The Benefits of Modern CMOS Sensors in Industrial Line Scan Cameras”, available as a pdf download from Basler’s Document Downloads page. (Name and email required, but it is worth it.)

Now obviously, Basler’s White Paper isn’t exactly impartial, but it is well-written and filled with technical “meat”, so I suggest you take a look.

Let’s hear it for CMOS sensors

I’ve written extensively on the CCD versus CMOS sensor issue, (“Why CCD sensors will become obsolete” will get you there, or just search for posts labeled “CMOS”) but I’ve recently learned of another CMOS advantage.

CMOS sensors, by virtue of their architecture, consume less power than equivalent CCD’s. Less power means less heat, and if you’ve been following along, (“Dealing with camera noise”) you’ll know that noise is bad.

So in principle, and I’ve yet to actually test this hypothesis, all other things being equal, images from a CMOS sensor will exhibit less variation in grayscale values than images from a CCD. I wonder if that means CMOS sensors perform better in high temperature environments?

Would anyone care to comment?

Building better cameras

Bad news: that $3,000 camera you just bought isn’t perfect. But it’s not just your camera. No camera is really perfect.

As every good mechanical engineer will tell you, tolerances and variation in manufacturing mean that the sensor, (that little square of photon-capturing silicon,) is not going to sit exactly on the optical axis of the lens, and neither will it be perfectly perpendicular.

Depending on just how the sensor is mounted, this means that the image will be slightly out of focus in some areas. Most likely, I’m surmising, across diagonal corners. In most machine vision applications this might not matter, but I suspect that as resolutions increase and pixels get smaller it will become more of an issue.

Good news: Kasalis, who build machines that mount CCD and CMOS sensors, are working on ways to improve sensor positioning. I learnt this from “Adaptive software eases camera lens-to-sensor alignment” published in Laser Focus World, March 2013, but you can find out more by visiting the “Active Alignment” page on the Kasalis website.

If you’re wondering why you should be interested, let me briefly explain. If you use cameras you should (a) know how they work, and (b) understand what differentiates the inferior from the superior. Clearly, one such factor will be the precision of the sensor alignment, for which we will no doubt be charged a premium.

Why CCD sensors will become obsolete

It seems the massively-talented Andy Wilson and I are thinking along the same lines when it comes to the future of camera technology. While I was scribbling “Noise and CMOS sensors” and “Exploring the CCD vs. CMOS issue” he was hard at work on the extended-length “Machine Vision's Future Centers on CMOS and Consumer Advances” (Vision Systems Design, December 1^st, 2012.)

Andy’s article is interesting because he explores the drivers behind the growth of CMOS sensor technology, and because he talks about where it’s going. And where it’s going is described, to an extent anyway, in the complimentary VSD article, “Low-cost embedded devices boost end-user applications.” (Also Dec 1^st ’12.)

One big takeaway from these is that automotive applications are rapidly becoming the driver. Vehicle manufacturers want to put ever-more intelligence into our cars, (probably because we’ve demonstrated that we’re too dumb to handle them safely.) What’s more, they will offer huge volumes to those who can drive the cost down.

What does this mean for the developers and users of industrial machine vision? If want to take advantage of these economies of scale I suggest we look for ways to subvert blind spot warning, collision detection, and other such technologies, to our service. Otherwise we’ll be left playing with the scraps from the big boys table.

Exploring the CCD vs. CMOS issue

If this is a subject of interest, and judging by the traffic I saw for “Noise and CMOS sensors”, (January 10^th, 2013,) it is, you might want to look at the Adimec blog.

These makers of high-end cameras have devoted considerable time and effort to debating both the merits of the respective image sensor technologies and future trends. The posting, “CCD vs. CMOS Image Sensors for Machine Vision, Defense, and Traffic Applications”, (which dates from May 3^rd, 2012,) references some of their many articles. You might want to take a look.

Noise and CMOS sensors

“Old school” machine vision people are often dismissive of cameras with CMOS sensors. “Too noisy,” they say, and indeed that was once true. But “CMOS Sensors Increase Inspection Speed and Accuracy,” published in the December 2012 Photonics Spectra sets out to explain why the future will be CMOS.

Read the article for details, but the bottom line is this: CMOS technology has been steadily improving, to the point where it looks like it will be used in all new machine vision cameras. The Photonics Specttra article also includes a handy little table summarizing “The merits of CMOS sensors, at a glance.”

Well worth a look.

But, top of the list of merits is “Good full well capacity”. Now I ask you, is that always a desirable characteristic?

If you plow through the very technical “Balancing sensor parameters optimizes imaging device performance,” published in Laser Focus World, December 1^st, 2012, you’ll gain a better appreciation of the complexities of sensor noise. For there is not one source, but several. And driving down one tends to increase the others.

It all boils down to what you want the sensor to do. Scientific, low-light applications place very different demands on the sensor than do most machine vision applications where you can flood the target with photons.

And the takeaway for us machine vision craftsmen? It’s this: noise is a complicated issue, but it pays to get a better appreciation of the nature of the sources. That way, you’ll know which camera parameters matter most to your application. And yes, a CMOS sensor may be in your future.

 

Is CMOS as good as CCD?

Camera purists will tell you that the image from a CMOS sensor is inferior to that from a CCD. I’m not going to argue, because I suspect that, in absolute terms, it’s correct. However, I’ll take a different approach and suggest that the difference is so small it doesn’t matter. And if it does matter, compensation can be applied right in the CMOS camera.

A good place to learn about the differences and possible methods of compensation is on the Adimec blog, specifically, “CCD vs. CMOS: Image Artifacts to Consider with CMOS Image Sensors” posted August 24^th, 2012. (Be sure to click the links provided.)

And why doesn’t the inferiority of CMOS matter? Because you should be engineering your systems with plenty of robustness. If a few random defective pixels are going to alter the result I would suggest you’re right on the edge rather than in the stable, predictable zone. Apply a filter to smooth out the noise, optimize your lighting; those are the kinds of things you could do.

And I think you should want to, because CMOS has two big advantages: lower price and higher frame rate. But if you really need the best, go with CCD.

Interesting new camera

Point Grey has just announced an 8.8 Mp version of their compact Flea 3 USB 3.0 camera. Inevitably, packing an array of 4096 by 2160 pixels into a C-mount format sensor means the pixels are pretty darned small, and that’s where this is interesting.

This hi-res Flea 3 is using a Sony IMX121 color sensor with pixels just 1.75 microns in size. On learning that, my first thought is “poor sensitivity and high noise,” but that might not be the case. Sony are making a big deal of their “Exmor R” CMOS sensor technology that effectively switches the positions of the internal wiring and the photon-capturing silicon. (Click the link to see a great animation of how it works.)

This, it is claimed, increases sensitivity by 6db and reduces noise by 2db. Those might not seem big numbers but remember that it’s all logarithmic. Sony has one comparison image posted on their site; it would be interesting to see a few more.

Two last points to mention: the camera puts out 21fps at full resolution, which is a pretty impressive number. Good thing USB 3.0 offers 5Gbits per second of bandwidth! And pricing? $945, which seems like a bargain.

CMOS sensor machine vision sales growing or declining?

The machine vision blogosphere has lit up with debate over the trend in CMOS-based camera sales. Well not exactly lit up, but there’s been some discussion.

Machine vision people tend not to like CMOS sensor technology. They perceive it as producing inferior quality images, mainly as a result of higher noise levels. I would say though that in my experience CMOS sensors work well in many applications – if yours is that noise sensitive perhaps there are other problems to address.

However, for years there have been predictions that sales of CMOS-based cameras would grow, taking share from CCD’s. Yet as reported on the Image Sensors World blog, the AIA’s recently published camera study shows the opposite. This shows the share taken by CMOS as declining from around 26% to 21% over the last 6 years.

Writing in the Adimec blog, Gretchen Alper postulates that the decline in CMOS share is because the early adopters were disappointed and gave up with the technology. She then suggests that as expectations become more realistic, market share growth will pick up.

I’d like to offer two different perspectives. The first is that it’s market share we’re talking about, not actual units. Thus, since the market is growing it’s probable that CMOS unit sales are up, but just not as much as those of CCD sales. Perhaps another conclusion to be drawn from this is that buyers are gravitating towards higher-end cameras.

Second, I’d be curious as to what was included in the AIA’s definition of “camera”. How about all those datamatrix code readers for instance? Were they counted? I suspect a look at the source data might clear some of the fog.

Unfortunately, my pockets aren’t deep enough to spring for the survey, so I suppose I will have to remain forever ignorant. But like all good bloggers, that won’t stop me from having an opinion!

Beer snobs and camera snobs

Some beer drinkers are very particular. Their beverage must have been brewed in exactly the right way, with the appropriate hops content, and be served at exactly the right temperature. Drinkers like these refuse to accept that something poured from the bottle can ever be as good as something hand-pumped from a cellar.

Some machine vision professionals get the same over cameras. For them a CMOS sensor can never be as good as a CCD.

They may be right – I’m no physicist – but I believe the difference is narrowing to the point where it really doesn’t matter any more. And what leads me to this conclusion?

Two recent articles on the state-of-play in CMOS sensor design. First up is “CMOS Imaging Technology Advances” by Eric Fox of Teledyne Dalsa and published in the April 2012 Quality Magazine. And second, a white paper by sensor manufacturer Aptina: “Global Shutter Pixel Technologies and CMOS Image

Sensors – A Powerful Combination” (it opens up as a pdf.)

Between them, these two articles provide some good detail on what’s being done to address the perceived weaknesses of CMOS sensors, and also point out some of the advantages.

There are times when a bottled beer is to be preferred over draught, and the same goes for camera sensors. Read the articles and keep an open mind.

A special camera for welding

The idea of using machine vision to monitor and control arc welding is very attractive. If it was possible to see exactly what was going on in the melt pool the process could be automated and variability reduced. The problem though is that the weld arc is by definition, extremely bright. The conventional way of overcoming this is to use a heavy optical density filter, but that means only the arc and not the workpieces is visible.

PhotonFocus have some very high dynamic range cameras, but even they don’t have the range to see everything in the scene, and that’s created an opportunity for a team of entrepreneurial camera specialists.

Massachusetts-based Visible Welding lacks both a sexy name and a slick website, but maybe its better that they let the product do the talking. The product in question is the V2010-UDR Ultra-Dynamic Range Video System. It doesn’t look much but the videos on the website are really most impressive. (I had to download them and then use RealPlayer.)

I should in fairness mention that I only became aware of Visible Welding thanks to a newsletter from PixeLINK. PixeLINK offer a wide range of cameras, generally CMOS-based, so my assumption is that’s what’s at the heart of the V2010-UDR.

Currently the system is intended only for monitoring and review but my feeling is that it’s only a short step to use this to control an automated arc welding cell. If that comes to pass I hope Visible Welding have the intellectual property rights sewn up tight!

Machine vision in automotive safety

When cars and pedestrians meet it’s rarely the pedestrian that gets the better of the collision. This is why Continental has announced a stereo camera system designed to help prevent this type of accident. (“Two Eyes Are Better Than One – The Stereo Camera”)

The system uses two CMOS cameras mounted 20cm (8 inches) apart and facing through the windshield. This separation apparently allows the distance of an object in the 20 to 30 meter range to be determined within 20 to 30cm (that’s 8 to 12 inches.) Stereo vision is becoming quite well known for tasks like robot guidance, so I guess it’s a logical extension to move it into an unconstrained urban environment. What’s not clear to me is how the system works at night. Perhaps there’s a passive IR illumination system built-in? I also wonder how it will deal with rain, frost or snow. In fact, why not just go with a Kinect-style IR pattern projection approach?

So many unanswered questions.

Regular readers might also wonder if this is the same system that Volvo has under development: “Machine vision for pedestrian safety.” At the time though, (February 2011,) we were told that Volvo were using a single camera combined with a radar to detect pedestrians. Perhaps the Volvo system was actually being developed by Continental, and perhaps the engineers decided a stereo approach worked better.

An alternative hypothesis is that Mobileye was involved somewhere along the way. Mobileye is an Israeli company that offers the “C2-270” vision-based collision prevention system to the automotive aftermarket. This is a single camera system (so how does it determine distance?) that, like the Continental product, looks through the windshield, and as with the stereo system, I’m puzzled as to how it will work at night. There again, perhaps the assumption being made is that drivers will just turn on their lights.

Whatever the development path (and wouldn’t that make for an interesting Vision Systems Design article?) one thing is clear: vision technology is coming to our cars and before too long drivers will be obsolete.

CMOS sensors versus CCD’s

The question of whether CMOS-based cameras are suitable for machine vision is one of those debates that can arouse a surprising amount of passion. I’m somewhat agnostic since I’ve found modern CMOS cameras to be good enough. There aren’t many applications where a little noise makes the difference between success and failure, and if you are operating on a knife edge perhaps this means you just haven’t engineered a robust application.

But opinions aside, there’s not much question that CCDs still provide lower noise than do CMOS sensors, and that’s a pity because CMOS cameras can run at much higher frame rates. So, you might be asking, is it possible to get the best of both worlds?

Well according to “CCDs lose ground to new CMOS sensors,” (Laser Focus World, March 1st, 2011,) the gap is certainly closing. This fascinating article, written by a pair of camera gurus from Andor Technology, describes a new “scientific” CMOS sensor that provides both high frame rate and low noise. Currently it’s aimed at molecular biologists rather than machine vision folks, but if it works as well as they claim I would expect to see other camera companies start to pick it up.

Does sensor technology matter?

Many of the new cameras coming on to the market, and especially those with high resolutions, like the 10Mp UI-1495LE from IDS, make use of CMOS rather than CCD sensors. In addition to their pixel count, these cameras have several things going for them. They tend to be less expensive than CCD-based cameras, (I think the IDS camera referenced above sells for under $1,000,) and they generally have a higher dynamic range than their CCD equivalents. (This makes it possible to get good images in situations with extremes of contrast.) In addition, CMOS sensors can be coerced into delivering a high frame rate simply by reducing the image size. But they also have at least one drawback. Prudent engineers should be sure to understand this before using CMOS-based cameras in their machine vision applications.

The issue is the rolling shutter. Now not all CMOS cameras use rolling shutter technology, but it’s important to check the specs for this, especially if you intend using the camera in an application where the target is in motion. The reason for this is that the rolling shutter exposes pixels row by row. That means the image of an object that moves past the camera horizontally during the exposure period will be distorted. (There will be distortion if the part moves vertically too, but it won’t be so evident.)

The preferred alternative would be to look for a sensor with a global shutter. In a global shutter all the pixels are exposed simultaneously, so there’s no distortion of the image.

If you’d like to see how these two exposure techniques differ in practice, hope on over to “Sensor Artifacts and CMOS Rolling Shutter” by Barry Green and published on the dvxuser website. There you’ll find some great animations that really illustrate the difference.

While you’re there, make sure to read about the other sensor artifacts, smear, wobble and partial exposure because they can all play havoc with your vision application too.

Lastly, no discussion of CMOS versus CCD sensors would be complete without talking about image quality. The conventional wisdom is that CMOS sensors produce lower quality images than do CCD’s. My view is that while this may have been true in the past, I’m not sure it’s really an issue any more. Now I say this, not by analyzing performance specs but based on what my eyes tell me. I recently purchased a beautiful Nikon camera for home use, and it came with a CMOS sensor. Admittedly, it’s a large format sensor, but it produces gorgeous images. (Notice how I’m crediting the hardware and not the photographer?!) So my feeling is that if CMOS is good enough for the “pro-sumer” camera buyer, then it’s probably good enough for 95% of machine vision applications.

So does sensor technology matter? Yes, but because of artifacts like smear, wobble and partial exposure rather than “image quality.” As always, caveat emptor.

Learning about cameras

Most users of machine vision don’t care too much about how a camera works, just so long as it works, and does so day-in, day-out, without fail, so classes on CCD and CMOS camera technology may not be of much interest. But those of my readers working at the sharp end of camera design and development may want to follow these links.

Fundamentals of CCD and CMOS Imagers and Camera Systems

Applications, Design, and Testing of CMOS and CCD Sensors and Camera Systems

These classes, each of two days, are being put on by the UCLA Extension program, and take place over the period February 28th to March 3rd 2011 in Los Angeles, California. I’ve found it difficult to track down programs that give more than a superficial overview of aspects of imaging technology, so if you could benefit from a more thorough understanding, talk to your boss about these classes.

Don’t forget, the weather’s pretty nice in LA at the end of February too, certainly sunnier than northern Europe or the frozen Midwest, so send me a postcard!

CMOS or CCD?

My colleagues, being a conservative bunch, are adamant that CCD sensors are far superior to CMOS. I however try to keep an open mind. I think CMOS-based machine vision cameras have a few things in their favor, like high speed, dynamic range and resistance to blooming. However, I am forced to admit that the rolling shutter can be a significant drawback.

(A rolling shutter exposes a single line of pixels at a time, which mean that when acquiring an image of something in motion it will appear to slant. The alternative, a global shutter, exposes all the pixels simultaneously.)

So all this means that I was interested in a new product announcement from camera-maker IDS Imaging. They’ve recently unveiled a line of cameras that incorporate a new CMOS sensor from e2v. This, it is claimed, gives the best of both CCD and CMOS sensors. Hopefully that includes lower cost!

Now that’s high resolution!

Cypress Semiconductor have announced a 25 megapixel CMOS sensor capable of 53 frames per second (“Cypress Introduces Industry’s Highest Throughput Pipelined Global Shutter CMOS Image Sensor.” EDACafé blog, June 3rd, 2010)

How long will it be before someone incorporates it into a machine vision camera? I have a feeling we’ll see a 25Mp camera advertised before year end. It won’t be cheap though. My camera price rule of thumb has long been $1,000 per megapixel, although that does seem to be shifting downwards at the moment. Even so, my guess is that a 25Mp camera will be priced at a whisker under $20,000.

That might not sound like a bargain, but it depends on your application. For traffic and security applications a sensor like this provides terrific ability to zoom in on fine details while simultaneously capturing a large field of view. (Just like the way, in cop shows and movies, the hero will ask the resident nerd to “zoom in – now enhance” to produce a beautifully clear image of a license plate.)

As for my industrial, and price sensitive, applications – I guess I’ll be sticking with sensors of 1600 x 1200 pixel format for now.