Why GenICam 3.0 deserves your attention

Some engineers love standards. They get all excited about reading them, and positively orgasmic when offered the chance to be on a standards committee.

Me, I’m not so interested. I get that standards are important, and I understand that my work is much easier as a result of machine vision standards like GenICam, but I can’t get excited about them. However, a press release from the European Machine Vision Association (EMVA) regarding the new GenICam 3.0 was quite interesting. Subtitled, “3D machine vision made easy,” this explains how EMVA has standardized interfaces for 3D cameras in the same way as 2D cameras have been standardized for some years.

This is good news. As vision engineers have been waking up to the possibilities of 3D – and I confess to being a big fan – we need the cameras to be as close to ‘plug-and-play’ as possible.

Also in the new release, some enhanced point-cloud capabilities. Learn more by reading the press release.

Who’s got RealSense?

Raise your hands please if you’ve played with this new Intel camera technology.

Like a Kinect on steroids, the RealSense promises to make 3D imaging easy. Apparently it came out at the beginning of the year, but between doing my day job and dealing with illness it passed me by. It was only Vision Systems Design’s interview with Arnaud Darmont that alerted me to the fact that this is something new.

I may have several applications, but I’d like to learn more about how it works. I can’t find any details on the Intel website, although a post on the Image Sensors World blog, “Intel Releases More Details on its F200 RealSense Camera” suggested it’s an IR light pattern projection system.

However, commenters in a discussion forum argued that it’s actually a Time-of-Flight system. I’m not sure about that, but I can’t find anything definitive. So, can anyone share links to definitive info about how the RealSense F200 camera works?

Summarizing your 3D options

One of my takeaways from Automate was that 3D is no longer a novelty, at least not as regards robot guidance. Pretty much every robot integrator with a Fanuc on their stand also had 3D system in place. These all used the same DLP approach where a series of lines are projected in quick succession onto the target surface.

Andy Wilson summarizes this, and all the other options, in “Choosing a 3D vision system for automated robotics applications” which was published on the Vision Systems Design website back in December 2014. It’s a very comprehensive and I don’t think he omits a single camera-based method. (He doesn’t mention LIDAR or time-of-flight methods.)

If you have any interest in robot guidance I’d encourage you to read his article. Even if you’re more interested in part inspection, as I am, it’s an excellent primer. Ever helpful, Andy includes links to the many camera and illumination companies with something to offer the 3D imaging world. It’s something you might want to bookmark.

Opportunity to do machine vision research

The Bristol Robotics Laboratory, “a collaborative research partnership of the University of the West of England and the University of Bristol” is offering a “Funded full-time PhD studentship in 3D computer vision”.

Details are available at on their “Jobs at BRL” web page.

This is a huge opportunity for a young person who wants to build a name for themselves in the world of 3D machine vision. I don’t know what it would be like getting a visa to study in the UK, but if I was thirty years younger I’d want to give this a go.

I’d add that I’ve been to Bristol, and seems like a pretty decent place to live. Expensive and a little damp, but hey, you’d be in the lab doing exciting things with cameras, robots and software, so what’s the problem?

What 3D machine vision can do for you

Staying with what is becoming a theme, “Dimension Monitoring Savings Across the Board”, (Control design, November 5^th, 2013,) details a 3D application in the lumber industry. Here sawmill operator Canfor used 3D to reduce waste and increase yield.

You can read all the details in the Control design article, so I’m not going to repeat them here. I will however provide a link to the website of the scanner manufacturer, Hermary Opto of British Columbia, Canada.

Hermary Opto looks to be a small company, but one that has found and developed a profitable machine vision niche. The website has a brief primer on scanner technology, and a lot of detail about their products and applications. If you’re wondering what 3D can do for, I encourage you to take a look.

Putting 3D to work

If you read my post “Playing in the 3D space” you might be wondering what 3D can do for you. Well an interesting application story in Quality magazine might prompt some ideas.

“Integrating A 3-D Inspection System” (May 2013,) describes a system that Cyth Systems of San Diego put together. Cyth is a National Instruments Alliance partner, so naturally this uses LabVIEW (not that there's anything wrong with that.)

Take a look, and perhaps you'll be inspired.

Advice for laser triangulation systems

Using a laser line to get height information is now so easy there are smart cameras that can do. (Check out www.Sick.com and even the Cognex DS1000 to see for yourself.) Yes despite the advances, 3D still has some challenges.

Chief among these is calibration. If you’re doing 3D you probably want height information in inches or millimeters, so how do you convert pixels to real-world units?

There’s some good advice on this in “Calibration of 2D Cameras for 3D Inspection” published in Photonics Tech Briefs, May 1^st 2013. And the article also discusses lens issues to be aware of when you’re doing triangulation.

If there’s a 3D project in your future, I encourage you to read and learn.

Solve your machine vision lighting problems

In machine vision contrast is king. Creating contrast between what you want and do not want to see makes it possible to apply edge tools that are the basis of just about every machine vision application. (Yes I know about blob analysis, but that too relies on creating contrast. And pattern-matching is little more than fancy edge detection.)

But there is another way. By measuring distance to the target, laser triangulation 3D eliminates the variation in contrast resulting from changes in surface reflectivity. And in the real world the things we need to inspect often vary slightly. Ink dries faster or slower, there’s natural grain in the finish, or the condition of molding dies changes through a production run. All these can upset a conventional front lighting approach.

Note though that low angle and back lighting techniques, by virtue of using geometry rather than surface appearance, are largely immune to changes in appearance.

Take corks as an example, the kind that keep wine in bottles. Being a natural product (I hope you’re not using those awful ‘plastics’,) they have lots of lines on them, so how you do distinguish between lines and edges? With 3D vision of course.

Now I didn’t just happen to pick on corks; they are the subject of an application study in Inspect-online. “No Leakage” (June 27^th, 2013,) describes a system developed by Spanish integrator baxicat visión S.L. This uses two camera-laser pairs to inspect the ends of corks as they pass by in a kind of Ferris wheel arrangement.

The article gives lots of good details on how the task was accomplished, which included enlisting the assistance of point cloud specialists Aqsense. If you’re struggling with a surface that varies in appearance, I suggest you read it.

Cooking up an application

MasterChef is a cooking reality show. Contestants, selected apparently more for their looks or “alternative-ness” than their culinary skills, compete to produce something delicious from a diverse range of ingredients.

Machine vision integration is the same, but with longer timescales. The integrator pulls together conveyors and bowl-feeders, cameras, lights, sensors, controllers and software in the expectation that something almost magical will emerge. Some do it well. A few produce disasters of unimaginable grotesqueness.

Fortunately commercial Darwinism takes care of those who can’t deliver, leaving us with talented integration “chefs” like the folks at Cyth Systems. Their “Integrating A 3-D Inspection System”, case study published on the Quality Magazine website May 13th, 2013, (registration required – sorry,) describes one such system.

This uses six Gocator 3D sensors from LMI mounted over a conveyor. Their outputs are fed into a LabVIEW system that provides the information on product quality. Wonderware handles the user interface, and I’m guessing, a lot of database storage of inspection statistics.

It all sounds great, and is just missing a movie to turn it up to “fantastic”. If a TV channel one day decides to make an integration reality show, Cyth Systems should definitely take part. If they have a sufficiently diverse team they might even win!

Two articles on 3D machine vision

Control Engineering magazine usually has good machine vision articles, and the May 2013 edition is no exception. “Quality control measurements for smooth surfaces” is a light introduction addressing the use of a “laser camera hybrid system”. Think of it as an appetizer before diving in to the delicious meaty steak that is “3D vision system performs rope wear analysis”.

The latter article describes a system using three camera-laser pairs to obtain continuous, (okay, not truly continuous, but sampling at a high frequency,) data about the surface for of wire rope. Then they shine the laser longitudinally on the rope and discuss what data that provides. There are also some good links to related material, so all-in-all, there’s plenty to chew on.

I can’t help thinking though that Bytewise does much the same thing with their Profile360 system for monitoring continuous extrusions. But perhaps the Bytewise software didn’t provide all the information that was needed.

Good articles, all the same.

Stereo vision with a twist

I’m agnostic when it comes to 3D machine vision. I see the value in acquiring another channel of data but I’m concerned about the complexity of processing it. So I read Winn Hardin’s “3D Vision Thinks Outside the Envelope”, (Visiononline.org, May 17^th, 2013,) very carefully.

I didn’t learn a whole lot – Cognex say their 3D camera and VisionPro tools are great, LMI think their Gocator is the bees-knees – but I did follow a link to the Ensenso N10 stereo 3D camera, sold by iDS.

After satisfying myself that it’s not a rebadged Point Grey Bumblebee, I wondered about the third port midway between the two camera lenses. Turns out, this is where they project a structured light pattern from. It’s the technique at the heart of “Projected Texture Stereo Vision” which is what makes stereo a useable tool.

Don’t feel bad if that phrase is unfamiliar to you, I hadn’t heard of it either, but Google has. A paper titled “Projected Texture Stereo” and put out by Willow Garage explains the whole thing. Math isn’t my strong suit, but the bottom line is straightforward: projecting a pattern onto an image provides the system with a whole lot more data from which to compute a disparity map. In other words, more data equals a better result.

Seeing the potential in 3D

3D imaging and inspection seems to be the new frontier in terms of machine vision innovation. There’s lots of application work going on, and on the back of that, there’s quite a bit of Mergers and Acquisitions activity too.

Most recently (May 8^th, 2013,) LMI announced their purchase of white light scanner company 3D3 Solutions. That’s interesting for two reasons. First, it broadens LMI out beyond their laser triangulation base, and second, LMI is itself part of AVT parent, Augusta Technologie. So Augusta are clearly very serious about being a force in machine vision. (And it’s probably their financial muscle that has allowed LMI to launch a slew of new and improved products.)

In the same vein, although it’s hardly news, back in November of 2011 measurement specialists L.S. Starrett acquired laser profile company Bytewise, adding their products to the Starrett range.

So it seems to me I need to launch my 3D business quickly, then start a bidding war between those who are getting left out. Anyone care to join me?

3D sensor

3D is the new frontier in machine vision right now, (yes I know it’s been around a long time but it’s only starting to get out into the real world,) and the leading technology is laser triangulation.

Until recently I thought the only way to do this was to set up a camera and laser, trusting I’d get the angle and resolution I wanted. Now though there are some products that do this for me. In other words, camera and laser are integrated into one package with a preset working distance.

IVC, subsequently acquired by SICK were one of the first companies to do this, followed by LMI with their Gocator, and now a German company, SmartRay, has a similar product.

The SmartRay offering of greatest interest to me is the 700. It’s really small and light, which means I can envisage mounting on the end of an arm and moving it over a surface to be inspected. Sadly, there’s not much detailed info about connectivity on the SmartRay website – it would be great if it used USB – but I guess I’ll have to wait for the sales rep to come calling.

Interesting product, all the same.

A stereovision primer

When it comes to 3D machine vision, my preferred technique is laser triangulation. However, it needs relative motion between laser and object under inspection, which isn’t always feasible. One alternative is to use two cameras and do stereo imaging.

If you’re interested in learning more, especially about the underlying math, let me refer you to “A Guide to Stereovision and 3D Imaging” by Dinesh Nair of National Instruments and published on the NASA Tech Briefs website October 1^st, 2012.

This article gets in to the whole area of “disparity”, (which is the key concept in stereo imaging,) but is a little light on the subject of accuracy. The best information I’ve seen on the accuracy of stereo imaging was given by Dr. Lau, (one of my “friends” over on the right of this page,) in a webcast he delivered for Vision Systems Design. If you visit that site and look under their “webcasts” tab you should be able to find it (unless they’ve rearranged their titles.)

Whether stereo is better than triangulation rather depends on your application, but remember: there’s usually more than one way to skin the cat.

Laser triangulation application

Time is short, so I shall very quickly point you at an interesting 3D application. This is on the website of Netherlands vision company, Ellips. Ellips specialize in fruit and vegetable sorting, and you may have clicked through to them from the article I linked to in my last post, “Where the volume is.” Clearly though, they also dabble in 3D, and as the pictures show, it looks very interesting.

3D machine vision gets a little easier

Despite what the sales folks will tell you, 3D is not simple. Indeed, my experience is much the opposite, so anything that simplifies application development should be welcomed.

Put your hands together then, for a second time in just over a month, for Aqsense. (See “News from NI Week”, if you need reminding about the first time.)

The big deal about their brand spankin’ new 3DExpress product is that it enables any machine vision software to work with 3D data – without any programming. So you can use Sherlock or whatever your favorite product happens to be on what started life as 3D images.

I can’t pretend to have grasped all the implications of this product, though I sense it’s somewhat revolutionary. I rather suspect that what’s needed now is a shift in vision developer’s mindsets so they, or we, can begin to recognize applications.

The speckle-free laser is coming!

Laser speckle is a royal pain in the behind for anyone trying to do 3D imaging by laser triangulation. As I showed in my post back on July 1^st, (“Fixes for laser speckle”) a laser line is far from homogenous. The only way to get a solid line is to crank up the exposure – basically to saturate the image – although motion is the low-cost fix.

Speckle reducers are available, at a price, but surely the answer is to have a speckle-free laser. Until now saying that made it sound as though I don’t understand how a laser works, but a team a Yale University is working to make it a reality.

A summary of their work is given in a brief blogspot posting, “Speckle-free laser imaging using random laser illumination” and there’s a link given to the full paper on the nature.com website. Unfortunately I couldn’t get this link to work, but I did track it down on the Yale website: you’ll find it here: http://www.eng.yale.edu/caolab/papers/nphoton12.pdf .

From my brief scan of the very technical paper it’s apparent that this is just a proof-of-concept: actual random laser products are a long way off. But it’s encouraging that such work is happening. The days of dealing with laser speckle may be limited.

Lasers for your sheet-of-light project

August seems to have been “sheet-of-light” month for MV4U, (“Sheet-of-Light Application”, “Sheet of light on a robot”,) so why not round it off with a discussion on where to buy machine vision-ready laser line projectors?

Four companies that I’ve dealt with are, in no particular order, ProPhotonix, Schäfter + Kirchhoff, Power Technology, and Z-laser. I should also mention that Coherent is back with a reincarnation of the old Stocker-Yale line of lasers. They all offer good products, yet I’ve always struggled with two particular aspects of laser line-based 3D scanning; line intensity and width.

Line width is always an issue because it affects resolution. If the line is wider than what you want to see the local height change tends to get drowned out. The problem though is that width is tied in to depth of focus and working distance. Essentially, a narrower line has a limited depth, which is a problem when you’re trying to measure height, and most narrow line lasers have very little standoff.

Intensity becomes an issue when scanning a surface quickly. The exposure has to be shut right down, so little light gets through, meaning that more light is needed. Most laser line projectors are in the 5 to 50 milliwatt range, which, depending on the surface being lit, is often insufficient.

I see two solutions to the intensity issue: more output (which quickly garners attention from your Laser Safety Officer (and if you don’t have one of those, you should,)) and matching wavelength to peak camera sensitivity. This last idea worked OK when cameras peaked in the red, but these days peak sensitivity is in the green region. And have you tried finding a green laser line projector? Not too many around.

So with all that said, let’s talk about Osela. This Canadian laser company doesn’t offer a green laser but they check the other boxes. They understand laser optics and can offer very thin lines, and they have some honkin’ great outputs – 3watts is enough to do some damage! So if you need an intense yet thin line, try talking to them.

News from NI Week

Having spent a few scintillating days in very hot Austin, I thought I’d share with you what I consider the big news: LabVIEW 2012 has support for stereo imaging.

You can read all about it on the NI website, under “3D Imaging with NI LabVIEW,” and there’s some good stuff to take in. In particular, NI has provided some of the math behind stereo and spends some time exploring possible applications.

What’s interesting to me though is how NI seem to be steering clear of line-of-light based 3D for part inspection and instead are pursuing the bin picking and autonomous navigation uses of 3D. Why should this be?

They have a link with Aqsense, so they can offer some tools for 3D part inspection, but was part of the deal an agreement that they wouldn’t compete with the Spanish software house? If so, that was a smart move on the part of Aqsense. What’s your theory?

Sheet of light on a robot

Laser triangulation-based 3D scanning systems have become rather “old-hat” of late. Sick have been selling their Ranger product for several years and AT have a similar 3D camera. But even so, I hadn’t seen anyone mount the laser and camera system on the end of a robot.

That’s what the video below, from AQSENSE shows.

AQSENSE is an interesting company. Several years ago there had the foresight to recognize that laser scanning was on the cusp of going mainstream, but rather than develop a camera or scanning, they decided to tackle the software. Specifically, they started, (if my memory serves me correctly,) with an improved peak detection tool.

More recently, this has grown in to a set of products all aimed at simplifying the business of acquiring and handling point clouds. This vid gives an overview of their technology, (though I recommend watching with the sound muted,) and I encourage those seeking more info to visit their website.