Who’s got RealSense?

Raise your hands please if you’ve played with this new Intel camera technology.

Like a Kinect on steroids, the RealSense promises to make 3D imaging easy. Apparently it came out at the beginning of the year, but between doing my day job and dealing with illness it passed me by. It was only Vision Systems Design’s interview with Arnaud Darmont that alerted me to the fact that this is something new.

I may have several applications, but I’d like to learn more about how it works. I can’t find any details on the Intel website, although a post on the Image Sensors World blog, “Intel Releases More Details on its F200 RealSense Camera” suggested it’s an IR light pattern projection system.

However, commenters in a discussion forum argued that it’s actually a Time-of-Flight system. I’m not sure about that, but I can’t find anything definitive. So, can anyone share links to definitive info about how the RealSense F200 camera works?

3D without the glasses

Let’s get back to the Kinect. In a blog article titled “Playing in a virtual sandbox”, Vision Systems Design editor Andy Wilson describes an augmented reality system based around a Kinect. Most augmented reality systems that I’ve seen seem to involve wearing fancy glasses, especially when 3D is involved. This one doesn’t.

Now much as I enjoy Andy’s writing, his title and description really don’t do justice to the coolness of this idea. To begin with, there’s nothing virtual about the sandbox. It’s 100% real, and is used a surface that can be sculpted as needed. A digital projector paints lines, colors, shapes etc onto the surface.

You’ve seen systems that paint structured light patterns onto surfaces, thereby deducing the contours, but this is different. The Kinect maps the surface in of the sand in 3D, and software then decides what to project onto the sand, based upon the local height.

In the detailed description that Andy links to, (follow my link to his page, and from there to the UC Davis site – I don’t want to divert his page views,) the system is being used to illustrate topography. Heap the sand up to form a hill and a lake appears at its base. Contour lines are also projected, illustrating lines of equal height.

The UV Davis site has a couple of movies that illustrate the interactive, real-time nature of the mapping and projection. Take a look because they just hint at the potential of this idea. I already have some thoughts about how this could be commercialized, but if you have ideas please feel free to share them.

Build your own Kinect

You’re probably tired of me banging away about how Microsoft’s Kinect is a great low cost 3D vision sensor, so today I’m taking a different tack: why not build your own?

The obvious answer is “because the Kinect is only $149,” but may I suggest a reason beyond saving money. If you build it yourself you’ll understand it better than if you buy a black box. And that may let you do things that you haven’t yet even thought of.

How do you get started? Go to “Kinect-Like 3D camera” on the hackengineer website and follow the instructions provided.

Let me know how you get on.

Kinect machine vision application

Regulars to these pages know that I’m pretty excited about the potential of Microsoft’s Kinect, in fact I’m working on an industrial application right now, but I haven’t yet seen any real applications.

That has all changed with the announcement of the “Collision-Proof” power wheelchair from GeckoSystems. This uses the Kinect to detect hazards and stop with wheelchair user/driver from hitting them. There’s lots of info and a couple of movies on the page I’ve linked to, so just click on that. I think you’ll agree that this seems like a great application for this inexpensive 3D imaging system.

Gesture recognition

eyeSight Mobile Technologies is a company doing interesting things with machine vision. They’re using the camera integral to almost every mobile gizmo on the market as an input device. The video on their website shows how a simple sweep of the hand can be used to turn an electronic page, adjust volume, or select from on-screen icons.

What intrigues me is that they’re doing this without 3D. They use a single camera and ambient lighting: there’s no IR projector as with the Kinect, and no time-of-flight hardware either. So how does it work? No details are provided, although I suspect there’s some kind of feature extraction – the hand – and perhaps some optical flow analysis to determine how the hand is moving. But I may be simplifying things too much. A quick Google threw up a link to this research on vision-based hand tracking.

The eyeSight video, naturally enough, makes the system look very robust. I’d be interested to know just how much precision is needed with the gestures – I can imagine a significant training time, just as there was with the old handwriting recognition systems. Neither am I convinced by all the applications shown. For in-car use I think voice recognition is a better way to go. In my oh-so-humble opinion, the killer apps will be those where messy hands need to be kept away from keyboards and mice. The medical field springs to mind, and perhaps manufacturing, especially in places where gloves have to be worn.

In short, this looks like a technology with a future, and judging by the funding they’ve received, I am not alone in this view. Now, who will be first to introduce a shop-floor version?

Something for the weekend

It’s not often machine vision gives us chaps an excuse to look at attractive young women, so I thought I’d save this post for Friday and the weekend. And I should mention that this comes courtesy of gadget blog gizmag.

You’ll recall that the driving force behind the Kinect was Primesense. Well Primesense have now moved into the tricky business of 3D body scanning. Partnering with Bodymetrics, they have produced a machine that maps the human body and advises on clothes that fit.

Okay, I realize that few of us scruffy male engineers are going to rush over to London, England to get scanned, but it does give us a great reason to watch the technology in action.

Strangely enough, no portly, middle-aged male engineers were featured in the “The science behind the fit” video, which I heartily endorse, purely for its overview of the 3D technology of course.

Gesture recognition – the next big opportunity for machine vision?

Is it safe to say that you are all familiar with the Kinect from Microsoft? If not, allow me to summarize: the Kinect allows people to control their Xbox game system by making gestures. The system projects a pattern of infrared dots out into the room, and these are seen by a camera set some inches to the side of the projector. With a little bit of math, the Kinect can work out the 3D coordinates of each one of those dots. Thus when a user moves his hand in a certain way, the system sees that motion and can translate it into a command.

However, there is an alternative method, and that is to use 2 cameras in a stereo set up. This is the approach taken by TYZX in their 3D products. In fact if you take a look at their website you’ll see they’re doing some really cool things, like tracking people in retail environments.

Now TYZX is an interesting company, not least because a Paul Allen is listed as one of their investors (the Paul Allen? I’m guessing so.) In fact they are going in the direction that, a few years back, I expected Cognex to take: broadening the application of machine vision to sophisticated 3D tasks, like intelligent control of entrance doors. Well TYZX aren’t doing that yet, but I feel sure their portfolio of patents would let them go in that direction.

And related to all this is the recent news that Qualcomm has acquired the gesture recognition technology of GestureTek. Now Qualcomm is a big, smart organization, so I’m sure the only reason they would make such an investment is because they see a big opportunity in the years ahead. As does Paul Allen. As, indeed does Microsoft, who are starting to look very smart in quickly leaping in to bed with Primesense.

It all points to an area that’s rapidly becoming ‘hot’, so I suggest we watch these companies closely.

I should buy a Kinect

Regular readers know that the Kinect, (a low cost, 3D imaging sensor from Microsoft, for those of you who’ve been living in another galaxy,) fascinates me. I see great potential for this device as a very inexpensive way of getting 3D vision into the manufacturing environment. And according to “When human gestures direct machines” published in Machine Design, May 19^th, 2011, Microsoft share my vision.

Unfortunately there are no specifics in the article, which leads me to think that Microsoft aren’t actually working on this themselves. Instead, I suspect they’re just trying to roll out an idea in the hope that someone else will pick it up and run with it.

Some of you will know that machine vision software house MVTec have already been exploring ways to utilize the Kinect, as reported in Vision Systems Design back in Aprill 2011. Of course, we shouldn’t forget that the technology behind the Kinect actually came from Primesense, which leads me to wonder who I need to talk to about my industrial application; Primesense, MVTec or Microsoft?

Sadly though, this is all just idle dreaming because I don’t yet own a Kinect. Neither does my workload allow me to explore what I could actually do with the device. But if there are any Venture Capital people reading, it would take little encouragement for me to break away and pursue the opportunities that the Kinect offers.

Leaping on the Kinect bandwagon

A number of hobbyists have interfaced the Kinect with their pet projects, but MVTec is, to my knowledge anyway, the first company to have tied it in to a commercial machine vision software product (Halcon.) You may have seen that a reader posted a comment to this effect a few days ago but as I couldn’t find any conformation on the MVTec web site I thought it best to wait until it became official.

The announcement I picked up was on the AIA web site, “HALCON with Microsoft Kinect for 3D Vision,” (www.machinevisiononline.org, posted February 16th 2011.) This gives a fair bit of detail, along with some screenshots that seem to demonstrate the effectiveness of the Halcon implementation. This seems to confirm that the Kinect may have industrial potential, although perhaps what the world is waiting for is an industrially-hardened version.

I wonder if that’s the business opportunity I’ve been waiting for!

Hacking the Microsoft Kinect for machine vision

If you’re a regular reader you’ll know I’m excited by what the Kinect can do. Back on January 31st I posted some links to a site that explains how to interface the Kinect with LabVIEW, (“Microsoft’s Kinect as a machine vision platform?”) Well life just keeps getting better and better, because now it’s possible to use the Kinect with the RoboRealm machine vision software product.

In truth, much as I like RoboRealm, the really interesting thing about this article (click the link above to read it,) is that it explains how to use the Kinect with Windows. Once you’ve achieved that, then I imagine the possibilities are limitless.

I don’t yet own a Kinect myself, but I think that is going to change in the very near future.