Q&A with RoboRealm

RoboRealm is an intriguing image processing product. Developed by a virtual team, and with roots in robot guidance, it’s available as a free download from www.roborealm.com.

The RoboRealm web site gives little detail about people behind the product, so, eager to learn more, I fired off an email. Here are my questions and their answers.

Q. For people who are not familiar with your company or your product, perhaps you could begin by giving us an introduction to both?

A. RoboRealm is a small software company that produces an application that specializes in machine vision for robots. We are a remote company and created by unique individuals around the world that contribute to the development of the company and product by heavily utilizing the Internet as a communication medium. Because of this structure we are able to keep costs very low and distribute the application for free. The application is a Windows based program that provides several hundred "modules" which are isolated pieces of functionality that either alter or analyze an image in some way or connects to devices such as servo controllers or robot kits. One uses the application by inserting these modules into the main processing pipeline in order to achieve a specific desired effect. While there are many configurations available on the website the application is meant to be experimented with or altered in order to customize the functionality towards a user's specific project. As anyone in robotics knows, there is no one solution that fits all. With so many custom projects and the lack of industry standards even with simple communications such as between servo controllers it is really necessary for a "robotic application" to have that kind of flexibility in mind. If that structure is adhered to it becomes simpler to start with what we've done in a tutorial and switch in and out other modules in order to slightly change the actual outcome. That's what allows users with little machine vision and robotic experience to become more confident in this field through interactive experimentation.

Q. And can you describe how people are using RoboRealm?

A. One of the most amazing things we've discovered over the years is the breath to which people are using RoboRealm. But in review that should probably not be such a surprise as the number of things we use our own eyesight for is truly monumental. We certainly do have a lot of users in the robotic space that are using RoboRealm to control robots that one might purchase off the shelf, but that is only the beginning. Many are using RoboRealm for artistic projects that cause machines to move in reaction to a person or an objects presence. Many classrooms are using RoboRealm to teach the basics of image processing. Many users are simply using RoboRealm to stream modified or processed images out to the web. And even more are using it as a tool in much larger projects such as home automation, entertainment for the kids, program automation, etc.

Q. Can you tell us about some industrial applications, or perhaps some that you’d like to see?

A. The machine vision industry is huge! There are thousands of companies that are and that could benefit from image understanding. The most frequent case we see has to do with quality assurance. This ranges from the typical application meant to ensure that an object is the right shape and that it has the right marking in the right spot to checking that baked bread has the right sized "holes" in them due to yeast expansion. There are many situations in a manufacturing line that if someone just were to see a defective product for an instant that would be all that is needed to ensure that only high quality products make it to sale. Many other situations exist in the actual manufacturing process such as parts picking and placement. Robots often use vision to orient their perspective in relation to parts in order to pick them up in the most reliable way and then to place them into the appropriate spot of the item being constructed.

What we'd like to see is more people either sharing their solutions or discussing problems that they are having in these settings. There is still a lot to learn about machine vision and sharing what works and what does not is invaluable. Many vision techniques work well in some situations might fail instantly in others. Knowing what works and what does not still takes a lot of experience to know. As there is no recipe for a correct machine vision solution it is often a black art of experimentation to determine what works. Having an experience log of past projects would help others understand the decisions and why they were made in order to complete their project with success. In too many machine vision projects one can still ask "Can it be done?".

Q. What do you consider the greatest strength of the product?

A. The greatest strength of RoboRealm is in its ease of use through interactivity. By interactivity we mean the process of loading up RoboRealm and connecting to a camera and then "playing" around with all the modules by changing their values to see how the image is affected. Any of the configuration items when changed in the GUI interface immediately change the current image. Likewise, the modules to control hardware devices also share this same behavior and allow users to quickly start controlling their devices. This allows for beginners to learn how certain modules affect the image or hardware and in what way. The fact that you can use a simple webcam or an industrial firewire camera helps one to approach the application in an incremental way. You can start with a small project, gain confidence, and then expand to larger installations all within the same application.

Q. Any hints as to what we can expect in later releases?

A. We have an unusually large list of new features that are being released as fast as possible. We attempt to release new modules frequently to allow others to provide feedback such that we may learn from their experiences and desires. Some of the more anticipated modules will include Stereo disparity calculation from two viewpoints, generic object recognition, face detection and ultimately recognition. We also have a couple modules coming out soon surrounding SLAM (Simultaneous Localization and Mapping) using various visual techniques. These are some of the visual features that are anticipated for next year and will be accompanied by many additional hardware devices including many new robots that are will be sporting cameras as part of their sensor package. We see vision as becoming an essential part of any robotic package and we will certainly position RoboRealm to be at the forefront of that need.

Q. And finally, what’s next for machine vision? Or perhaps more specifically, where do you think the real challenges, and opportunities, lie in machine vision?

A. The easy way to answer that question of "What's next for machine vision" is "A LOT more work!!". We've spent many years investigating the properties of vision that allow us to achieve simple tasks when compared with the human visual system. The human vision system or HVS is the goal. We know where we need to get but we don't know the exact path of how to get there. The machine vision field is filled with a tremendous amount of research that requires many months of programming that can stop at a dead end with nothing to show. But that's the nature of research, you may or may not succeed in any particular direction. Algorithm failure is such a common experience in this field that we view the only way to truly fail is to stop trying to succeed. Eventually the secrets of the HVS will reveal themselves most likely in strange occurrences of sudden understanding more about yourself and the way that you "see". We try to understand why we see the things in the way that we do. Staring at an object until it ceases to be an object but instead becomes a collection of pixels is a common practice with us in order to understand the nature of vision. Perhaps it a little bizarre to the outside observer to see someone so fascinated by the simplest of scenes but if you take the time yourself to really REALLY look at an image it is amazing what can be revealed.

The real challenge to machine vision is to deal with the frustration of trying to replicate something in a machine that already exists in humans. However, we also need to distance ourselves from the biological counterparts as the silicon platform will probably require a different solution. Examples of this have already occurred many times in history. For instance, flight was only truly mastered once we stopped copying the birds and actually stood back to understand the principals of flight. How close is an airplane to a bird? The basic theory behind why birds and airplanes fly is the same but the actual means is vastly different. We suspect that visual intelligence and intelligence in general will probably share the same fundamentals as human intelligence but the actual implementation will probably be quite different.

The opportunities for machine vision are vast. What is the value for being able to see and interpret a scene? We've only begun to understand where and how machine vision can benefit many areas of our lives. We hope that a free tool like RoboRealm helps that understanding and stimulates the creativity in people to use that ability in new and fascinating ways!