An elastic tape measure?

There’s an interesting application story in the August 2009 edition of Vision Systems Design. In an article titled “Eye Defense” Andy Wilson discusses a system developed by AV&R Vision & Robotics of Montreal, Canada to inspect safety glasses.

It’s a challenging task, and I was fascinated by both the development process and the solution that AV&R finally implemented: I certainly encourage you to read the piece, However, what really caught my eye was this quote in the closing section: ““In the development of the system,” says Muldoon [an engineer with AV&R], “it is vital that the machine be capable of performing equally well as a fully trained operator.””

So they want the system to be as good as a human inspector? Let’s think about that.

There’s an oft-bandied-about statistic that human inspection is only 80% reliable. That may or may not be accurate; what we do know is that the human eye is brilliant at detecting small flaws but absolutely rubbish at doing it repeatably. Is that what AV&R set out to achieve? I doubt it.

This goes to the heart of a very big issue in machine vision integration. Why do we measure the performance of an automated system against that of a human inspector? It’s like using an elastic tape measure to check the length of pencils coming off a production line. The comparison of machine to human performance is meaningless because the human is uncalibrated and lacks repeatability.

So what’s the solution? Well if you visit Inspect-online.com and register for their whitepapers, you’ll find a fascinating piece on this very topic. Authored by Dr. Ralph Neubecker and titled, “Performance of Inspection Systems,” this scholarly paper explores the issue of false rejects, false accepts, (which he calls “slippage”,) and the comparison of human to automatic inspection.

Regrettably though, Dr. Neubecker only takes us 80% of the way towards a solution. He provides a very lucid explanation of the problem but fails to leave us with any specific actions to take. I guess he’s holding the door open for other people to explore this area further. Any takers?