Wednesday, October 30, 2013

Avoiding hubris


It’s always dangerous to claim you have the best of anything. Inevitably, someone will point to something better. I suspect that’s why Thor was careful to include a question mark at the end of his blog post, “New in Scorpion Vision X.II – The best circle finder in the world?”

The problem Thor describes – robustly finding a circle – is one with which I’m familiar, so I read his blog post carefully. In summary, he’s using a RANSAC approach to reduce the impact of outliers. RANSAC is a contraction of RANdom SAmple Conesenus algorithm. It’s explained in great detail by Marco Zuliani in a long paper titled, “RANSAC for Dummies”, or for a much briefer introduction you can check out his webpage: http://vision.ece.ucsb.edu/~zuliani/Research/RANSAC/RANSAC.shtml

Returning to the “best in the world – question mark” claim, how could we prove this? I can imagine producing an image of a circle with defined levels of contrast and noise, then using a number of vision packages report side, roundness and so on. The one that gets closest to the actual numbers is then declared “the best”.

I’ve touched on this kind of benchmarking in previous posts. But who would do it? I envisage a kind of Consumer reports organization that would perform standardized testing, but I can’t think how it would be financed. Unless the AIA felt it was within their remit.

Until then, always pose your claims in the form of a question.

1 comment:

Thor said...

The CircleFinder2D core is ransac, the claim for being the best in the world is of course not based on ransac concept but on how the user in challenging 3D application can tailor cost function and multi search to actually find the best points that is actually is the best fit taken into consideration that this is real-time machine vision - to the people that does not believe - try yourselves or ask for demo :)