Halcon for dummies?

Power, or so we are told, is an aphrodisiac. If so, that would explain why many machine vision professionals are so drawn to Halcon. While some, (mostly those in Natick,) will argue VisionPro is at least comparable, there’s no dispute that Halcon is one of the most powerful vision tool libraries.

It’s also rather hard to use. It needs C programming skills combined with a deep understanding of machine vision algorithms, and that puts it out of reach for many of us. And then, late in 2014 MVTec started promoting a product called Merlic.

While it’s a simplification to call it ‘programming-free Halcon’ it is built on a subset of the underlying tools. That should, in principle anyway, make it rather attractive for those of us not so skilled in advanced coding, which is why I was eager to take a look.

I think pretty much everyone treated to a preview came to the same conclusion: like a High School football player, it had potential, but really needed development.

Well it seems MVTec listened to the feedback and are making changes. The official launch is now planned for June 1^st 2015 (so they still have time to test and implement bug-fixes.) There’s no word on pricing, but I figure if I can save $1,500 a month between now and then I should have enough in my piggybank to buy a copy.

There is however one claim in the press release that grates. “users will be able to create machine vision solutions quickly without any knowledge of imaging technology…”.

I’m sorry but I find that hard to believe. Surely any user needs to know how to get an image. Isn’t that one of the most basic skills?

But that gripe aside (and I wouldn’t be me if I didn’t gripe,) Merlic promises to be one of the most interesting new products of 2015. Join me in taking a look when it goes on sale.

Next-gen smart camera technology?

Xilinx makes FPGA’s, the kind of programmable board that can process images really fast. Marry that with an image processing package, sprinkle some Halcon tools over the top, and you’ve got yourself a real-time machine vision system.

At least, that’s how I understand, “End-to-End Smarter Vision Demo at SPS IPC Drives Conference Showcases VisualAppl​ets, HALCON, and Zynq”, which is a post on the Xilinx blog from November 25^th, 2013.

Electronics has always been a mystery to me – I prefer manufacturing where I can see what’s going on – but reading the post a couple of times, I gather this could be the basis of the next generation of smart cameras. I can envisage such devices being incorporated all over manufacturing and packaging equipment, making vision an integral part of every operation.

VisionPro versus Halcon – a summary

This is one of the hoary old chestnuts of the machine vision world – which is better? As with most things in life, there’s no simple answer, but let me start with a very brief introduction.

VisionPro is the highly capable PC-based vision software product from Cognex. It does more than their smart cameras can do. Halcon is another highly capable PC-based vision software product, this time from German vision software company MVTec. MVTec however do not make smart cameras or even smart camera software (so far as I know, anyway.)

If you’re in the market for vision software, and aren’t currently wedded to any particular provider, both deserve consideration. However, it is very difficult to make a side-by-side comparison. Don’t worry though because that’s already been done. No, not by me but by New Hampshire-based vision reseller, JMAK Automation.

There’s a page on their website that provides pros-and-cons for the two packages. (Halcon versus VisionPro.)

You should be aware that JMAK is a Halcon reseller, so they are not impartial. But, I found the comparison stacked up pretty well against my knowledge of the two. In other words, I think it’s pretty accurate.

Does it tell you which is better? No. As any good machine vision professional will tell you, consider what your application needs and then decide. Put another way, the answer to the question is – it depends.

How to invest in machine vision

If you read “Analyzing the global machine vision business” you’ll know machine vision has excellent growth potential. Today let’s talk briefly about how to seize this opportunity.

I know of only three pure machine vision companies that are publicly quoted, Cognex in the US and Basler and Augusta in Germany. (Augusta is the parent of camera-maker AVT and is pretty much just in to cameras.) If you’re interested in other vision companies you’ll need to investigate their parentage.

For example, Dalsa is part of Teledyne, a far bigger data acquisition company. Likewise, PPT is owned by Datalogix, and Microscan by Spectris. Then there are the sensor companies like SICK and Keyence, which I’m not sure how you’d get into. Investing in any of these means exposing yourself to risk beyond the pure machine vision industry, (which may or may not be a good thing.)

Many other vision companies – probably the majority - like MVTec (Halcon software vendor,) are privately held. If you want a piece of their action you’ll need to talk with the existing owners. Good luck with that!

So are there any other options? Well I think you’re left with starting your own machine vision business, developing some innovative technology, and selling out to one of the big vendors. But here’s a question to my community of readers: are there any private equity or venture capital businesses specializing in machine vision?

The Robonaut uses Halcon

Last week I posed a question: “What can you do with Halcon?” A day later I picked up the latest issue of Photonics Spectra and learnt that NASA and GM have teamed up to find out what Halcon can do in space.

“Vision Software Enables NASA Robonaut to See” (May 2012,) discusses how Halcon is being put to work on the Space Station. In particular, NASA is using its 3D stereoscopic capabilities in a number of applications.

I’m not sure what this tells us about the more terrestrial, and more industrial, applications of Halcon, but there is one thing I find interesting. Why did the team choose a German software product over say National Instruments or Cognex?

I think we can be pretty sure that the decision came after a very thorough evaluation process, so I have to conclude that Halcon is indeed an extremely capable product. If it’s good enough for the Robonaut, perhaps it’s good enough for you.

I do however question the wisdom of putting software called HAL – con in space. Are the people at NASA too young to have seen 2001?

“I’m sorry Dave. I’m afraid I can’t do that.”

What can you do with Halcon?

The simple answer is – pretty much any machine vision task can be done with Halcon from MVTec. It’s one of the world’s leading vision software packages, although I don’t see it a lot in North America. (If you have a good Halcon application, please let me know.)

So why isn’t everyone using it? I think there are three barriers.

First, it’s not simple to learn and use. I’m sure MVTec will email me to explain why I am wrong, but in my defense let me say that I have looked at it, I am impressed with it, but with my bias towards a graphical programming interface I see one heck of a learning curve. More of a learning cliff really, think north face of the Eiger. Of course, to cater to folks like me MVTec also offer ActivVisionTools.

The second barrier is price. This is not inexpensive software.

The third barrier is, I believe, a simple lack of knowledge of what one can do with Halcon. This is the one I’m going to put right.

I recently stumbled across an excellent machine vision blog written by Stéphane François of Ontario-based Computed Vision Solutions. Stéphane is a Halconist. I’m not sure if that’s a formal designation like being a Certified LabVIEW Developed or just a title one bestows on oneself, but it’s clear that he knows his Halcon inside out and from front to back.

Stéphane seems to have a little time on his hands – or just manages his time well – because on his blog, titled simply, “Computed Vision Blog”, he describes, in great detail, some of his explorations in machine vision. This year he’s investigated how to run Halcon on a Vision Components nano smart camera, and is now getting to grips with the 3D SICK Ranger product.

There are two things I really like about Stéphane’s blog. First, there’s no BS. He just writes about what he does and observes, with lots of pictures to help explain. And second, he posts lots of video. So if you’re at all interested in learning what Halcon could do for you, (or you want to learn about the VC nano or the SICK Ranger,) log on and get reading.

I should buy a Kinect

Regular readers know that the Kinect, (a low cost, 3D imaging sensor from Microsoft, for those of you who’ve been living in another galaxy,) fascinates me. I see great potential for this device as a very inexpensive way of getting 3D vision into the manufacturing environment. And according to “When human gestures direct machines” published in Machine Design, May 19^th, 2011, Microsoft share my vision.

Unfortunately there are no specifics in the article, which leads me to think that Microsoft aren’t actually working on this themselves. Instead, I suspect they’re just trying to roll out an idea in the hope that someone else will pick it up and run with it.

Some of you will know that machine vision software house MVTec have already been exploring ways to utilize the Kinect, as reported in Vision Systems Design back in Aprill 2011. Of course, we shouldn’t forget that the technology behind the Kinect actually came from Primesense, which leads me to wonder who I need to talk to about my industrial application; Primesense, MVTec or Microsoft?

Sadly though, this is all just idle dreaming because I don’t yet own a Kinect. Neither does my workload allow me to explore what I could actually do with the device. But if there are any Venture Capital people reading, it would take little encouragement for me to break away and pursue the opportunities that the Kinect offers.

Leaping on the Kinect bandwagon

A number of hobbyists have interfaced the Kinect with their pet projects, but MVTec is, to my knowledge anyway, the first company to have tied it in to a commercial machine vision software product (Halcon.) You may have seen that a reader posted a comment to this effect a few days ago but as I couldn’t find any conformation on the MVTec web site I thought it best to wait until it became official.

The announcement I picked up was on the AIA web site, “HALCON with Microsoft Kinect for 3D Vision,” (www.machinevisiononline.org, posted February 16th 2011.) This gives a fair bit of detail, along with some screenshots that seem to demonstrate the effectiveness of the Halcon implementation. This seems to confirm that the Kinect may have industrial potential, although perhaps what the world is waiting for is an industrially-hardened version.

I wonder if that’s the business opportunity I’ve been waiting for!

ITC ruling seems to support MVTec

It looks as though MVTec are edging closer to victory in their patent dispute with Cognex. On July 16th ALJ Charneski of the US International Trade Commission (USITC) issued a ruling to the effect that Halcon machine vision software product doesn’t infringe upon Cognex’s patents 7,016,539 and 7,065,262. He also appears to have said that both patents are invalid as they fail to claim patent-eligible subject matter. (I’m getting my info from the ITCblog.)

The case isn’t over yet; apparently we have to wait until November for a final ruling. If I were a gambling man, I think I’d wager a few bucks on MVTec coming out on top.

Unfortunately for MVTec though, it seems as if the long running dispute has still done them some damage in the US. You may recall that several importers and users of Halcon were also named in the original Cognex complaint, and some of them have already fled the MVTec ship. One wonders if the real purpose of the case was just to cause trouble for a competitor.

Cognex, MVTec and Bilski

July 4th is a pretty big deal where I live, so although the 4th actually falls on a Sunday I had planned to take the 5th as holiday. (All this blogging is hard graft, and I need a break too y’know!)

But in a very polite email, Lutz Kreutzer of MVTec suggested that I get back to work by commenting on the latest development their legal wrangle with Cognex. The news actually broke a few days ago and it seemed pretty widely disseminated so I wasn’t sure I needed to add my two cents, but to keep Lutz happy I will do just that.

The big news was that on June 22nd the US Patent and Trademark Office (USPTO) cancelled all claims of US Patent 6,850,646 that Cognex had asserted against MVTec. To understand what this means, I took a look at the particular patent.

With the scintillating title of “Fast, high-accuracy multi-dimensional pattern inspection,” the ‘646 patent basically covers the PatQuick™ pattern matching algorithm. I will admit that I did not plow through the entire patent – my insomnia isn’t that bad – but it seems to cover a lot of the geometric pattern matching technology at the heart of Cognex’s IP and their dispute with MVTec.

In other words, by invalidating much of the patent (it’s not clear to me that whether or not the entire patent has been invalidated,) the USPTO has severely weakened Cognex’s case. No wonder Lutz wanted to make a big song and dance about it.

But what does all this have to do with Bilski?

Well, after lengthy consideration, the US Supreme Court finally issued a ruled on the Bilski case. This concerned the question of whether software can be patented. Unfortunately, the ruling was not particularly clear (see here for a summary) but the court seems to be leaning against the patentability of business methods software.

This too seems to be in MVTec’s favor as pattern-matching, along with many other software algorithm-type patents, is probably lumped under the “business methods” heading.

One thing is for sure though: the legal people working for Cognex are likely to be studying their briefs over this long weekend!

At least Lost finally ended!

Much like the TV series Lost, the Cognex-MVTec battle goes on and on. And while I wonder why I keep following the unfolding events, I shall continue to bring you the news.

The news as of today is this: there is no news.

MVTec have in recent weeks filed two motions for summary determination, one claiming that the Cognex patents 7,016,539 and 1,065,262 are invalid because they don’t cover patentable material and the other claiming that Halcon doesn’t infringe either patent anyway.

In both cases Administrative Law Judge Carl C Charneski ruled in favor of Cognex, saying as best I understand it, that the argument is complicated and needs a thorough examination. What makes things even more complex is that the MVTec claim of unpatentable material relates to these complex business process software issues (think Amazon’s One Click ordering system,) where a Supreme Court ruling is expected any day. (Google “Bilski” +”Supreme Court” for more details.)

If you’re interested in following along, here are a couple of links, courtesy of itcblog.com:

ALJ Charneski Rules On Motion For Summary Determination Of Invalidity In Certain Machine Vision Software (337-TA-680)

ALJ Charneski Rules On Motion for Summary Determination Of Non-Infringement In Certain Machine Vision Software (337-TA-680)

Correction on MVTec HALCON availability

Earlier this month, under the heading of “Cognex – MVTec update”, I said that the ITC had blocked the importation of HALCON while they investigated Cognex’s patent infringement complaint.

Seems that was yet another slip of my pen. A reliable source emailed me to say that HALCON can still be purchased legally in the US, which is obviously good news for MVTec. Said source went on to say that the case seems to nearing a conclusion and that the folks at MVTec appear optimistic.

I’ll have more information on this as soon as it becomes available.

But in a related development, Law360 reports that two of the codefendants, Rasco and Subtechnique, have settled with Cognex. Perhaps not what one would expect from companies that believe their case is strong?

Cognex – MVTec update

For those interested in this case, I have an update.

(Background: Cognex have alleged that the Halcon package, sold by German machine vision software company MVTec, infringes several of their patents. The case is currently before the court in Massachusetts but since patent cases move at glacial speed Cognex have also taken the matter to the US International Trade Commission. The ITC has blocked the importation of Halcon while it investigates with a view to making a determination regarding patent infringement. As a side note, the block includes products that incorporate Halcon, which obviously increases the pressure on MVTec.)

So where have things got to? Well according to a posting on the itcblog, the case is apparently in the “discovery” phase where all relevant documents have to be produced.

Naturally enough, MVTec are arguing for the production of documents that demonstrate the existence of prior art, claiming that such are available from “a company called Acumen” and also that Cognex has documents relating to “a former Cognex product known as ‘CNLPAS’ or ‘CONLPAS’”.

Cognex has responded that it will produce the “Acumen” documents but not those relating to CONLPAS on the grounds that the request “is burdensome and untimely.”Unfortunately for MVTec, the ITC ruled against them, meaning that Cognex don’t have to produce the documents.

I’m no legal scholar and I don’t want to take sides, but I do find it a little strange that Cognex won’t produce these documents. I’m not familiar with ‘CONLPAS’ either, but I’d like to know what possible bearing it has on the case.

If any of you dear readers can shed any light on this, please drop me an email.

By the way, the details of the ITC ruling came from “Order No. 54” (which is a pdf.)

Where’s Halcon now?

I consider Halcon one of the top tier of machine vision software products. It’s up there with Sapera, Common Vision Blox, and of course, VisionPro.

And there’s the problem. As part of their patent infringement action against MVTec, Cognex have an application before the US International Trade Commission (ITC) to block the sale of Halcon in the US. As a result, I’m not sure if I’ll ever be able to play with some of the novel Halcon configurations that the gentlemen in Germany are working on.

First up is Halcon embedded on a Beagleboard (see picture.) A Beagleboard is a single board computer utilizing a Texas Instruments processor. I can imagine hooking it to a USB camera to create a rather useful stand-alone vision system – the kind of thing that could be incorporated into a piece of factory automation. I also have to wonder if this means I could run Halcon on those rather nice smart cameras from Vision Components.

Second, MVTec have also configured Halcon to run on a Nokia cellphone (see second picture.) I will admit that my first reaction was, “why?” Having thought about it though, I think there is value in a portable machine vision system, even if only for OCR and bar code reading. I can imagine the Halcon-equipped Nokia being used by anyone who needs to acquire data while in the field – meter-reading would only be the beginning.

All this leads me to wonder, wherever will we see Halcon next? More importantly, will we ever see it in the USA?

Two great machine vision jobs

Here are a couple of terrific opportunities, both on the East Coast:

Machine Vision Software Engineer

Machine Vision Bio-Imaging Research Scientist

The first one looks to be a fairly standard kind of vision job, working for a machine builder and using Halcon. The second is rather unusual; they are seeking someone with bio-medical experience and pattern recognition algorithm expertise. Judging by the salary on offer - $120,000 up to $175,000 - they’re looking for a pretty special individual. What an opportunity though!

Scorpion rumor

I hear that Scorpion are looking to put their 3D software on the Sony smart camera. If true, this will have them going head-to-head with Sick in the industrial 3D camera business.

As I understand things, this will be part of the upcoming release of Scorpion Vision Version 7.2. No word on pricing, although I anticipate something upwards of €6000 for the full camera+software bundle. Scorpion isn’t the cheapest machine vision software package out there, but founder Thor Vollset is working hard to establish his software product as a credible alternative to Cognex VisionPro and Halcon from MVTec.

If you’re looking for a standard machine vision platform you could do a lot worse than to consider this Scandinavian startup.

Another crack in the wall

Cognex continues to chip away at MVTec in their ongoing intellectual property dispute. After the surrender of Multitest at the beginning of September, Yxlon is the most recent domino to fall. As I write this there’s nothing on either the Cognex or Yxlon web sites: I got my information from the Yahoo Finance site: “Cognex Receives Second Settlement in Vision Software patent Dispute.” No details of the settlement are provided but I think it's safe to assume that Yxlon have agreed to stop incorporating Halcon into products sold within the US.

I also think this proves that when it comes to protecting US interests, the ITC has some teeth. Regardless of the merits of the case, it appears that Multitest and Yxlon preferred backing down to having their US business caught up in the ongoing dogfight.

Where’s this heading? My prediction is that, with their US business under threat, MVTec will have to come to the table and reach some kind of settlement with the powerhouse from Natick.

Changing sides

This post is for those of you who have been following the Cognex-MVTec patent infringement dispute.

First, the story so far: Cognex have alleged that the Halcon machine vision software product infringes some of their patents. In their complaint, filed in Massachusetts last year (I’m working from memory here so forgive me if I don’t have the dates quite right,) Cognex named MVTec, the product developers, along with a number of other companies who use Halcon in products they sell in the US.

More recently, and probably because patent litigation grinds notoriously slowly, Cognex made a complaint to the US International Trade Commission (ITC) regarding the importation into the US of goods incorporating Halcon. The ITC tends to move faster than do the patent courts and has the power to block the importation of patent-infringing good, so presumably Cognex felt this would be an effective way to defend themselves.

Are you with me so far? Good, because here’s the interesting update. It’s reported in motioncontrol.com (September 08, 2009, also on the Cognex web site,) that one of the defendants in the ITC case, Multitest Elektronische Systeme GmbH, has settled with Cognex. No details of the settlement are given, except that Dr. Bob is quoted as saying, “… we look forward to working with Multitest to integrate Cognex software in its products in the future.”

In other words then, Multitest got themselves off the ITC “hook” by agreeing to drop Halcon for one of the Cognex software products, presumably VisionPro. Will any of the other companies named in the ITC complaint take the same route out? Watch this space!

Cognex seem determined to prevail

If someone copies your idea you will likely be pretty upset about it, and more so if you’d gone to the time and expense of obtaining a patent. Now a patent doesn’t actually give you a monopoly, but it does entitle you to sue anyone who uses your intellectual property. The downside is that patent cases can be very expensive to prosecute, and tend to move slowly. If your pockets are deep enough you’ll be able to afford the lawyer’s fees, but while your case grinds through the courts your competitor can grab some of your market by selling the infringing product.

If you’re doing business in the US, this is where it can pay to lodge a complaint with the US International Trade Commission (ITC.) The ITC has the power to block the importation of goods that are alleged to infringe intellectual property laws, and will move faster than the court system. This means that if you believe a competitor is importing infringing products, you can have the ITC block their shipments while waiting for the legal system to work. (I hope I haven’t made that overly complicated.)

Back in 2008 Cognex filed suit against MVTec alleging infringement of certain patents. (From memory, I think they are related to pattern matching – Cognex are very possessive about their PatMax technology.) But if you read the machine vision magazines you’ll know that Halcon (from MVTec,) is still on sale in the US.

Cognex seem to have a problem with this, because on May 28th of 2009 they filed a complaint with the ITC. Well bureaucratic wheels turn slowly, but on July 15th the ITC announced that they will be investigating the complaint. There are fourteen companies listed as “respondents,” so if you’re selling Halcon, or a machine/system that incorporates it, you might want to get some legal advice on the matter.

Don’t ask me though. I’m not a lawyer and I can’t comment on the merits of the case. Consider me an impartial observer.