Planes, Birdhouses And Image Recognition

My recent blog post on the limits of neuromorphic computing took an optimistic view: even neuromorphic systems that are relatively crude by the standards of biological brains can still find commercially important applications. A few days after I finished it, I was reminded that the pessimists are not wrong when a friend of mine shared this image.

Fig. 1: Trover Gourds in purple martin nesting colony. Photo courtesy of Cara Rayno Seiffert.

You may not have seen anything like this before. But based on the birds perched on several of the objects, and the visible pieces of some kind of support frame, you might guess (correctly) that you’re looking at nesting gourds designed for a type of bird that lives in communal groups. Also, having seen this photograph once, it’s likely that you’ll recognize these objects if you see them again, even from a different angle or in a different context, such as on a shelf in a store.

When I showed this photo to Google Image Search, widely viewed as one of the best general-purpose image recognition tools available to the general public, it responded with pictures of airplanes. That answer makes a certain amount of sense. Airplanes in flight are also roundish objects silhouetted against a blue sky. And it makes no sense at all. Airplanes don’t fly nearly this close together, don’t hang from a frame, and are much too big relative to ordinary birds.

This one image, then, highlights many of the differences between automated and human image recognition. As a human, you combined clues from the image itself with your previous knowledge of birds (and gourds) to reach a conclusion that you’ll be able to draw on in the future. Google Image Search, in contrast, compared this photo with visually similar objects in its database, but has no inherent understanding of “birds,” “birdhouses,” or “airplanes,” much less the fact that “birds” and “birdhouses” are related concepts. Nor is it likely to “learn” from its mistake unless a human reader of this post assembles a training series of similar images for it. Even then, it won’t necessarily be able to recognize natural nests or other types of nesting boxes as performing similar functions for other types of birds. (This is a connection that you probably made automatically.)

One of the key challenges of neuromorphic computing is conceptual. Human intelligence comes naturally to humans, but is difficult to even describe in the linear, algorithmic terminology of machine computation.