Sketch Recognition

Imagine being able to shop for your ideal pair of shoes simply by sketching them on your touch-screen device. That’s the goal of SketchX, a computer program that can recognize sketched images and retrieve search results better than text-based systems. Using a technology known as fine-grained, sketch-based image retrieval (SBIR), the new program is superior to using verbal descriptions, particularly when the level of detail needs to be precise. Even traditional photo searches are not as effective, since they tend to restrict the results too narrowly.

Designed to emulate the human brain by processing arrays of simulated neurons, the system was “trained” by matching about 30,000 photos and sketches. Tests have demonstrated a top-10 retrieval accuracy (i.e., the product is displayed on the first page) of close to 100 percent on some object categories.

The technology could revolutionize online retailing, which is still primarily designed around category-level search techniques and can be cumbersome to browse. With touchscreens becoming more ubiquitous, sketching could easily replace text and photos as the search method of choice.