Type a word or phrase into an Internet search engine such as Google or Yahoo, hit return and in an instant, dozens of “hits”—Web sites containing words that match your query—appear on the computer screen. Now, imagine a similar database that operates not with words but with shapes, specifically, leaf shapes.

It would work like this: carry a camera cell phone into a forest, pick a leaf from a tree and snap its portrait. In an instant, the phone transmits the image to a computer that matches a silhouette of the leaf against a database of leaf shapes from thousands of plant species around the world. Exact matches for the leaf are returned to the screen of your phone along with species names and detailed botanical information.

Sound farfetched? Such a device is already very close to reality, thanks to recent collaboration between the Department of Botany at the Smithsonian’s National Museum of Natural History and the computer science departments of Columbia University and the University of Maryland. Tentatively called the Image Identification System or IIS, the invention has the potential to revolutionize the identification of plant species in the field and greatly accelerate the naming of new plant species.

Electronic field guide

For Peter Belhumeur, a computer scientist at Columbia who studies the use of computer vision to identify human faces, it all started years ago in the Connecticut woods. Belhumeur recalls just how hard it was to use a standard field guide to identify different tree species during walks with his children. “I rarely found the right answer on the first try,” he says.

Brainstorming with computer scientist David Jacobs of the University of Maryland about possible new uses for computer object recognition,“We both thought of leaves,” Belhumeur recalls.

So in 2001, Belhumeur and Jacobs came to visit John Kress, a curator in the Natural History Museum’s Botany Department, which houses the National Herbarium, a resource with 95,000 catalogued botanical type specimens—the definitive reference specimens used to identify new plant species—and an additional 4.8 million representatives of plant species from around the world.

What the scientists came up with was an ambitious plan to develop an electronic field guide—a portable system that could automatically identify a tree species from the shape of one leaf. They also wanted the device to simultaneously provide researchers in remote locations Internet access to botanical data on species in the

Smithsonian’s database.

Soon after, the IIS prototype was born. The device operates on a mathematical formula known as the “inner distance shape context algorithm.” Specifically developed for the project, the computer uses the algorithm to analyze 64 uniformly spaced points along the boundary of any given leaf. It measures and matches the distances between the points on a leaf specimen with the same data from leaves in the computer’s database.

To use it, a botanist takes a digital photograph of a leaf, transfers the image to a small portable computer and, in seconds, the computer matches the photo to a series of images of similarly shaped specimens.

“In the lab, we get about 96 percent correct identifications,” Kress explains. In the woods, “it’s more like 75 to 80 percent.”

Race to catalog

Ultimately, the project’s biologists and computer experts would like the device to be able to determine, almost instantly, if a plant is new to science.

“Speeding things up is advantageous, to say the least,” Kress says. Naturalists around the world are in a race to catalog and understand biological diversity before many endangered habitats and species disappear. According to Kress, of an estimated 400,000 species of plants on Earth, as many as 10 percent have yet to be identified.

Traditional methods of identifying and describing a new species can take years. Often, a taxonomist must compare new specimens with published literature and preserved plant specimens in museums around the world.

“Comparing a new plant to all known plants, that’s the time-consuming part,” Kress explains. By harnessing computers to make specimen comparisons, the electronic field guide promises to greatly accelerate new species identification.

Online archive

To help in that process of speedier discovery, the IIS project has recently been augmented by the completion of an online digital archive of the Smithsonian’s botanical type specimens. Building on work initiated by Botany Collections Manager Rusty Russell, the project, which got underway in 2002, has created high-resolution digital images of some 88,000 plant specimens which botanitst also can use in the field as a visual reference.

At the same time, Smithsonian botanists working on the project have collected thousands of leaves from 300 species of plants native to Plummers Island, a small, wooded tract in the Potomac River. Images of leaves from Plummers Island make up the database for the first working prototype of the electronic field guide. In Kress’ lab at the Natural History Museum, botanist Ida Lopez gives a demonstration of how the system works.

Arranging a fresh green “unknown” leaf on a sheet of white paper, she snaps its photo with the small, digital camera built into a Sony Vaio laptop computer. In a matter of seconds, 20 thumbnail photos of potentially matching leaves appear on the computer screen.

“You can then enlarge what you think is a possible match,” says Lopez, doing just that by clicking on a thumbnail photo. She compares the base of the potential matching leaf with the base of the leaf she photographed and declares them the same species, common hackberry (Celtis occidentalis). Just to be certain of her identification, Lopez next uses the field-guide computer to search for and study other examples of hackberry leaves from the digital archive photos of dried specimens in the Botany Department collections.

Eventually, a full set of color images of each species will be available for comparison. When Lopez finally clicks the match button on screen, the electronic field guide records the Global Positioning System coordinates of her location—useful information to help botanists using the system in the field.

iPhone

The field guide isn’t perfect. When Lopez and other museum botanists took it for a test drive in September in Washington, D.C.’s nearby Rock Creek Park, the autumn leaves threw it for a loop. “We had drying leaves,” Lopez explains. “As the leaves dry, they tend to get distorted in shape. That was problematic.” Many of the late-season leaves also had been chewed by insects, another challenge for the system’s shape-matching computer algorithm.

Since creating the electronic guide to the trees on Plummers Island, the project has assembled comparable datasets for woody plants of the entire Washington, D.C.-Baltimore area and, in a nod to Columbia University’s role in the project, a dataset for the 125 tree and shrub species in New York City’s Central Park. Soon, Kress hopes, a fourth dataset will exist for the flora of Barro Colorado Island in the Panama Canal, part of the Smithsonian Tropical Research Institute.

The project also is focusing on a version of the guide that might be used by anyone curious about trees in their own backyard.

“We hope to have a system for identifying the woody plants of the entire continental United States working over the iPhone in the next few years,” Peter Belhumeur says.

And while it may be a while before botanists and other scientists can get their hands on a commercial version of the electronic field guide, Kress thinks demand will be strong. At a recent meeting in Mexico of some 600 tropical biologists from around the world, Lopez and post-doctoral fellow Ling Zhang put the prototype field guide through its paces.

“Everybody was approaching us and saying things like, ‘Can you help us develop this for our field site in Panama?’ or ‘We need one of these for the dry forest of Brazil,’” Kress recalls happily. “Everybody wants this.”