CES 2012: The Metadata Generation

For more news from CES, check out our complete coverage.

Who isn’t a digital packrat these days? Many households have three or four computers loaded with music, videos, photos, games, books, and software, with still more material stored on CDs, DVDs, or, increasingly, in the cloud. The challenge is no longer owning what you want, it’s finding it when you want it—especially if you weren’t the person who originally labeled and saved it.

Fortunately, technologists are also users, and they can usually be counted on to solve their own problems. Sure enough, in 2012 we’ll see products that automatically crawl through your content and analyze each file individually. They’ll then generate metadata—a machine readable descriptive catalog of your content—that contains such information as date, location, file type, and often permission rights. All this information about information may seem redundant but without it, frankly, we haven’t a prayer of finding, managing, protecting and accessing all the digital stuff we’ve accummulated, in just the same we wouldn’t ever find useful stuff on the Web without search engines and their vast indexes.

Having metadata about the date and time a file was stored, and its location, seems obviously useful, but what’s this about permission rights? When you want to enjoy a specific video or photo stream, you may well want to first search the metadata index to confirm you own it and to determine if you have the rights to access it, or transfer it to the platform you want to enjoy it on—such as watching a video on your smartphone, when it’s stored on your PC.

In such a system you can locate and enjoy your content simply by describing it to the system—a bit like googling your own computers. You could issue a search request like “find me all still photos and video clips that show family members—including our dog—walking in the snow within the last two years.”

How exactly will that work? After all, automatic indexing isn’t difficult when the content involves machine-readable text like a book or e-mail. But until very recently, indexing music, photos, and video, could only be done manually. Even today, automatic indexing pushes current technology to its limit in respect to both computing power and machine intelligence—and still doesn’t do as good a job as we would like when it comes to understanding content in the same way an end-user would when manually creating such an index. To accomplish this, the technology will need a rich mixture of, at a minimum, pattern recognition, fuzzy logic, artificial intelligence, parallel processing, and expert systems.

For example, when indexing photos the machine needs to automatically determine where on the planet the photo is taken, the season or time of year, and recognize the people in the photo and even the background and other elements. In the case of music files, the application should automatically recognize the music genre, the artist, and the song title, even for live performances.

In 2012, we should start seeing the rise of this technology and information about information will significantly enhance how we manage, access and protect all our personal information at hand  Already we see the building blocks of the underlying technology in the consumer marketplace for music.  For example Shazam and Midomi are two smartphone apps that automatically recognize a song based on a small sample of the tune. (Midomi will even try to identify snippets of a tune that you sing or hum.) With a little additional effort the very same recognition engines that power these two apps can be extended to create the metadata to populate a music content addressable system.

Similarly, the facial recognition systems already deployed today in commercial and law enforcement security systems could be used to create a content addressable system for photos and, eventually, to create metadata for the photographs in an individual’s photo collection.

Stuart Lipoff is a partner of IP Action Partners and an IEEE Fellow recognized for his “contributions leading to commercialization of advanced consumer electronics products.”


Tech Talk

IEEE Spectrum’s general technology blog, featuring news, analysis, and opinions about engineering, consumer electronics, and technology and society, from the editorial staff and freelance contributors.

Newsletter Sign Up

Sign up for the Tech Alert newsletter and receive ground-breaking technology and science news from IEEE Spectrum every Thursday.