A Prototype of Pivot Searching

Finding information by what you were doing the last time you saw it

3 min read

A Prototype of Pivot Searching

screen shot

Image: Joshua Hailpern
Click to enlarge.

17 October 2011—Imagine if you could conjure up a key piece of knowledge you had forgotten by having a computer summon everything you were seeing, hearing, and doing at the time to help jog your memory. Researchers in Illinois are now developing such technology, which will help people relive the past to search for lost data. The aim of the software, called YouPivot, currently in beta, is to find digital information by tapping into how human memory works.

"I like giving the example of searching for your car keys," says computer scientist Joshua Hailpern at the University of Illinois at Urbana-Champaign, who started to develop YouPivot while at Google in 2010. "You are always told, ’Think about where you last had them.’ When you mentally retrace your steps, you may realize that when you were making your shopping list, you left your keys in the fridge next to the milk."

Hailpern says that YouPivot allows you to search your digital activity in the same way. For instance, it allows you to ask, "What was that website or PDF I was looking at when AC/DC’s "Hell’s Bells" was last playing?".



Essentially, YouPivot keeps track of everything you were doing on the computer at any given moment. This allows you to reconstruct your past environment so you can then browse or "pivot [PDF]" around to see what else was going on and hopefully rediscover what you are looking for. This could be especially useful as people are inundated with more and more data, perhaps too much to otherwise easily recall. This strategy does not rely only on specific keywords or dates like when you search through old e-mails or a browser history—it can help you find something even if you have only a faint hint of what else it might be linked to.

"Ideally, the YouPivot system would be built as part of an OS such as Apple OS X or Google Chrome or Windows," Hailpern says. "That tight coupling would let YouPivot move fast and gather more data."

YouPivot stores only a few kilobytes of metadata on each file, not the files themselves or their content, keeping its footprint low, Hailpern notes. Users can also keep the majority of this information in the cloud, leaving almost no footprint on their own devices.

A related system Hailpern and colleagues have developed called TimeMarks, which can work in conjunction with YouPivot, allows users to bookmark such points in time. "You tell the computer, ’This moment is important,’ " Hailpern says. "You can then see all the websites, files, and activity you had going on at that time. Then you can look a bit earlier or later."

So far, YouPivot and TimeMarks are free extensions available for Google Chrome’s Web browser in alpha versions. They are designed to draw on cloud-based data, allowing you to search your history on all of your computers, regardless of location. Hailpern and his colleagues are working on a full release for spring 2012.

A groundbreaking predecessor of YouPivot was a project at Microsoft Research called MyLifeBits, which began in 2001. It sought to capture virtually every iota of knowledge of a person’s life, "from heartbeats to frames of an on-body camera to what TV programs I watched to all past records," says computer pioneer Gordon Bell, a principal researcher at Microsoft Research.

MyLifeBits project leader Jim Gemmell notes in Your Life, Uploaded, which he wrote with Bell, that "the information about your life becomes vastly more useful when data is viewed in a unified way, rather than being separated into silos for e-mail, pictures, and so forth. One way it becomes more useful is by making finding things easier by ’pivoting’ from one data type to another."

"We are excited to see YouPivot bring pivoting ideas previously only found in research demos to market and predict that this space will see tremendous growth in this decade," says Gemmell, a senior researcher at Microsoft Research, who did not work on YouPivot.

The real challenge YouPivot faces is dealing with privacy, Hailpern cautions. "As a system gathers more and more data, people get worried, and rightly so," he says. "YouPivot itself does not try to determine what you are doing or track your activity, but the challenges of privacy in a Web-based system are an important concern." To take YouPivot to the next level, Hailpern says they would need to address methods for ensuring privacy—an issue facing most major products and websites.

About the Author

Charles Q. Choi is a science writer based in New York City. His reporting has appeared in The New York Times, Scientific American, and Wired, among other publications. In September he reported on a brain computer interface based on ultrasound.

This article is for IEEE members only. Join IEEE to access our full archive.

Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, podcasts, and special reports. Learn more →

If you're already an IEEE member, please sign in to continue reading.

Membership includes:

  • Get unlimited access to IEEE Spectrum content
  • Follow your favorite topics to create a personalized feed of IEEE Spectrum content
  • Save Spectrum articles to read later
  • Network with other technology professionals
  • Establish a professional profile
  • Create a group to share and collaborate on projects
  • Discover IEEE events and activities
  • Join and participate in discussions