Update 29 March 2023: Since this story ran nearly two-and-a-half years ago, Google has in fact posted an entire blog about how hum-to-search works on their system. “In contrast to existing methods,” they wrote, “This approach produces an embedding of a melody from a spectrogram of a song without generating an intermediate representation. This enables the model to match a hummed melody directly to the original (polyphonic) recordings without the need for a hummed or MIDI version of each track or for other complex hand-engineered logic to extract the melody.”
This direct-to-digitization approach appears to be shared with a number of other digital products and related innovations researchers have rolled out (or proposed) in the intervening time since our original story ran. Last November, for instance, researchers from the International Institute of Information Technology in Hyderabad, India proposed their own music meta-search tool that could pin down lyrics, metadata, and the original song itself based on hummed input. Meanwhile, Vietnamese researchers waxed a little more first-principles about the challenge: “Recognizing a song name based on humming sound is not an easy task for a human being and should be done by machines,“ they wrote. ”However, there is no research paper published about hum tune recognition.” Which they proceeded to do, basing their work in part on the results of an online competition to develop code that can better reduced a hummed tune to its most searchable sonic components.
Ever have a song you can’t remember the name of, nor any of its words? Now Google has a new feature where you can simply hum the melody and it can hopefully name that tune.
The idea of identifying songs through singing, humming or whistling instead of lyrics is not a new idea—the music app SoundHound has possessed hum-to-search for at least a decade. Google’s new feature should help the search engine with the many requests it receives to identify music.
Aparna Chennapragada, a Google vice president who introduced the new feature during a streamed event Oct. 15, said people ask Google “what song is playing” nearly 100 million times each month.
To use the new feature on a mobile device, open the latest version of the Google app or find the Google Search widget. Tap on the mic icon and say “what’s this song?” or click the “Search a song” button. Then start humming for 10 to 15 seconds. On Google Assistant, say, “Hey Google, what’s this song?” and then hum the tune. Perfect pitch is not needed.
The new feature is based on machine learning models that analyze each hum, whistle or singing and remove details such as accompanying instruments and the voice’s timbre and tone. They next compare the melody to thousands of songs from around the world.
The feature will show users a list of the most likely songs based on the melody. They can then select a match, explore information on the song and artist, view any accompanying music videos or listen to the song, find the lyrics, or check out other recordings of the song if they are available.
“It could certain help connect musical artists and the music industry with customers,” says Chris Rodgers, CEO and founder of Colorado SEO Pros. “In the music creation process, musicians might come up with amazing ideas, and it’ll turn out those came from something they heard and replayed in their mind one-hundred times and then thought it was their own brilliant idea. So maybe this new feature could almost be a way to do an [intellectual property] check. ‘I’ve got this amazing song, but is it really similar to something else out there?’”
Moreover, a user might hear a jingle in a commercial or some message from social media and want to identify those melodies. “I can see Google try to monetize that opportunity like they try to monetize everything,” Rodgers says.
The new feature is currently available in English on iOS, and in more than 20 languages on Android. Google plans to expand it to more languages in the future.
“It’s a cool feature. I don’t think it has big commercial applications at this point, but I do think it helps the Google brand,” Rodgers says.
One concern with this new feature is that Google may use such technology to covertly identify people by the sounds of their voices. “We know the technology is already there for the big tech companies to turn on receivers in phones,” Rodgers says. “And there’s a lot of anecdotal evidence that some of them may be listening to you. Facebook has denied this up and down, but I myself have anecdotal evidence.”
“These are hard questions none of us have the answers to,” Rodgers says. “We’re all navigating a world of variables and unknowns hoping these companies have our best interests in mind while trying to enjoy the quote-unquote free technology they have, but at the end of the day, we know it comes at some kind of price.”
Charles Q. Choi is a science reporter who contributes regularly to IEEE Spectrum. He has written for Scientific American, The New York Times, Wired, and Science, among others.