How Machine Learning Is Reinventing the Way We Discover Music

Software engineer at Pandora explains how it analyzes hundreds of data points to recommend artists and predict hits

3 min read
Photo: Andrew Harrer/Bloomberg/Getty Images
Photo: Andrew Harrer/Bloomberg/Getty Images

THE INSTITUTEIf you’re a folk rock fan like me, you might be surprised when a music-streaming service suggests songs for you in other genres, such as country or reggae—and you actually enjoy them. Apple Music, Pandora, Spotify, and similar services are taking subtle cues from your listening habits, not only to recommend new artists but also to create personalized playlists for you with tracks you’ve likely never heard before.

To understand the technology making that possible, The Institute spoke with IEEE Member Josh Perline, a software engineer at Pandora. Perline is responsible for content personalization, which involves using data analytics and machine learning to automate the creation of suggested playlists.

As one of the first music-streaming services on the market, Pandora has some 17 years of data to work with. The company relies on data from its Music Genome Project, an undertaking that analyzes hundreds of songs’ musical attributes including which instruments are featured and the range of notes and vocal sounds.

“We have the same tracks as every other provider, but what keeps people coming back to us is the strength of our recommendations to help them discover new music,” Perline says.


Say you’re listening to Pandora’s Adele station. It not only plays the songs of the Grammy award winner; it plays artists with a similar sound, as well as artists from decades ago who influenced the singer-songwriter’s music, Perline notes.

What differentiates Pandora, he says, is the human input that informs its algorithm on how to recommend music. The algorithm, based on machine learning, makes decisions by incorporating listeners’ preferences as well as information about the songs. Staff members working on Pandora’s Music Genome Project have backgrounds in musicology—the scholarly analysis of music. They assign attributes to the songs—some of which can have as many as 450 unique traits. That can include the emotions the lyrics convey, such as angry, sad, or joyful; each instrument in the recording and whether they’re synthesized or acoustic; to what degree the vocals are nasal-sounding; and sound effects.

The metadata allows the algorithm to get much more precise. It not only classifies a song as rap, for example, but that of its sub-genre such as proto-industrial hip-hop, a fusion of industrial music and spoken words combined with hip-hop rhythms and vocals. “It gets extremely nuanced, down to even working from the liner notes,” Perline says, referring to information such as the artist’s biography and the lyrics.

The service can also tweak recommendations based on listeners’ feedback. When people hear a song, they can tap a “thumbs up” or “thumbs down” icon. If “thumbs down” is tapped the playlist skips to the next song, and Pandora likely won’t play that number or similar ones again. In fact, the algorithm pays attention to everything listeners do, including which stations they save and listen to most. That way, when a new artist releases an album the algorithm determines listeners might like, Pandora will recommend the music to them, Perline says.


The data the company collects is helpful not only to listeners but to artists as well. Its Artist Marketing Platform gives the musicians free insights on, for example, how many people have listened to their songs and for how long. They can submit new music through the platform, with the potential for it to be played on Pandora.

Pandora in 2015 bought Next Big Sound, a company that provides analytics not only from its own service but also from third-party sources including social media platforms, radio stations, and video sites such as YouTube. Artists can access Next Big Sound to analyze the data and strategize on ways to reach their fans. Data showing geographic areas with concentrations of fans could be used to plan a tour, for example. In fact, Perline says, such analytics have prompted bands to perform in towns they otherwise might have skipped.


Next Big Sound has used its data to make accurate predictions as well. It determined several artists who would win Grammy awards, for instance. Data about the number of music downloads and social media shares can help inform the predictions.

Next Big Sound also can predict the next big hit or emerging artist before the music goes mainstream. That happened with rapper Lil Yachty, months before he first hit the Billboard Hot 100 chart. Such predictions are made possible in part when data shows how fast and from how many sources music has been shared. If, for example, a YouTube music video uploaded by an unknown artist with only two dozen subscribers to his channel gets 5,000 hits in an hour, “We can see something is hot,” Perline says. In other words, the algorithm can spot the activity a song is receiving and whether this surpasses the traffic it’s expected to get.

Perline sees music-streaming services as leveling the playing field for independent and undiscovered artists by giving them exposure even if they don’t have a record deal. “I think that’s the direction music is going in,” he says, “with more self-made artists making their way to mainstream culture.

“Millions of tracks are available. Our job is to get the right ones to you.”

This article is for IEEE members only. Join IEEE to access our full archive.

Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, podcasts, and special reports. Learn more →

If you're already an IEEE member, please sign in to continue reading.

Membership includes:

  • Get unlimited access to IEEE Spectrum content
  • Follow your favorite topics to create a personalized feed of IEEE Spectrum content
  • Save Spectrum articles to read later
  • Network with other technology professionals
  • Establish a professional profile
  • Create a group to share and collaborate on projects
  • Discover IEEE events and activities
  • Join and participate in discussions