Close

The Neural Network That Remembers

With short-term memory, recurrent neural networks gain some amazing abilities

12 min read
The Neural Network That Remembers
Illustration: John Hersey

“On tap at the brewpub. A nice dark red color with a nice head that left a lot of lace on the glass. Aroma is of raspberries and chocolate. Not much depth to speak of despite consisting of raspberries. The bourbon is pretty subtle as well. I really don’t know that find a flavor this beer tastes like. I would prefer a little more carbonization to come through. It’s pretty drinkable, but I wouldn’t mind if this beer was available.”

Besides the overpowering bouquet of raspberries in this guy’s beer, this review is remarkable for another reason. It was produced by a computer program instructed to hallucinate a review for a “fruit/vegetable beer.” Using a powerful artificial-intelligence tool called a recurrent neural network, the software that produced this passage isn’t even programmed to know what words are, much less to obey the rules of English syntax. Yet, by mining the patterns in reviews from the barflies at BeerAdvocate.com, the program learns how to generate similarly coherent (or incoherent) reviews.

Keep reading... Show less

Stay ahead of the latest trends in technology. Become an IEEE member.

This article is for IEEE members only. Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, podcasts, and special reports. Learn more →

Membership includes:

  • Get unlimited access to IEEE Spectrum content
  • Follow your favorite topics to create a personalized feed of IEEE Spectrum content
  • Save Spectrum articles to read later
  • Network with other technology professionals
  • Establish a professional profile
  • Create a group to share and collaborate on projects
  • Discover IEEE events and activities
  • Join and participate in discussions

Meta Aims to Build the World’s Fastest AI Supercomputer

The AI Research SuperCluster could help the company develop real-time voice translations

3 min read
A brightly lit, high-ceilinged room with rows of silvery-black cabinets and yellow pipes near the ceiling.

Meta's new AI supercomputer.

Meta

Meta, parent company of Facebook, says it has built a research supercomputer that is among the fastest on the planet. By the middle of this year, when an expansion of the system is complete, it will be the fastest, Meta researchers Kevin Lee and Shubho Sengupta write in a blog post today. The AI Research SuperCluster (RSC) will one day work with neural networks with trillions of parameters, they write. The number of parameters in neural network models have been rapidly growing. The natural language processor GPT-3, for example, has 175 billion parameters, and such sophisticated AIs are only expected to grow.

RSC is meant to address a critical limit to this growth, the time it takes to train a neural network. Generally, training involves testing a neural network against a large data set, measuring how far it is from doing its job accurately, using that error signal to tweak the network’s parameters, and repeating the cycle until the neural network reaches the needed level of accuracy. It can take weeks of computing for large networks, limiting how many new networks can be trialed in a given year. Several well-funded startups, such as Cerebras and SambaNova, were launched in part to address training times.

Keep Reading ↓ Show less

Rooftop Drones for Autonomous Pigeon Harassment

Have invasive flying rats met their match?

3 min read
A pigeon on a poop covered wooden plank.
iStock photo

Feral pigeons are responsible for over a billion dollars of economic losses here in the United States every year. They’re especially annoying because the species isn’t native to this country—they were brought over from Europe (where they’re known as rock doves and are still quite annoying) because you can eat them, but enough of the birds escaped and liked it here that there are now stable populations all over the country, being gross.

In addition to carrying diseases (some of which can occasionally infect humans), pigeons are prolific and inconvenient urban poopers, deploying their acidic droppings in places that are exceptionally difficult to clean. Rooftops, as well as ledges and overhangs on building facades, are full of cozy nooks and crannies, and despite some attempts to brute-force the problem by putting metal or plastic spikes on every horizontal surface, there are usually more surfaces (and pigeons) than can be reasonably bespiked.

Researchers at EPFL in Switzerland believe that besting an aerial adversary requires an aerial approach, and so they’ve deployed an autonomous system that can identify roof-invading pigeons and then send a drone over to chase them away.

Keep Reading ↓ Show less

Learn How to Use a High-Performance Digitizer

Join Teledyne for a three-part webinar series on high-performance data acquisition basics

1 min read

Webinar: High-Performance Digitizer Basics

Part 3: How to Use a High-Performance Digitizer

Date: Tuesday, December 7, 2021

Time: 10 AM PST | 1 PM EST

Keep Reading ↓ Show less