The July 2022 issue of IEEE Spectrum is here!

Close bar

Deep Learning at the Speed of Light

Lightmatter bets that optical computing can solve AI’s efficiency problem

4 min read
Image of colorful circles splitting in two directions.
Illustration: Greg Mably

In 2011, Marc Andreessen, ­general partner of venture capital firm Andreessen Horowitz, wrote an influential article in The Wall Street Journal titled,“Why Software Is Eating the World." A decade later now, it's deep learning that's eating the world.

Deep learning, which is to say artificial neural networks with many hidden layers, is regularly stunning us with solutions to real-world problems. And it is doing that in more and more realms, including natural-language processing, fraud detection, image recognition, and autonomous driving. Indeed, these neural networks are getting better by the day.

But these advances come at an enormous price in the computing resources and energy they consume. So it's no wonder that engineers and computer scientists are making huge efforts to figure out ways to train and run deep neural networks more efficiently.

An ambitious new strategy that's coming to the fore this year is to perform many of the required mathematical calculations using photons rather than electrons. In particular, one company, ­Lightmatter, will begin marketing late this year a neural-network accelerator chip that calculates with light. It will be a refinement of the prototype Mars chip that the company showed off at the virtual Hot Chips conference last August.

While the development of a commercial optical accelerator for deep learning is a remarkable accomplishment, the general idea of computing with light is not new. Engineers regularly resorted to this tactic in the 1960s and '70s, when electronic digital computers were too feeble to perform the complex calculations needed to process synthetic-aperture radar data. So they processed the data in the analog domain, using light.

Image of Lightmatter's prototype board.Plug and Play: Lightmatter's prototype board uses a normal PCI bus.Photo: Lightmatter

Because of the subsequent Moore's Law gains in what could be done with digital electronics, optical computing never really caught on, despite the ascendancy of light as a vehicle for data communications. But all that may be about to change: Moore's Law may be nearing an end, just as the computing demands of deep learning are exploding.

There aren't many ways to deal with this problem. Deep-learning researchers may develop more efficient algorithms, sure, but it's hard to imagine those gains will be sufficient. “I challenge you to lock a bunch of theorists in a room and have them come up with a better algorithm every 18 months," says Nicholas Harris, CEO of ­Lightmatter. That's why he and his colleagues are bent on “developing a new compute technology that doesn't rely on the transistor."

So what then does it rely on?

The fundamental component in Lightmatter's chip is a Mach-Zehnder interferometer. This optical device was jointly invented by Ludwig Mach and Ludwig Zehnder in the 1890s. But only recently have such optical devices been miniaturized to the point where large numbers of them can be integrated onto a chip and used to perform the matrix multiplications involved in neural-network calculations.

Keren Bergman, a professor of electrical engineering and the director of the Lightwave Research Laboratory at Columbia University, in New York City, explains that these feats have become possible only in the last few years because of the maturing of the manufacturing ecosystem for integrated photonics, needed to make photonic chips for communications. “What you would do on a bench 30 years ago, now they can put it all on a chip," she says.

Image of Lightmatter's integrated Mach-Zehnder interferometer simulation.Lightpipe: Lightmatter uses an integrated Mach-Zehnder interferometer, part of which is simulated here.Gif: Lightmatter

Processing analog signals carried by light slashes energy costs and boosts the speed of calculations, but the precision can't match what's possible in the digital domain. “We have an 8-bit-equivalent system," say Harris. This limits his company's chip to neural-network inference calculations—the ones that are carried out after the network has been trained. Harris and his colleagues hope their technology might one day be applied to training neural networks, too, but training demands more precision than their optical processor can now provide.

Lightmatter is not alone in the quest to harness light for ­neural-network calculations. Other startups working along these lines include Fathom Computing, ­LightIntelligence, LightOn, ­Luminous, and Optalysis. One of these, Luminous, hopes to apply optical computing to spiking neural networks, which take advantage of the way the neurons of the brain process information—­perhaps accounting for why the human brain can do the remarkable things it does using just a dozen or so watts.

Luminous expects to develop practical systems sometime between 2022 and 2025. So we'll have to wait a few years yet to see what pans out with its approach. But many are excited about the prospects, including Bill Gates, one of the company's high-profile investors.

It's clear, though, that the computing resources being dedicated to artificial-intelligence systems can't keep growing at the current rate, doubling every three to four months. Engineers are now keen to harness integrated photonics to address this challenge with a new class of computing machines that are dramatically different from conventional electronic chips yet are now practical to manufacture. Bergman boasts: “We have the ability to make devices that in the past could only be imagined."

This article appears in the January 2021 print issue as “Deep Learning at the Speed of Light."

The Conversation (0)

This Wearable Neck Patch Can Diagnose Concussions

Self-powered sensors convert neck strain into electrical pulses to detect head trauma in athletes

4 min read
image of back of man's head and shoulders with a patch taped to his lower neck; right image is a time lapse image of a man's head extending far forward and back, simulating a case of whiplash

The prototype patch in this research is shown in (a) on the left; on the right (b) is the kind of head rotation that can yield an electrical response from the patch.

Juan Pastrana

Nelson Sepúlveda was sitting in the stands at Spartan Stadium, watching his hometown Michigan State players bash heads with their cross-state football rivals from the University of Michigan, when he had a scientific epiphany.

Perhaps the nanotechnologies he had been working on for years—paper-thin devices known as ferroelectret nanogenerators that convert mechanical energy into electrical energy—could help save these athletes from the ravages of traumatic brain injury.

Keep Reading ↓Show less

Video Friday: PoKeBo Cubes

Your weekly selection of awesome robot videos

2 min read
A young girl looks at a cluster of three simple robots facing each other on a table

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

RoboCup 2022: 11 July–17 July 2022, BANGKOK
IEEE CASE 2022: 20 August–24 August 2022, MEXICO CITY
CLAWAR 2022: 12 September–14 September 2022, AZORES, PORTUGAL
ANA Avatar XPRIZE Finals: 4 November–5 November 2022, LOS ANGELES
CoRL 2022: 14 December–18 December 2022, AUCKLAND, NEW ZEALAND

Enjoy today’s videos!

Keep Reading ↓Show less

Harnessing the Power of Innovation Intelligence

Through case studies and data visualizations, this webinar will show you how to leverage IP and scientific data analytics to identify emerging business opportunities

1 min read

Business and R&D leaders have to make consequential strategic decisions every day in a global marketplace that continues to get more interconnected and complex. Luckily, the job can be more manageable and efficient by leveraging IP and scientific data analytics. Register for this free webinar now!

Join us for the webinar, Harnessing the power of innovation intelligence, to hear Clarivate experts discuss how analyzing IP data, together with scientific content and industry-specific data, can provide organization-wide situational awareness and reveal valuable business insights.

Keep Reading ↓Show less