Deep Learning at the Speed of Light

Lightmatter bets that optical computing can solve AI’s efficiency problem

4 min read
Vertical
Image of colorful circles splitting in two directions.
Illustration: Greg Mably
Purple

In 2011, Marc Andreessen, ­general partner of venture capital firm Andreessen Horowitz, wrote an influential article in The Wall Street Journal titled,“Why Software Is Eating the World." A decade later now, it's deep learning that's eating the world.

Deep learning, which is to say artificial neural networks with many hidden layers, is regularly stunning us with solutions to real-world problems. And it is doing that in more and more realms, including natural-language processing, fraud detection, image recognition, and autonomous driving. Indeed, these neural networks are getting better by the day.

But these advances come at an enormous price in the computing resources and energy they consume. So it's no wonder that engineers and computer scientists are making huge efforts to figure out ways to train and run deep neural networks more efficiently.

An ambitious new strategy that's coming to the fore this year is to perform many of the required mathematical calculations using photons rather than electrons. In particular, one company, ­Lightmatter, will begin marketing late this year a neural-network accelerator chip that calculates with light. It will be a refinement of the prototype Mars chip that the company showed off at the virtual Hot Chips conference last August.

While the development of a commercial optical accelerator for deep learning is a remarkable accomplishment, the general idea of computing with light is not new. Engineers regularly resorted to this tactic in the 1960s and '70s, when electronic digital computers were too feeble to perform the complex calculations needed to process synthetic-aperture radar data. So they processed the data in the analog domain, using light.

Image of Lightmatter's prototype board.Plug and Play: Lightmatter's prototype board uses a normal PCI bus.Photo: Lightmatter

Because of the subsequent Moore's Law gains in what could be done with digital electronics, optical computing never really caught on, despite the ascendancy of light as a vehicle for data communications. But all that may be about to change: Moore's Law may be nearing an end, just as the computing demands of deep learning are exploding.

There aren't many ways to deal with this problem. Deep-learning researchers may develop more efficient algorithms, sure, but it's hard to imagine those gains will be sufficient. “I challenge you to lock a bunch of theorists in a room and have them come up with a better algorithm every 18 months," says Nicholas Harris, CEO of ­Lightmatter. That's why he and his colleagues are bent on “developing a new compute technology that doesn't rely on the transistor."

So what then does it rely on?

The fundamental component in Lightmatter's chip is a Mach-Zehnder interferometer. This optical device was jointly invented by Ludwig Mach and Ludwig Zehnder in the 1890s. But only recently have such optical devices been miniaturized to the point where large numbers of them can be integrated onto a chip and used to perform the matrix multiplications involved in neural-network calculations.

Keren Bergman, a professor of electrical engineering and the director of the Lightwave Research Laboratory at Columbia University, in New York City, explains that these feats have become possible only in the last few years because of the maturing of the manufacturing ecosystem for integrated photonics, needed to make photonic chips for communications. “What you would do on a bench 30 years ago, now they can put it all on a chip," she says.

Image of Lightmatter's integrated Mach-Zehnder interferometer simulation.Lightpipe: Lightmatter uses an integrated Mach-Zehnder interferometer, part of which is simulated here.Gif: Lightmatter

Processing analog signals carried by light slashes energy costs and boosts the speed of calculations, but the precision can't match what's possible in the digital domain. “We have an 8-bit-equivalent system," say Harris. This limits his company's chip to neural-network inference calculations—the ones that are carried out after the network has been trained. Harris and his colleagues hope their technology might one day be applied to training neural networks, too, but training demands more precision than their optical processor can now provide.

Lightmatter is not alone in the quest to harness light for ­neural-network calculations. Other startups working along these lines include Fathom Computing, ­LightIntelligence, LightOn, ­Luminous, and Optalysis. One of these, Luminous, hopes to apply optical computing to spiking neural networks, which take advantage of the way the neurons of the brain process information—­perhaps accounting for why the human brain can do the remarkable things it does using just a dozen or so watts.

Luminous expects to develop practical systems sometime between 2022 and 2025. So we'll have to wait a few years yet to see what pans out with its approach. But many are excited about the prospects, including Bill Gates, one of the company's high-profile investors.

It's clear, though, that the computing resources being dedicated to artificial-intelligence systems can't keep growing at the current rate, doubling every three to four months. Engineers are now keen to harness integrated photonics to address this challenge with a new class of computing machines that are dramatically different from conventional electronic chips yet are now practical to manufacture. Bergman boasts: “We have the ability to make devices that in the past could only be imagined."

This article appears in the January 2021 print issue as “Deep Learning at the Speed of Light."

The Conversation (0)

The EV Transition Explained: Can the Grid Cope?

Palo Alto offers a glimpse at the challenges municipalities and utilities face

8 min read
A man plugging a charger into an outlet

Enel’s JuiceBox 240-volt Level 2 charger for electric vehicles.

Enel X Way USA

There have been vigorous debates pro and con in the United States and elsewhere over whether electric grids can support EVs at scale. The answer is a nuanced “perhaps.” It depends on several factors, including the speed of grid-component modernization, the volume of EV sales, where they occur and when, what kinds of EV charging are being done and when, regulator and political decisions, and critically, economics.

The city of Palo Alto, Calif. is a microcosm of many of the issues involved. Palo Alto boasts the highest adoption rate of EVs in the United States: In 2020, one in six of the town’s 25,000 households owned an EV. Of the 52,000 registered vehicles in the city, 4,500 are EVs, and on workdays, commuters drive another 3,000 to 5,000 EVs to enter the city. Residents can access about 1,000 charging ports spread over 277 public charging stations, with another 3,500 or so charging ports located at residences.

Keep Reading ↓Show less

The James Webb Space Telescope was a Career-Defining Project for Janet Barth

NASA’s first female engineering chief was there from conception to first light

5 min read
portrait of older woman in light blue jacket against dark gray background Info for editor if needed:
Sue Brown

Janet Barth spent most of her career at the Goddard Space Flight Center, in Greenbelt, Md.—which put her in the middle of some of NASA’s most exciting projects of the past 40 years.

She joined the center as a co-op student and retired in 2014 as chief of its electrical engineering division. She had a hand in Hubble Space Telescope servicing missions, launching the Lunar Reconnaissance Orbiter and the Magnetospheric Multiscale mission, and developing the James Webb Space Telescope.

Keep Reading ↓Show less

Learn How Global Configuration Management and IBM CLM Work Together

In this presentation we will build the case for component-based requirements management

2 min read

This is a sponsored article brought to you by 321 Gang.

To fully support Requirements Management (RM) best practices, a tool needs to support traceability, versioning, reuse, and Product Line Engineering (PLE). This is especially true when designing large complex systems or systems that follow standards and regulations. Most modern requirement tools do a decent job of capturing requirements and related metadata. Some tools also support rudimentary mechanisms for baselining and traceability capabilities (“linking” requirements). The earlier versions of IBM DOORS Next supported a rich configurable traceability and even a rudimentary form of reuse. DOORS Next became a complete solution for managing requirements a few years ago when IBM invented and implemented Global Configuration Management (GCM) as part of its Engineering Lifecycle Management (ELM, formerly known as Collaborative Lifecycle Management or simply CLM) suite of integrated tools. On the surface, it seems that GCM just provides versioning capability, but it is so much more than that. GCM arms product/system development organizations with support for advanced requirement reuse, traceability that supports versioning, release management and variant management. It is also possible to manage collections of related Application Lifecycle Management (ALM) and Systems Engineering artifacts in a single configuration.

Keep Reading ↓Show less