Tech Talk iconTech Talk

Amazing New Motion Capture Tech Makes Games Look Like Films

Check out this amazing new video on YouTube.  It's a short clip on the making of the upcoming video game, L.A. Noire.  Specifically, it focuses on a new motion capture technology called MotionScan, which creates the most lifelike scenes yet. 

Loading the video player...

Brendan McNamara, lead developer  for L A Noire, has said that “We’re definitely blurring the lines now. I want this game to be the flashpoint where people start to think of games and film as being on the same level, because I’m confident they already are.”  It's hard to argue after seeing the footage.  The company behind MotionScan is depth Analysis, based in Sydney, Australia.   Depth Analysis announced the innovation earlier this year, and gave a few hints about how it works. 

"MotionScan uses 32 High Definition cameras to capture true-to-life three-dimensional performances at up to 30 frames per second," the company revealed. "Capable of capturing up to 50 minutes of final footage and processing up to 20 minutes of facial animation automatically per day, the technology revolutionizes traditional motion-capture and post-production animation. MotionScan records every emotional detail, mannerism, and facial nuance accurately frame by frame as 3D models.  No markers or phosphorescent paint needs to be applied to the actors at the time of recording, and no manpower is required to clean up data and animate the finer details by hand after the shoot. For directors and cinematographers, an additional advantage of MotionScan is the ability to view an actor’s performance from any angle and re-light in any way from one take without the need for multiple camera and lighting setups that quickly drain production time and budgets."

This comes at a momentous time for motion-capture innovation.  Microsoft, of course, recently rolled out the Kinect motion-sensing camera for the Xbox 360.  The Kinect is sort of the DIY version of MotionScan, letting gamers transport themselves into the action.  It'll be interesting to see how this increased realism impacts game development in the coming year.  Perhaps most significantly, it may lead more A-List Hollywood actors into games.   After all, as one can see by the LA Noire footage (which features an actor from the hit show Mad Men) there's more "acting" that can actually come through now given the mocap precision.

LA Noire, which is made by Rockstar Games, will likely be a breakthrough title, ushering in a new era of cinematic game play.  With so much attention focused now on smaller, social games like Farmville, cinematic epics are primed for reinvention.   I have no doubt that while gamers may be spending more time on iPhones, there's always an appetite for big brash immersive epics like LA Noire.  

Electronic Cigarettes: Unregulated and Untested

The U.S. Appeals court in Washington D.C. ruled last Tuesday that the Food and Drug Administration lacks the authority to regulate electronic cigarettes as either a drug or a device, leaving the FDA with only as much regulatory power as it wields over tobacco products. The ruling came just days after researchers in California published a study detailing safety and marketing concerns with every brand of e-cigarettes.

E-cigarettes are designed to look like tobacco cigarettes, but they deliver hits of nicotine vapor. When the smoker draws on the mouthpiece, air passes through and activates a battery-powered atomizer that vaporizes liquid nicotine from a disposable cartridge. Manufacturers market them, in part, as a safer alternative to smoking (one company calls its product the "Safe Cig"), and as an aid to quitting.

These claims, however, have never been established to the satisfaction of the FDA. And in April, the agency moved to ban imports of e-cigarettes into the country until companies gained approval for the devices, a process that would require rigorous proof of their safety and efficacy. Nicotine patches and nicotine gum were held to similar standards before eventually gaining approval.

In turn, Sottera Inc., a company that makes and imports e-cigarettes under the name NJOY sued the FDA, claiming that the gadget should be classified as a tobacco product, although it contains no tobacco.

Just days before the ruling, a group of researchers in California published a study calling into question the safety of e-cigarettes. The group claims that the toxicity level of vaporized nicotine is virtually unstudied. They also found various design flaws with the devices themselves, including leaky cartridges and insufficient labeling. Another study in January warned that e-cigarettes may not be any healthier than tobacco cigarettes. 

The debate has attracted passionate opinions in the blogosphere. For now, at least, the courts have resolved the issue in favor of e-smokers.

WikiLeaks Demonstrates Web Resiliency

If you’ve been following the news lately, you’ve probably heard that WikiLeaks has been feeling the heat after publishing confidential cables between the U.S. State Department and its overseas missions. But many of the technical details behind these current events might be confusing.

For example, almost everyone knows by now that WikiLeaks had been using Amazon’s Web Services to host its Website. It chose to use Amazon apparently because its prior Web hosts became subject to a denial-of-service (DOS) attack. But what exactly does that mean?

Such attacks have many variations. The archetypal example involves the use of as many as several million individual computers spread throughout the Internet. At some point, the people operating those computers inadvertently downloaded and installed software that allowed their computers to be manipulated surreptitiously from elsewhere. These computers then became part of a “botnet,” which some distant master could then activate at a later time for malicious purposes.

The compromised computers could, for example, send a connection request to the servers of the targeted Website. The Website’s file servers are, of course, configured to establish such connections. But the attacker purposefully arranges things so the connections are requested but never completed. If the servers are inundated with too many such requests, they will not be able to service the legitimate requests for connections from people who genuinely want to visit the site.

The assault on WikiLeaks.org differed from this scenario, though. It was a denial-of-service attack, but not a distributed one. It appears that an online vigilante who calls himself “The Jester," using a single computer running software tool of his own making, targeted WikiLeaks.org and succeeded in temporarily silencing it.

When this happened to WikiLeaks, it decided to use Amazon Web Services. Amazon soon booted it off its servers, though, ostensibly because WikiLeaks violated Amazon’s terms of service, although the impetus may have been questioning Amazon received  from the staff of Senator Joe Lieberman (I-Conn.), chairman of the senate’s Homeland Security and Governmental Affairs Committee.

WikiLeaks’ troubles keeping its Website online didn’t end there. Next up was the notice it got from EveryDNS.net that it was to be terminated. EveryDNS.net provides domain name system (DNS) lookups. WikiLeaks had been using this company’s name servers to translate the human-readable “WikiLeaks.org” into the numerical code—the Internet Protocol or IP address—needed to find the Website’s servers on the Internet.

Actually, if you had tried to surf over to WikiLeaks.org before this happened, it’s unlikely that your computer would have queried EveryDNS.net’s name servers to get such a translation. Your Internet Service Provider (ISP) probably had WikiLeak.org’s IP address already stored on its own name servers. That’s the usual way things work. But the information on your ISP’s name servers gets periodically refreshed from the authoritative sources that each Website owner specifies. So at some point, your ISP’s computers would need to consult EveryDNS.net to find out how to route its customers to WikiLeaks.org’s servers. EveryDNS.net itself became subject to a DDOS attack, at which point it decided that it couldn’t responsibly keep trying to service WikiLeaks, which would have threatened its ability to provide similar service to other Websites.

How did WikiLeaks fare after all that? Its response to Amazon ousting it was simply to shift Website hosting to OVH in France and Bahnhof in Sweden (which it had used previously). It found alternative name servers, too, 13 of them at this writing. And it began using the WikiLeaks.ch domain name instead of WikiLeaks.org, for reasons that are not entirely clear.

One possibility is that WikiLeaks felt jittery about further name-server disruptions. To understand why that might be the case, you should know that in 2008, a U.S. District Court handling a civil suit against WikiLeaks issued a order to Dynadot, the US company that registered the WikiLeaks.org domain name. A domain-name registrar acts as an intermediary between the person or organization that sets up the domain name and the company that maintains and operates the relevant top-level domain’s name servers—here the .org name servers.

The idea was to force WikiLeaks offline by coercing its domain-name registrar to delete all records of it existing. That court order was quickly reversed and the suit dropped, but the threat of such actions was probably still weighing on WikiLeaks. So you can understand why at this point it might not want to depend on any company in the United States for hosting, maintaining its domain-name servers, or even for registering its domain name. But why switch from WikiLeaks.org to WikiLeaks.ch?

That’s still a bit of a mystery, but I can speculate. First I have to confess that my earlier description of what happens when your ISP needs to update its name server didn’t tell the whole story. In fact, the ISP’s name server starts by sending a query to a name server that handles the relevant top-level domain, .org in the case of WikiLeaks.org. The .org name server then provides an IP address for the name server that knows the IP address of WikiLeaks.org.

The name servers for the .org top-level domain are run by a U.S. company called the Public Interest Registry. In theory, the Public Interest Registry could configure the .org name servers not to point to the name servers for WikiLeaks.org. That seems a remote possibility, but you can certainly understand why WikiLeaks might want to avoid any such vulnerability. Switching to WikiLeaks.ch means that the name servers that begin the process of translating its domain name into an IP address are controlled from Switzerland rather than the United States. That better insulates WikiLeaks from the influence of U.S. courts or government agents.

What’s more, a Google search of “WikiLeaks” now turns up a numerical IP address for WikiLeaks.ch, perhaps because so many others have linked directly to this IP address. So it’s difficult to see how any attack on or suppression of name servers could at this point cut off the site.

And even if WikiLeaks itself were to go offline, the information it has put out would hardly be suppressed. Unlike most other publishers, which guard their offerings, WikiLeaks has urged others to copy its content, giving detailed instructions for doing so on its Website. This creates what are called “mirrors”—multiple sites with different names in different places that all look and work like WikiLeaks.ch. As of 10 December, WikiLeaks.ch shows that there are 1559 mirror sites scattered around the globe, a number that grows day by day.

It just goes to show, I suppose, what we’ve all known for a long time: For good or bad, once someone pours a bunch of juicy information into cyberspace, there’s no putting it back in the bottle.

Unmasking the "Anonymous" Hackers

This week, there has been a lot of breathless press surrounding Wikileaks cyber-warring minions, Anonymous.  The loose-knit collective of hackers is being credited with some impressive technical feats, as ABC News trumpeted:   “What’s most surprising about "Operation Payback," cybersecurity experts say, is the simplicity of its approach to wreaking havoc on the web.  The massive hack attack appears to have been orchestrated by a handful of organizers with control over a virtual army of tens of thousands of computers. The networks -- called botnets -- can inundate their targets with denial of service attacks, so overwhelming a site's server that regular customers can't get through.”

So who are Anonymous?  Anonymous didn’t start with Wikileaks.  They formed, more or less, on 4chan.org, a website where people upload and discuss random images culled from the Web.   A monster truck DeLorean.  A pink-eyed chinchilla.  The images must be legal, other than that anything goes.  Whenever a thread is particularly weak, discussants mark it with an image of Guy Fawkes, the pyrotechnic 16th century revolutionary, as re-imagined in the Wachowski Brothers’ dystopian movie V for Vendetta.

The point, besides laughs, is free expression, and to foster it they register on 4hcan under the same handle, Anonymous.   Compared to ordinary life offline - where, like everyone, they have to watch what they say - the power they feel while cloaked is awesome.  They can say anything, and some do with abandon.  As the FAQ on 4chan reads, “Anonymous is not a single person, but rather, represents the collective whole of 4chan.  He is a god amongst men.”  

Empowered by anonymity, Anonymous began doing online pranks they called Raids.  Once, hackers in the group busted into an online children’s game, flooding an animated swimming pool with their own characters.  Another time, they posed as kids to entrap an Internet pedophile.  Critics have accused them of more nefarious deeds, from flooding a guy’s MySpace page with pornography to calling in bomb threats at the Super Bowl.  Because they’re unknown and anyone can claim to be among them, you never really know what, if anything, Anonymous is responsible for at all.

Anonymous went wide in 2008 when they waged a global protest against the Church of Scientology, which they accused of censorship and unjustified tax exempt status, among other things.   It was an epic battle waged from YouTube to Utah. Scientology websites got flooded with denial of service attacks and crashed.  Prank calls rang at Scientology headquarters off the hook.  Black faxes spooled through Scientology faxes depleting the ink.  Pizzas arrived at Churches around the world, including a reported 300 at the headquarters in Amsterdam alone.  The Church of Scientology released a statement calling Anonymous members “cyberterrorists who hide their identities behind masks and computer anonymity.”

Anonymous marks something far larger and more interesting:   all the fears and fascination of online anonymity made real.   As one member of Anonymous told me, “We identify with Guy Fawkes.  He’s a symbol of reform.  He’s mysterious.  You never know who he is.  You never get his identity.” 

 

U.S. Air Force Drone Touches Down Amid Questions

In a development that has absolutely nothing to do with a mysterious vapor trail, an unmanned space plane designed by NASA researchers touched down at Vandenberg Air Force Base in California on 3 December after seven months in orbit. The 8.9-meter-long X-37B drone, which has a 4.5-meter wingspan, handled its controlled reentry and landing autonomously—a first for the U.S. space program.

Though much is known about the design of the craft because of its origins as a NASA project, the aims of the plane’s recent mission and details about how it was equipped for the flight are a tightly guarded secret because the U.S. Air Force launched it. Speculation over what the craft—which is powered by a solar array and lithium-ion batteries—was doing in orbit these past few months has abounded. Amateur satellite watchers say they tracked the X-37B as it changed its orbital path in ways similar to the maneuvers spy satellites perform when trying to capture a series of images of a specific area on the earth. Another educated guess is that the craft could have been inspecting other nations’ military satellites.

Despite the fact that a branch of the U.S. military launched the craft into orbit, the Air Force was quick to dismiss concerns about the weaponization of space.  In April, when the X-37B was launched atop an Atlas 5 rocket, Gary Payton, the Air Force's deputy undersecretary for space programs, said the miniature Space Shuttle is “just an updated version of Space Shuttle type of activities in space."

One question: If the project is a continuation of what NASA has been doing for decades, why isn’t NASA doing it?

Payton added that, “We, the Air Force, have a suite of military missions in space and this new vehicle could potentially help us do those missions better.”

It’s possible that none of those military missions have anything to do with weapons, but just how remote are those chances?

What Computer History Should Be Saved?

On January 13, the Computer History Museum in Mountain View, California will be relaunching with a new exhibit, "Revolution:  The First 2000 Years of Computing." 

According to the museum's site, "Revolution is a 25,000 square foot, multimedia experience that will immerse visitors in the sights, sounds and stories of the computer revolution. More than 1,000 artifacts from the Museum’s vast collection will be on view, including rare computers, audio and video, photographs, games, and hands-on displays. Revolution will also feature more than 100 media stations and three mini-theaters."

How do they determine what to include?   When the Wall Street Journal asked Dag Spicer, the senior curator at the museum, he said "We have what we call a 10-year rule. We prefer to wait 10 years in order to assess its historical importance. We've made exceptions—for example, the iPod. We didn't need to wait 10 years to know that would be a world-changing technology."

Across the blogosphere, I’ve been reading about the machines that gamers would include.  The Apple II.  The Commodore 64.  The Atari 2600.   I’ll add one more to the mix, but it doesn’t come from the States.  It comes from Dundee, Scotland. 

In the 1980s, Dundee seemed like the last place on earth for a computer revolution, but, as a burgeoning legion of Scottish programmers knew, something electric was coursing over the cobblestone streets.   The Timex plant in town was churning out the UK’s first popular wave of home computers, the Sinclair ZX81 and Sinclair ZX Spectrum. 

The Spectrum in particular, with its jet black keyboard and rainbow streak on the side, felt like a secret panel to another world.  All you needed to know was the code, and you were in.   The word about town was that a lot of Spectrum computers were falling off the trucks – purposefully – so that the town’s aspiring players could get their hands dirty.   The revolution was built on stolen PCs.

Growing up in the town of the Spectrum had its perks.   The local high school was among the first in the UK to offer computer studies.  One of the students was Dave Jones.  Gifted at math and terrible at sports, Jones put all his energy into teach himself to code, tinkering with his own programs and buildings his own rudimentary machines.  Upon graduation, he knew exactly where he wanted to work:   building Spectrum computers at the big brown Timex plant of his dreams.  

Building hardware all day as an apprentice engineer was cool enough, he thought, but what he really burned to do was make games.  At the time, a homebrew computer game scene had been burgeoning around the world. Especially in the United States, home computers such as the Commodore 64 and Apple II had developed a cult following among gamers making and distributing their own titles to play.  Jones had joined a ragtag gang of computer coders called the “Kingsway Amateur Computer Club” who met at the local technical college.  Before long, they were churning out some of the most influential games of their era, including Lemmings and Grand Theft Auto.  And it all started with the Spectrum.

If you were the curator of the Computer History Museum, what would you include?

LEDs for Water Purification, Solar Cell Manufacturing Process Honored in Clean Tech Open

It started back in 2005, when two EEs who wanted to somehow advance green technology but didn’t have money to invest or a big company behind them decided to hold a contest. They called it the CleanTech Open, and rounded up $500,000 in prizes. It was mostly a local thing, with teams of California entrepreneurs entering business plans.

But in five years it has turned into the Academy Awards of green technology, complete with sealed envelopes and acceptance speeches. (See video, below).

This year 271 teams entered regional competitions; the regions sent 18 teams to the national finals. And though the competition continues to be held in California, the home field no longer seems to have any advantage.

The winner, Puralytics is from Beaverton, Oregon—the first time a company won from outside of California. Puralytics developed a technology that removes contaminants like petroleum byproducts, pesticides, or microorganisms from water using multiple wavelengths of light generated by LEDs.

Puralytics may have captured the judges’ hearts (and $250,000 in prizes), but the audience at the awards, made up of other competitors and industry notables, was enamored with Silicon Solar Solutions. Silicon Solar, from Fayetteville, Ariz., developed a process for crystallizing amorphous silicon into large-grain polysilicon almost instantly. Solar cells built from the large grain material, the company says, are much more efficient than traditional cells.

The judges also honored two runners-up: On-Chip Power from Boston, for its miniaturized power-supply technology, and EarthClean of Minneapolis for its nontoxic fire suppressant

A complete list of winners in all categories and regional finalists is available here.

The 2011 competition kicks off on 3 March 2011.

The Hottest Game System On the Planet Is...

The other day I got a panicky call from my friend Andy.  His mother was holiday shopping for his two elementary school daughters, and he needed to tell her what video game system to buy.

Andy was considering the Nintendo Wii, but worried about the hassle of hooking it up to his TV, and how much time they’d really spend as a family on the system.   The other option on his mind:   a Nintendo DS.   As the world’s most popular handheld system, the DS seemed like an obvious choice – except for the fact that his kids were more interested in playing Angry Birds on his wife’s iPhone.   What to do?

If you want to know where the future of gaming is heading, it's worth paying attention to what I'll call the Andy Conundrum.  The pragmatic concerns of casual gamers are now having more impact than even the highest-tech innovations of this $40 billion industry. 

Consider Andy's first sticking point:  the hassle.  Fact is, the Wii isn’t much of a hassle at all to set up, right?  But for a lot of people, just the idea of plugging hardware into a TV is hassle enough.  There’s something decidedly 2009 about the whole prospect.  With every day, we’re becoming less patient with cables and cords and instruction manuals.  The age of Plug 'n Play is being replace by the age of simply Play.  And we also don’t want a lot of stuff cluttering our rooms.   Argue as much as you want with Andy, but these objections are real to him, and they’re the reason that – no matter how much I recommend the Wii – he won’t buy.

Next up:  the DS.  I extolled the virtues of the Nintendo 3DS, the new 3D handheld coming from Nintendo in March.   As I blogged here this summer, I played the 3DS and it was compelling:   a true, 3D experience delivered without kludgy glasses.  Even better, the 3DS doesn’t try too hard.  Compared to the explosive eye candy being shown by Sony on the Playstation 3, the 3DS games are dazzling in their simplicity.   The future of 3D games have one imperative:  less is more.

Andy was interested, but I could hear some hesitancy in his voice.  Then I realized what he, and I, were missing.  He already told me the answer.  His kids loved playing games on their mom’s phone.  We all know that kids love their parents’ phone.  I’m sure much of their interest isn’t about the hardware or software at all.  It’s just about the incredible thrill kids feel when they’re playing with something grown-up.  Kids see us on our phones all the time and they want one too.  In Andy's house (and others) the kids want a phone more than DS.  So I told Andy that he could get the kids an iPod Touch.   Andy thanked me, said he’d think about it, and that was that. 

Then the phone rang a few minutes later.  Andy had run the math, and realized that though iPod Touch was a tad more expensive, the games were way cheaper – say, 99 cents for Angry Birds versus $30 for a DS game.   In no time, the Touch would cost him less than the other systems.  He was sold.   The Touch it would be it.

Now, this isn’t to say the Touch is a better game platform than the DS or Wii or Xbox or Playstation either.  But for casual and young players – which represents a larger part of the population than core gamers – Touch/app games are tough to beat:  cheap, accessible, and no unruly wires.  Extrapolate the Andy Conundrum across the world, and that marks a seismic shift in the gaming landscape.  In the battle of Mario vs. Angry Birds, the Birds win. 

 

3-D Printing Could Make Smartphone Chips Cheaper

First, some apologies to all of you in the packaging industry: I find packaging one of the least interesting, hardest-to-spin-into-an-engaging-story corners of the semiconducting industry. But Eoplex CEO Arthur Chait stopped by IEEE Spectrum yesterday to show me what he expects will be a revolution in chip packaging, and I must admit it has me thinking differently. Eoplex, developer of a 3D printing technology used to make high-end cellphone antennas and other small, complicated contraptions, really might make packaging sexy.


Here’s a little background on Eoplex, because you’ve probably never heard of it. A California startup, Eoplex has come up with a combination of secret sauces and manufacturing techniques that lets it print sub-micron size voxels of stuff to mass produce 3D objects. After some simple, but secret, processing this stuff turns into metal, ceramics, and empty spaces. The result can be miniature machines with moving parts, metamaterials-enabled multi-function antennas, piezoelectric powered energy harvesters, coin-sized hydrogen fuel cells, pretty much anything


As amazing as all that stuff is—and the menagerie of micromachines Chait carries with him really is amazing—finding a way to make money off of it isn’t as simple as it might seem. They tried energy-harvesters for car tire pressure sensors, but automakers have scaled back on their plans for that. They’ve also made some devilishly complex smart phone antennas, but, because only the highest-end phones need them, that business has proved smaller than expected, says Chait.


But this time Chait thinks Eoplex has hit it big. There’s a type of chip packaging called QFN (for Quad Flat No leads) that’s all the rage for packaging chips for mobile devices. The key to QFNs are the lead-frames, delicate spider-works of metal laid out by the dozens on a rather expensive tape. The lead frames are made by etching away a film of metal from the tape, using various nasty chemicals. What you’re left with is a set of leads that look like q-tips surrounding a central slab of metal that conducts heat away from the chip. The long end of the q-tip is just to anchor it to the frame; it doesn’t have an electrical purpose. In the packaging process the chips are stuck to the lead frame, and delicate wires are connected between the chip and the leads on the tape. That familiar black plastic is then flowed over the chip, and then the frame, chip, and plastic are peeled from the tape. Finally, the packaged chips are diced up, tested, and shipped.


Sounds fine, right? Well there are a few hitches. First, in order for the leads on the lead-frame to be mechanically stable enough, they are all connected along the edges. That means that until the packages are diced up they’re all shorted to each other and they can’t be tested en masse. Second, the dicing itself has to go slower than you might expect in order to avoid smearing the delicate leads and accidentally shorting some. Third, even after dicing, those connection points leave a lot of wasted metal in the package (remember the stick end of the q-tip?), with all the parasitic capacitance and other electrical unwanteds that implies. Fourth, because of those q-tip sticks, you can’t fit more than 2, maybe 3, rows of leads at each edge of the chip. And fifth, did I mention the tape is surprisingly expensive?


So the ideal QFN package would not be made using a chemical etch (so it’s a relatively green process), have no extra metal (so it would have better electrical performance, so chips could be tested before dicing, and so dicing would be quicker), as many rows of leads as you want (so you could get data on and off the chip faster), and no tape (so it would be cheaper). That’s just what Eoplex has been peddling to major chip makers and packaging houses this season, according to Chait.


Instead of starting with a film of metal on tape, Eoplex prints a layer of recyclable steel alloy and then tops it with the electrically important parts of the leads (the tip, not the stick) and the thermal pad—both made of silver.


What’s really cool is that the 3D printing is so finely controlled. In order to make the interface between the silver and the steel strong enough to survive processing, but weak enough so that all the tiny leads stayed with the chip when its time to peel everything away, they had to engineering the interface down to the micrometer-level. Using a proprietary pattern of alternating steel and silver voxels they managed to get the interface just right. I peeled a set myself [see the photo, above] and there wasn’t a single lead missing.


Eoplex has its sights on about $700 million worth of the QFN market, which could grow to $1.3 billion by 2014. By then, he says, the process might be used in other packaging forms such as ball grid array, meaning a $4 billion market.
Although they’re in discussions and testing with some real industry heavyweights, Eoplex is going to need some luck to capture such fortunes. Then again, they’ve already had some. They discovered the secret voxel pattern that’s key to the QFN product by accident. And, according to Chait, that might be good for keeping competitors at bay


“The barrier to entry with luck is a lot higher than with hard work,” he told me. In other words it takes only a few scientists and engineers to stumble on something amazing, while it might take hundreds to engineer it on purpose.

A Retinal Prosthesis Turned On By Light

In a mere half-decade, the use of light to stimulate the brain has moved from basic science to the frontiers of bioengineering. By inserting into brain cells a light-sensitive protein originally found in swamp algae, engineers and scientists have begun to manipulate neurons with a dexterity that could soon vastly outstrip the capabilities of today’s electrical brain stimulation methods. This month, Patrick Degenaar reported early progress toward a non-invasive prosthetic retina that uses light to force retinal ganglion cells to fire on command, presented at the IEEE Biomedical Circuits and Systems conference.

The use of light to control neurons could enable significantly more powerful brain-machine interfaces. First, it allows biomedical engineers to activate chosen sets of neurons, not simply whatever cells happen to be near the stimulation site, as with electrodes. Light can also be used to inhibit a neuron’s firing, whereas electrodes can only stimulate. Most intriguingly of all, the engineering of light-triggered brain cells could begin to pave the way to a hybrid computer that uses an optical link to unite biological and silicon components.

But first, we’d need a light-emitting technology well suited to our neurons. One major engineering challenge is that the light source must emit 1-milliwatt-per-square-millimeter pulses to induce brain cells to fire, according to Degenaar and his colleagues at Imperial College, in London. So they set out to build a gallium nitride micro-LED array on a sapphire substrate that was capable of delivering the proper current density.

They extracted rats’ retinal ganglion cells--which ferry image information from the eye’s rods and cones to other regions of the brain--and inserted Channelrhodopsin-2 (ChRh2), one of the light-sensitive proteins. ChRh2 enables ions to flow into a cell when hit by light of a certain wavelength, causing the neuron to fire. The ChRh2-enhanced neurons were placed on an electrode array that would record their electrical activity.

The researchers' LED chip spells out the word 'optical.'

Each emitter in the 16-by-16 micro-LED array had its own current source, allowing for individual control. For the purposes of the experiment, the engineers sent the patterns to be lit from a computer to the LED chip through a USB connection. When the LEDs were centered on the cell body, the soma, they observed that each 500-millisecond pulse, delivered at a frequency of 1 Hz, indeed produced one neural spike, exactly as they’d hoped. 

The next-generation LED array will incorporate on-chip microlenses to spread the light over a wider area, says Degenaar, who has since joined Newcastle University’s electrical engineering department. Eventually they hope to build a headset equipped with a camera that performs a certain amount of image processing before activating neurons optically.

Patients suffering from retinal degeneration fail to send useful signals to the visual cortex when light hits the eye’s photoreceptors. So an enormous and central challenge here will be writing the software to induce appropriate neural patterns. But with the exquisite neural control that optical stimulation allows, engineers have at least brought the goal within sight. Indeed, there’s a certain beauty in having complex, human-engineered optical systems to replace biology’s eyes.

 

Advertisement

Tech Talk

IEEE Spectrum’s general technology blog, featuring news, analysis, and opinions about engineering, consumer electronics, and technology and society, from the editorial staff and freelance contributors.

Newsletter Sign Up

Sign up for the Tech Alert newsletter and receive ground-breaking technology and science news from IEEE Spectrum every Thursday.

Advertisement
Load More