Tech Talk iconTech Talk

The Future of Social Networking is the Surveillance State

Last time I wrote about the social panopticon, I was being tongue in cheek. But today I don my tinfoil hat for real to bring you the Danger Room story of Recorded Future, a company being funded by a CIA research branch and Google to mine publicly available data (including social networking data) for event prediction. If you haven't read it, go do that now. I'll be here when you get back.

The idea is to figure out for each incident who was involved, where it happened and when it might go down. Recorded Future then plots that chatter, showing online “momentum” for any given event. “The cool thing is, you can actually predict the curve, in many cases,” says company CEO Christopher Ahlberg.

Oh, Philip K. Dick, which dystopian future development can’t you predict? Minority Report gave us the department of pre-crime, and we seem to be pretty cool with galloping toward that reality. What else do you do with the "gusher of money" U.S. Defense Secretary Robert Gates describes in the Washington Post’s fantastic expose of the homeland security boondoggle?


That money gusher, according to the Post, enabled nine years of boundless privatization of national security work. This amazing graphic lays out just how many organizations are newly in existence thanks to 9/11 and taxpayer money. And they all basically exist to do one thing: predict the future.
For Recorded Future, the tool for predicting the future is pattern recognition through social networking.

[Recorded Future] scours tens of thousands of websites, blogs and Twitter accounts to find the relationships between people, organizations, actions and incidents — both present and still-to-come. In a white paper, the company says its temporal analytics engine “goes beyond search” by “looking at the ‘invisible links’ between documents that talk about the same, or related, entities and events."

Unfortunately Danger Room's rare sunlit example is only a hint of what’s to come. In a New Scientist article out this week (read it while it’s hot; I think it will soon disappear behind a subscriber wall) Mark Buchanan points out that researchers are using all this publicly available information to harden what has up till now been the squishiest branch of human knowledge: social science.
Our online footprints (those thousands of baby pictures, all your "likes" and interminable retweets) are helping the social sciences become more like the "hard" sciences by stepping toward quantifying once-elusive qualities like the effects of public opinion. And why worry about the "stolen" facebook data torrent when you're already voluntarily giving up the details of your offline existence with location-tracking smart phones, foursquare check-ins, and contact lists?
The researchers Buchanan profiles "ultimately hope to discover mathematical laws that describe human behavior, and which could be used to predict what people will do."
I have two predictions.
1) After prediction comes machinery.
What kind of machinery? Consider the once soft and squishy field of "deception detection," which now has actual hardware in use by the good people at the Transportation Security Administration. Back in May, Sharon Weinberger profiled the Future Attribute Screening Technology (FAST), hardware under development that is being financed "around $10 million a year," no doubt courtesy of the gusher of money.
FAST (a latter day polygraph) measures nonverbal cues: Weinberger says "the idea is to have passengers walk through a portal as sensors remotely monitor their vital signs for 'malintent': a neologism meaning the intent or desire to cause harm." Inside the portal, thermal cameras and something called BioLIDAR purportedly measure tiny physiological signals including flickers in eye movement, pupil dilation, heart rate and respiration.
How long before “GoogleMe” means something completely different?
2) Recorded Future CEO Christopher Ahlberg is about to lose at least one of his funding sources.
After his proud chat with Danger Room about his cool new project, it’s hard to say which funding source will be the first to disassociate itself from this story—will it be Google, which wants to protect its "don’t be evil” reputation but already has two sizeable black eyes from street-view-data-gate and the Google Buzz privacy debacle? Or In-Q-Tel, which is likely roasting under the gaze of some new scrutiny in the wake of the Washington Post report?


Phil's Vid-Minute: The Chevy Volt is Still a Costly Mistake

IEEE Spectrum senior editor Philip E. Ross, feels vindicated by the high sticker price of the Chevy Volt.

 Discussed in this video blog:

Loser: Why the Chevy Volt Will Fizzle
January 2010

Article: General Motors' Chevrolet Volt hybrid car is a courageous design, but it won't make money

Nissan's All-Electric Leaf Doesn't Stint on Performance
April 2010

Article: The Nissan Leaf is limited only by the range of its battery's charge

The Race to Design a Nanopore Gene Sequencer Heats Up

I just spent the last glorious week up at Jackson Laboratory in Bar Harbor, Maine listening to researchers explain the state of medical genetics. "Next generation sequencing" was a term I kept hearing. What's clear is that being able to quickly and inexpensively sequence the complete genome of a patient will revolutionize research into, and eventually the treatment of, nearly every disease out there. We now typically look at the genome with a flashlight, sequencing those areas for which we have a reliable lead. Soon, we will be working under a floodlight.

Robert Nussbaum, a geneticist from UC San Francisco said it best. "It's going to illuminate the dark matter of the genome."

The technology to keep your eye on is called nanopore sequencing (here's a good intro), and this summer has been particularly exciting. Two groups have published variations on a similar approach to sequencing DNA by passing it through a perforated sheet of graphene. Both are described in the journal Nano Letters (here and here).

In theory, the technology would work like this: directed by an electrical current, DNA would pass through nanosized holes (about 5-10 nm in diameter) in a single layer of graphene. Because the graphene is so unimaginably thin (1-5nm thin) only one pair of bases would inhabit the pore a a given time. Fluctuations in the electrical conductance of the membrane and measurements of the ionic current will be specific to each base pair in the DNA and can be used to sequence it as it passes through.

This is, so far, the most promising approach to be described. Let me give you a couple more of the consequences of next-generation sequencing that I've absorbed in the last week. It will give us insight into why individuals respond differently to drugs and will eventually personalize drug therapy. It will make individual gene testing obsolete. It will bring new, rare gene variants to light that we can then begin studying in relation to disease.

It will also, very importantly, put pressure on companies who have large databases of the genetic information they've mined from patients and clients to make their resources public and available for research. Nussbaum estimated that only about 5% of that information is now available publicly. Next generation sequencing would make it less of a commodity.

He also emphasized however, that this deluge of data makes it even more important for us to get on track with electronic health records. "We absolutely have to have electronic records," he says.

Is Microsoft Looking to Dominate Wireless with Strong "ARM" Tactics?

By now, everyone who keeps an eye on the consumer electronics market is aware of the way Microsoft operates. It has been repeatedly taken to task by European and U.S. authorities for wielding its Windows operating system monopoly like a cudgel to prevent upstarts from horning in on its territory or as a battering ram for breaking into new arenas.

So when it was reported that Microsoft has expanded its licensing agreement with mobile processor designer ARM, industry analysts immediately began plotting out the software maker’s possible chess moves. How will Ballmer & Co. put itself in a position to reap more from the fact that wireless handset makers currently sell more than a billion of these devices a year? Though the Redmond, Wash., software behemoth is being tight lipped about how it plans to use the parts of the ARM instruction set to which it has just gained access, it is believed that the motive is to learn how to optimize software packages like Windows Phone 7 so that they run faster on chips that require less power and therefore offer longer battery life in devices designed to slip into one’s pocket. That makes a lot of sense, considering its upcoming launch of the Windows Phone 7 handset with AT&T as its primary service provider.

But if history is a reliable barometer, what Microsoft really covets is what it has enjoyed with desktop and laptop PCs: being the maker of go-to software that is so basic as to be indispensable. Imagine what it would be like for Microsoft to find itself in a monopoly position with respect to some element of the new smartphones, e-book readers, and competitors to Apple’s iPad tablet coming to market in the next few years. With that in mind, Microsoft would be hard pressed to remain a bit player in a market segment where fortunes are continually being minted. But one question remains: Will the company claim a niche because a newfound understanding of how ARM processors work will yield superior software, or will it default to strong arm tactics of the type that once had it on the brink of being broken up like AT&T?


Why Mint Doesn't Measure Up

When Intuit bought and its 1.5 million users in September 2009 for $170 million, it was widely assumed that the company would eventually transition the current users of its Web-based Quicken Online personal finance application to the award-winning Mint. Intuit kept Mint founder and new media darling Aaron Patzer and its development team on board, and it promised Quicken Online users that “good things are coming your way.” Aaron even sent note on April 29, 2010 telling me and my fellow Quicken Online users:

Dear Valued Customer,

Since Intuit acquired, our personal finance teams have been hard at work (behind the scenes) combining Quicken® Online and into the single best way to manage your money online— from the makers of Quicken.

You will still enjoy the features you know and love in Quicken Online plus so much more. It will make personal recommendations based on your spending to help find ways to reduce your bills and cut your interest rates—on average, saving people hundreds of dollars. And, best of all, it will remain FREE!

We'll let you know when we get closer to a release date. In the meantime, you can continue to use Quicken Online just like you have. Once we have completed integrating all features to Mint, you will be able to easily transfer your information and data to ensure the smoothest transition possible.

Thank you for your continued loyalty.

Wow. Mint. All those awards. All that glowing praise from the personal finance and Web press. And I'll be able to easily transfer all of my information. Maybe they'll even do it for me, I thought. Bring it on!

Then two weeks ago Intuit reversed that decision. On July 16, 2010, good old Aaron sent me and all other Quicken Online users another note, this time with bad news:

Dear Valued Customer,

For the past several months, we've been working hard to combine the best features of Quicken Online and into a single online personal finance solution— With the improved, you can enjoy the features you love in Quicken Online, plus new benefits such as connecting to over 16,000 financial institutions, including Canadian banks—as well as tracking your investment and retirement accounts. There is also a new Goals feature that takes the tool you enjoyed in Quicken Online to the next level.

As a result of these changes, Quicken Online will no longer be available as of August 29, 2010. Creating a new account is easy, but for reasons of security and accuracy, we cannot create one for you. Once you're signed in, you can add your accounts and see your financial picture in just a few minutes.

While it only takes minutes to sign up, I do urge you to make the switch at your earliest convenience. After August 29, 2010, you will no longer be able to access Quicken Online or your data stored in it.

If maintaining a record of your Quicken Online data is important to you, you can export it to a CSV file. If you have any questions, we're here to help. Click here to view a list of common questions.

We look forward to continuing to serve you with—the best online personal money-management solution available. Sign up today.

Disappointing but understandable. No doubt that migration is tough, and transitioning all the user bank, credit card and bill information set up in Quicken would be a monumental task. It would have been nice if Aaron would have been straight with Quicken Online users about those difficulties instead of telling us about all the great new features but hey, I can take a hint. And killing off my Quicken Online account certainly makes sense from Intuit’s point of view—they did pay $170 million for Mint, after all. Might as well start recouping some of those costs and shuttering Quicken Online is a big step in that direction.

So migrating my own stuff on my own might be a hassle but it will be worth it, I thought. Aaron has personally assured me that his team has been working hard to combine the best of both sites. Mint debuted at TechCrunch, got raves on this very Web site, and won a slew of awards. 1.5 million users can’t be wrong, can they?

Sure they can.

Awards are fairly easy to come by on the Web and 1.5 million users isn’t exactly a bellwether of a great site—just look at MySpace (actually, spare yourself the trouble). But Quicken bought Mint for $170 million—the more I say it, the more my head hurts—so, hey, it’s gotta be good, right?

I’m here to tell you that for many users, especially those from the Quicken Online diaspora, Mint,—and I’m searching for a really nice way to say this—Mint sucks.

Here’s why. In addition to several usability quirks—like signing you out of the application while you pour yourself a cup of coffee or the practically unusable drop-down calendars that require a surgeon’s steady hand to select dates—Mint does not let you schedule transactions in future. While Quicken Online was far from perfect, it let me schedule transactions in future, so I could budget at a very granular level of detail. Not so with Mint.

In keeping with it's Web 2.0 Weltanschauung Mint does have some very active message boards. So I went there to get some satisfaction. Judging by the posts I read, anger and frustration among Quicken Online users is rising. Open rebellion is being proposed and most shocking of all is that for months users have been complaining of the lack of the most basic budgeting functionality and have been summarily ignored:

abloxom replied 4 months ago
This is the number one reason I use Quicken 2007. Scheduled transactions allow me to see in advance how much money i'll have if I make all of my CC, Phone, and Utility Bills on time and in full. I'm able to easily evaluate how much I can spend without going over my limit. Come to think of it, it's the ONLY reason I use Quicken at all.

Amen abloxom. And the affable Jami from Mint responded:

Jami, Official Rep, replied 4 months ago
Hi wlteague4289,

As this feature is not currently available, I've changed made your question an idea so it can be considered by the product team.

Thank you,

Mint Community Manager

So while Mint’s product team has been twiddling around merely considering adding the most basic functionality for the last few months, they’ve made absolutely sure that they have some kind of business model in place to make money off the Quicken Online users they are asking to migrate. Mint has put a ton of resources into partnering with financial institutions and credit card companies to offer you alternatives to your higher interest rate credit cards, for instance. Great. But if Quicken loses thousands of customers in the next few weeks because its product cannot meet user needs in terms of the most basic functionality, all this Webby-award winning new economy goodness will be for nought. That means zero, Intuit. As in ZERO Quicken Online conversions.

Thankfully, members of the Quicken Online diaspora are more than willing to help each other. Help each other escape Mint, that is:

amysun123 replied 9 days ago
I'm thinking of migrating to a new product (GreenSherpa) that has everything I need. I am even willing to pay a small monthly amount to have useful functionality. QUESTION to fellow forum users: What do you all think of GreenSherpa: ? It seems to have everything we need and it's only $5.95/month. Does anyone have any experience with that product? Should we all do a mass migration and leave Mint in the dust?

Wizznilliam commented 9 days ago
Why pay $5.95 a month when Yodlee has everything Quicken Online had and more for free. The only shortcoming I have seen so far is that it does not automatically give the pretty cleaned op transaction descriptions that Mint is good at doing and QOL was only partially good at. Everything else is there and it is HIGHLY customizable. And Free Bill Pay so you can do EVERYTHING all in one place. I'm digging it so far. There are a few quirks. But they all have them.

I’ll be checking out Yodlee, Greensherpa and whatever else I can find and reporting back in the next few weeks. If Intuit is going to force us to go to something other than Quicken Online, then we should investigate the alternatives to the sad excuse for personal finance site that they are offering. If you have any suggestions, let me know.

Low Tech Fixes for High Tech Gizmos

I’m tech support at my house. I replace batteries in remote controllers, trouble shoot buggy computers, and figure out why the TV suddenly has no audio. I taught my daughter how to reboot her buggy cell phone, gave the home wireless network a boost with cardboard and aluminum foil, and reprogrammed my car’s wireless key remote by rhythmically opening and closing the door.

So I guess I shouldn’t be amazed when solutions to high tech problems are extremely low tech. Still, I can’t seem to shake my initial skepticism whenever I hear about these bizarre cures—and how impressed I am by the collective knowledge online when they work. (I never can tell who first thinks of these things; wish I could give them credit.)

The latest such fix has gotten a lot of press—using a scrap of duct tape to fix the iPhone antenna problem. Love it; so simple, and everyone has duct tape (as heavy duct tape users, we’ve got at least 5 colors on hand in my house, so we could even make a fashion statement here).

My duct tape is handily stored next to my hot glue gun (on what I’m going to stop calling the craft shelf and rename the consumer electronics service center). A couple of months ago, that hot glue gun kept me from tossing a Flip video camera into the recycling box. It was a slightly older Flip model, passed down from one of Spectrum’s other editors, and I was trying to use it for video blogging. However, the simple, one-touch record was proving to be not so simple—I’d have to push the button 10, sometimes 20 times, before it would start recording, then another 10 or 20 times to get to stop. This rather significant glitch made it impossible to use the Flip for what it is meant to be best at, quickly catching a video-worthy moment.

Turns out, like the iPhone antenna problem, the Flip button problem has an easy, low tech fix—a drop of glue in the center of the button. This apparently works by making sure the button gets pushed in its sweet spot. As usual, I was skeptical, but couldn’t see that it would hurt to try. I heated up a glue gun then dabbed a blob of glue onto the center of the red button. And I haven’t had a problem since.

Got a favorite low tech fix to a high tech problem? Let me know in the comments below; include a link to a photo if you have one.

Rock Band Goes Pro

In September I profiled Harmonix, creators of the hit music franchise Rock Band, for IEEE Spectrum.  The company was founded by two MIT grads, Rigopulos and Eran Egozy.  Their music videogame franchises pioneered the idea of playing along with popular music using instrument-shaped controllers, and represent a global pop culture phenomenon with over $2 billion in revenues.  

Egozy and Rigopulos met while playing in the school's gamelan orchestra. When they were completing graduate degrees at MIT in the early 90s, they began looking for ways they too could bring out the musicianship of ordinary people.  And what better to engage a new generation than their medium of choice: Videogames.

They began with Joystick Music, a game which allowed people to create sophisticated music simply by maneuvering two Atari style controllers. By later coding a unique pitch analysis program that could grade a person’s voice in real-time, the two created Karaoke Revolution, a sing-along-with-the-music title for the Playstation 2.  But the lingering criticism remains – yes, videogames and sensor chairs let regular people make music, or at least feel like they’re making music, but of course they aren’t making music for real. “I think there is some basis to that criticism,” says Rigopulos, “and it’s something we had the ambition to address.”

Now comes the answer:   Rock Band Pro—a new version of the popular videogame coming this fall that will use new instruments to allow for a more life-like playing experience. “It bridges the gap between simulating music making and making music on real instruments,” Rigopulos says.  I got a chance to play around with the new instruments, and they really are an incredible feat of engineering. 

The Wireless Fender Mustang Pro Guitar Controller has a whoping 102 controller buttons built on to the neck, running over 17 frets.  To play it, you pluck and strum actual strings.  To make real music, you can even use the MIDI output built into the device.  Cool enough.  Then there's the wireless keyboard, which also sports a MIDI port for those who want to jam offline. 

With so many innovations achieved, the challenge to spread the power of music-making is now more a matter of changing perceptions than creating new technologies. “It’s a matter of getting past conceptual hurdles,” Egozy says, “people think music is fixed thing that they just download. We’re trying to break down barriers and get people to think about music in a more holistic way.  Music is not a fixed package, it’s much more than that.”

Highlights From the 2010 North Carolina Maker Faire

A mini Maker Fair in Durham, North Carolina in late April showed what some of the technically creative tinkerers in the Research Triangle Park area are doing with their free hours.  The event included some four dozen exhibitors, including several equipment vendors. For example, ShopBot, the Durham-based maker of computer-numerically controlled router tables that sponsored the event, also showed off it’s new small-scale machine.

Although equipment manufacturers were out in force, the most charming moments came from the display of less technically sophisticated products, including an exercise bike that let’s you explore some of the world’s most attractive the back country—all while being perfectly stationary.

Ideas for things a more technically ambitious hacker might build include a giant touch screen—just in case you ever want try turning your dining room table into a giant iPad. The ingenious strategy makes use a special kind of plastic designed originally for lighting up posters.

One of the more interesting vendors on hand was ioBridge, which sells boards that make it easy to form an Internet of things--perhaps even a Web-based dog-treat dispenser, which if you look carefully is shown in operation as the system is discussed.

Ray Kurzweil's Music Revolution

I've been doing some research into the next generation of music technologies, and, not surprisingly, futurist Ray Kurzweil had plenty to say - sending me a lengthy response as to what he sees as the innovations to come. 

Kurzweil is best known these days for his extensive writings and lectures on the Singularity, a phenomenon which Spectrum investigated in great detail.  But he has also been intimately involved in the field of music, both at his company, Kurzweil Music Systems, and in his personal life (his father was a composer).  Here's what he had to say to me when asked about the future of music:

Walk around a typical music convention (such as the National Association of Music Merchants in Anaheim which I attended in January), and aside from the cacophony of music in all directions and the flamboyantly dressed performers it looks just like a computer conference given all of the complex hardware and software products on display for creating music. 

Consider how far we’ve already come.  My father was a famous conductor (conductor of the Bell Symphony, the symphony orchestra of the Bell Telephone system which appeared on TV frequently) and composer.  In order for him just to hear his compositions he would have to raise money, hire an orchestra, run off mimeographed scores, and then engage in rehearsals.  If he wanted to make substantive changes to his score, he would have to start this process over again.  Now a student in her dorm room can command a full orchestra (or jazz band or rock group) in her dorm room with very inexpensive yet immensely powerful equipment and software.  She then has access to distribute her creations to the world market at no cost. 

The tools for creating music have thus become an information technology and as such are now subject to what I call the “law of accelerating returns,” which states that every form of information technology grows predictably and exponentially in price-performance and capacity, basically doubling about every year.  Our intuition about the future is linear, not exponential and there is a profound difference between these perspectives.  Thirty linear steps gets you to thirty, whereas thirty exponential steps gets you to a billion.  So the computer in my pocket today provides a billion times more memory and a billion times more processing power per dollar than the computer I shared with thousands of other students when I was an undergraduate.

Every industry and area of life will ultimately be transformed in this way.  Health and medicine, for example, has recently become an information technology now that we have the software of life (the genome), means of updating this outdated software (RNA interference to turn genes off and new forms of gene therapy to add new genes), tools to design interventions on computers and biological simulators to test them out.   

In the music field, we saw the beginning of these trends in the early 1980s, around the time that I founded Kurzweil Music Systems. Nearly thirty years ago we set out to develop a digital technology that could realistically emulate acoustic instruments, the most important and complex one being the acoustic grand piano. There were many challenges in doing this due to the many nonlinearities in the grand piano.  Due to the multiple strings for each note that are slightly out of tune with each other, the overtones of the piano are not perfect multiples of the fundamental frequency – they are “enharmonic,” which gives the piano its unique rich character of sound.  Traditional samplers at the time would loop the last waveform during the decay phase of the sound and this turned the piano sound into what sounded like an organ.  We developed a unique way to retain the enharmonicity of the partials throughout the evolution of each note.  We were able to model the effect of key pressure on timbre, and to capture many other complexities.  The result was the Kurzweil 250, which was considered to be the first digital instrument that could realistically capture the grand piano and other orchestral instruments. 

In the intervening decades the electronic music field has broken the centuries old link between controllers and sound.  Traditionally, if you wanted to create violin sounds you needed to master violin technique (and have a violin!).  The problem was that very few people could master more than one or a couple of these techniques.  Even if you mastered them all you still could not play them all simultaneously.  Most instruments are not even polyphonic.  Today a musician can create any type of sound response (and keep in mind that an instrument is more than just sounds) while using any type of controller (albeit there is still work to be done to create a good polyphonic midi guitar).  As a result a new industry of controllers has arisen that are not limited by the physics of creating sound. 

New technologies are also engaging the consumer as co creator.  Consider how movies feed the game industry so that a movie viewer is now able to enter the world of the movie and become a participant.  The same thing is happening in the music world as the success of music games such as guitar hero attests.  Tod Machover of the MIT Media Lab has created music toys that allow children to compose symphonic works that are surprisingly rich and satisfying.  Applications such as "Virtual DJ" and "iRemix" allow users to alter music tracks in real time, automatically splicing in synchronized beats from other songs, adjusting tempo and generally deconstructing and rearranging content into something entirely new and personalized.  Music applications for Microsoft’s Project Natal music controller to be released later in 2010 will allow the controlling of music through expressive body movement. 

Keep in mind that these information technology tools will be at least a thousand times more powerful for the same cost in a decade, more than a million times more powerful in two decades.  As a result, the ability to immerse users in music that they participate will move from a game to a mainstream experience in the years ahead. 

In the decade ahead, music education software will become very effective at teaching music and playing skills and will have the ability to intelligently assess and address areas of strength and weakness.  It will become routine to learn keyboard and other skills using intelligent computer assisted music instruction.  At Kurzweil Music (now a subsidiary of Hyundai), we’re working on software along these lines.  “Easy play” software will involve more than just preprogrammed patterns that the user needs to follow.  Rather the instrument will follow the user.  Intelligent software programs that understand music theory will instantly interpret and even predict the creator’s intentions, and adjust the composition to ensure it’s in key and follows inherent musical rules.

There will be applications that will be able to gather all the subtle patterns present in one song, and apply them to another song, like a tint. Playback platform technologies will emerge that allow listeners to deeply and spontaneously interact with the music they consume – for example letting them "assign" a new singer to any song.  You could have Sinatra croon your favorite Pink Floyd tune, or transform your Green Day rock anthem into a samba. You could sample any voice, and digitally paint the song with it, or add a synthetic back-up choir of realistic human voices.

As virtual worlds become more immersive, realistic and accessible, and with ever faster computing power and communications connectivity, real-time collaborations will take place in virtual environments that will feel and sound real. Humans will remotely collaborate with other humans, or with intelligent programs that exist as a virtual DJ or avatar composer. And the memory of these performances and work sessions can be densely preserved – all nuances and details – allowing intelligent software to recall, learn from and then modify any performance. 

The interactive graphics field is grappling with a phenomenon called the “uncanny valley,” in which animations are close but not convincingly identical to real human behavior.  The result is a visceral negative reaction.  For this reason, many animated characters such as Shrek are distinctly different from humans.  To provide completely human avatars the interactive animation field will need to leap over the uncanny valley.  There’s a similar issue with music.  The early Moog synthesizers did not need to sound like real instruments and early emulations of orchestral instruments were rejected because of the uncanny valley.  The synthesizer field has successfully leaped this chasm and synthesizers are now used for almost all commercial music (soundtracks, musicals, commercials, popular songs).  The same leap has not yet been made in music collaboration software but that’s coming over the next five to ten years.

In music controllers we will progress from devices such as keyboards which provide touch sensitive switches to full immersive environments in which we can use our hundreds of muscles to interact with and shape rich musical tapestries. 

There are still frontiers to conquer in music synthesis.  The state of the art are instruments such as our Kurzweil Music PC3 series combine sampling with extensive digital synthesis, signal and sound processing.  The broad trend in computing is to recreate the real world with realistic simulations and then expand beyond what is possible with real materials and real environments.  Digital modeling has been around for a couple of decades but the enormous computational requirements still limit the ability to realistically capture real world instruments and effects.  The next decade, however, will be the decade of digital modeling.  We will be able to realistically simulate what happens to sound in instruments of real world complexity – for example providing a several hundred pole filter to simulate the effect of lifting the dampers on the several hundred strings of a piano (note that many of the 88 notes have more than one string).  Once the real world is captured we can then expand to fantastic virtual instruments with the complexity of the natural world but limited by only our imagination.

Music will remain the communication of human feelings and stories through sound.  We all communicate our feelings with words but very few of us have the opportunity to express ourselves by creating original music.  I believe the innate capacity to do this exists in all of us – the music technology that will emerge over the next decade will give voice to the music creator in all of us.

Six questions about the Solar Impulse plane


This morning at 7 a.m. local time, the Solar Impulse plane lifted off from Payerne air force base in Dübendorf, Switzerland, for its groundbreaking 24-hour flight. The flight was supposed to take place last week, but problems with some communications electronics prevented lift off. But today everything is proceeding according to plan. According to spokesman Lucas Chambers, Solar Impulse chairman Bertrand Piccard started the day with an invocation for the assembled peanut gallery: "This airplane," he said, "is here to prove that renewable energies are not just pornography for tree huggers."

Meanwhile, Andre Borschberg, Solar Impulse CEO and daredevil, is tucked away in his tiny cockpit, from which he will pilot the monster with the 61-meter wingspan for the next 24 hours. But as you might imagine, this isn’t the kind of plane you set on auto-pilot and go have a coffee and a snack. Borschberg will need to be alert for the entire time because the plane must not tilt more than five degrees: If it does, its massive wingspan and light weight could cause the plane to spin out of control. So, Borschberg needs to remain alert enough to keep the plane straight for all 24 hours of his journey—even in complete darkness.

Here are six questions I thought you might have about the trip.

The plane is solar-powered, but it will fly all night. How does that work?

The plane’s approximately 12,000 photovoltaic cells soak in the sun’s rays from 7 a.m. until sunset about 15 hours later (sunset, according to the Solar Impulse site, is at 22:00). During that time, they simultaneously power the plane’s four 10-HP propeller engines and charge four polymer lithium batteries onboard, which don’t start being depleted until the sun sets. “The big question will be whether the pilot will be able to save sufficient energy as to fly right through the night,” Bertrand Piccard was quoted as saying in a press release.
But it’s looking good: According to a blog post on the Solar Impulse site, in its eighth test flight, “the HB-SIA operated entirely energy-positive for the first time. This means we came back with significant more energy than we took off with.” (The site does not allow linking to specific posts so if you’d like to find the exact post, go here and search by date for the entry marked 06-05-10.)

How will Borschberg stay awake all night?

I mentioned in my last Solar Impulse update that research has shown that just 24 hours of sleep deprivation mimics the effect of a 0.10 blood alcohol level, illegal in Canada, the United Kingdom and the United States.

But researchers have also found that 20 minute-catnaps, spaced out properly, can stall these effects. That’s because a full complement of sleep includes 90-minute cycles of four stages of sleep, each more deep than the previous one, and finally REM sleep, in which the sleeper dreams. The average person has four or five such 90-minute cycles per night. If you’ve ever been forcibly awakened 40 minutes into a night’s sleep you know how disoriented and groggy you feel. The magic bullet, research has shown, is the 20 minute nap—you’ve entered only the lightest stage of sleep, and on waking you feel completely alert.

How does ground control make sure the far-away pilot never stays asleep for more than 20 minutes? You might think staying awake for 24 hours is a piece of cake—we’ve all done the all-nighter—but these are extreme conditions. Borschberg will be in near-complete darkness, illuminated only by the soft light of the instrument panel. He’ll be strapped into place in a cockpit roughly the size of a sarcophagus. He won’t be able to stand up or even stretch to get his blood flowing.

The answer was designed at Solar Impulse partner Ecole Polytechnique Federale de Lausanne (EPFL), the Swiss university that is also home to the Blue Brain project. In 2008, EPFL student Walter Karlen and his colleagues showed me a haptic vest they designed to monitor the Solar Impulse pilot’s state of consciousness. The vest is also designed to zap him awake with a haptic buzz whenever the monitors show that he has dipped out of consciousness.

My colleague Morgen Peck wrote about the vest for Spectrum last year. Here’s how it works.

A belt running around the ribcage measures respiration; gel electrodes fixed to the pectoral area read out an electrocardiogram; and other electrodes detect muscle activation. The whole thing runs off a battery that's also mounted right on the shirt.”

I don’t know if the vest design I saw at EPFL will be used on this flight, or whether it will be used in future flights. But the principles behind it have definitely been integrated into the pilot’s suits, according to this Sunday Times article:

[Bertrand] Piccard and Borschberg have already “flown” a 25-hour non-stop stretch in the simulator at Dübendorf. They managed to stay alert, helped by a vibrating system in their flight suits that wake the pilot if he sleeps more than a few minutes and lets a wing drop.

Oh, also, the engineering team purposefully made the unpadded seat in the cockpit really, really uncomfortable.

If the pilot can't move for 24 hours, how does he go to the bathroom?

He’ll take along a few of these and hope for the best.

I'm not kidding. When I asked the question at the Solar Impulse hangar, spokeswoman Rachel Bros de Puechredon pulled out this very package to enlighten me. It contains blue crystals that form into a gel when hit by a liquid.

And how will the pilot get any privacy, given the intrusive, always-on camera that will capture Borschberg's every blink for the duration of the flight? "There will be a webcam on the pilot, but with a minute’s lag so he a take a piss in private,” Solar Impulse’s Bertrand Piccard told the Sunday Times.

How will he survive for 24 hours at 8500 meters?

Because Borschberg won’t be flying above 8500 meters, there will be no need to pressurize the cabin, and oxygen won’t be necessary until he hits about 3600 meters. The cockpit temperatures will range between +35 to -20 degrees Celsius. Borschberg will be wearing a warmed, insulated flight suit and so will the batteries: Solar Impulse engineers designed special thermal insulation to conserve the batteries’ heat to keep them nice and toasty.

What if something goes wrong?

Borschberg has a parachute.

Why 24 hours?

They’ve been ramping up with short flights for a few months. In early April, the HB-S1A flew for 90 minutes, and a month and a half later a Solar Impulse pilot had it in the air for over 14 hours.

Today’s test flight will ensure that the batteries really can power the plane through a full day-night cycle. If the flight succeeds, then the engineers will simply scale up in increments of day-night flights. Next year the Solar Impulse team plans to do a transatlantic flight, and by 2013 they vow to circumnavigate the globe. I don’t know if they mean "without landing," because it might be tough to survive five days on those catnaps, and I imagine you’d run out of fresh portable urinals.

But the Solar Impulse plane that makes the hop across the pond won’t be the same one that does today’s 24-hour flight. They’re working on an upgraded model that might be able to accommodate two pilots.

For more answers to questions you didn't know you had, go the Solar Impulse Night Flight web site, where their press team will be live-blogging the event.


Tech Talk

IEEE Spectrum’s general technology blog, featuring news, analysis, and opinions about engineering, consumer electronics, and technology and society, from the editorial staff and freelance contributors.

Newsletter Sign Up

Sign up for the Tech Alert newsletter and receive ground-breaking technology and science news from IEEE Spectrum every Thursday.

Load More