The Making of Diaspora

This is part of IEEE Spectrum's special report on the battle for the future of the social Web.

The Diaspora guys, four college kids turned chief engineers of the most-talked-about social networking start-up this year, get a lot of friend requests. Sometimes fans just show up at their office, uninvited, and ask to work with them. Every now and then, someone recognizes them in public, which freaks them out. On the day they moved into their current office in San Francisco, a commuter stopped them on the subway and commanded, "Go get 'em, guys! Kill Facebook!"

Journalists and bloggers have called Diaspora "the Facebook killer," "the Facebook rival," "the anti-Facebook," "Facebook's challenger," and "another Facebook wannabe." They have speculated about whether Diaspora is better than Facebook, whether Facebook will try to buy Diaspora, and whether Diaspora could "knock Facebook off its perch."

The guys, however, don't see themselves as competition. After all, Diaspora is a rookie company; its software is buggy and crash prone, and although the company tries to solve the biggest problem with Facebook by giving users better control over their private data, its site looks and acts like a vacant, amateur imitation.

Besides, the guys insist, they're not aiming to replace Facebook with "yet another social network." Rather, they're taking a stab at reengineering the way online socializing works by building an entire network of networks from the ground up. They hope that in the process they will help promote standards that other social sites—such as Digg, LinkedIn, Google Buzz, and perhaps one day even Facebook—will use to bridge their services. They imagine that during the next decade, the Web will evolve from a sea of social networking islands into what many developers are calling the federated social Web—one that lets you choose your networking provider, just as you now choose your e-mail provider, and yet still connect with friends who use other services.

Such a Web may be a distant, idealistic vision, but it's not Diaspora's alone. Many programmers and social media thinkers, including some at other start-ups, at universities, and at big companies such as Google, Mozilla, and Germany's Vodafone, have been working to develop open standards for a federated social Web since around the time "The Facebook" was a profile directory for Harvard students. They believe that such a Web is not only possible but also preferable. "If I couldn't e-mail people who don't share the same domain as me, that would be pretty stupid," remarks Joseph Smarr, a social Web engineer at Google. "But that's exactly the way social networks work today, and that's broken and should be fixed."

Of course, Google has a lot to gain if Diaspora can fix social networking. The search giant has tried to stop Facebook from encroaching on its share of online advertising revenue by offering its own networking services, including Lively (discontinued), Orkut (thriving only in Brazil), Buzz (trashed for its privacy flaws), and Wave (passed on to Apache). But so far, Google has failed to make anything as wildly popular as Facebook.

As far as the guys know, no prominent Googlers have invested money in Diaspora, but they have written most of their code using Google-engineered protocols. And if they can work out the kinks in their system and turn a profit, says Rob Enderle, an Internet technology analyst and president of the Enderle Group, in San Jose, Calif., "their success will showcase to the bigger players that if you work with Google, you too can be profitable, and you can pick up users more quickly."

Diaspora itself may not be a threat to Facebook. But a federated social Web built on Google technologies, Enderle says, "would destroy Facebook's value and certainly its power on the network." If Google, starting with small players such as Diaspora, can plant the seeds of a federation and cultivate a network of many social services that is richer, more addictive, and more ubiquitous than Facebook, "it would create an environment wherein Facebook can't be the next Google," Enderle says.

The name Diaspora (the logo includes an *) refers to many things. It is the name of the company founded by Maxwell Salzberg, 24; Daniel Grippi, 22; Raphael Sofaer, 20; and Ilya Zhitomirskiy, 21. It is also the name of the guys' website, at Joindiaspora.com, which hosts Diaspora's 50 000 users. Finally, it is the name of the open-source software that runs the site and that other people are using to run other Diaspora-like websites, known as pods.

How many Diaspora-powered pods are out there?

"Hundreds?" Salzberg guesses. Often the spokesman at pitch meetings with venture capitalists, he is the most eloquent and outspoken of the four guys, and the most disheveled.

"A handful?" Grippi offers. "Pods that are up and running, anyway." He wears black skinny jeans and a black leather jacket and is the only Diaspora guy who can be described as fashionable. Perhaps because he has an eye for aesthetics, he recently decided that Diaspora's user interface is "not cool" and is redesigning it.

"We really have no idea how many pods there are," Salzberg concludes. "We have nothing to do with them."

A site run by a Web designer in Germany that tracks the uptime of several pods recently listed 48 of them. They had names like Diasp.org, My-seed.com, Dpod.se, Diaspora.lordgandalf.nl, and Facebook.doesntexist.org. Each pod hosts anywhere from one to 15 000 users. Sign up with any one of them and the experience is pretty much the same as if you signed up at Joindiaspora.com, provided the servers running it are reliable and the host keeps the code updated.

The guys imagine that someday, once other pod hosts start writing their own software, each one will be very different. For example, Diaspora could charge users US $5 per month to encrypt all their messages, while the host of My-seed.com could provide a more Facebook-like service with advertisements for free. Diasp.org could extend invitations only to engineers, while Diaspora.lordgandalf.nl could offer a Lord of the Rings theme and games. But because all pods built using Diaspora's source code and standards speak the same language, users on different pods are still findable and approachable.

Choice, interoperability, and the chance to invent your own networking experience are what federated networks such the Diaspora pods are all about. If I create an account at Joindiaspora.com but its servers are constantly down, or I find out that the Diaspora guys are selling my data to advertisers, I can export my profile and my list of friends and sign up for a new account at Diasp.org, for example, which is run by a business analyst in Seattle. Or, if I'm acutely privacy conscious and tech savvy, I can set up my own pod on my own server. "That's going to be one of our measures of success: How many people can we get off our servers using their own servers?" Grippi says. "We want our numbers to shrink."

Clearly, shedding users isn't an overtly smart business plan, and there are those who doubt Diaspora will succeed. Even the guys know their company could very well be dead within the next six months, once its seed money runs out. They admit their plan is ambitious and unformed, which "could totally come and bite us back in the ass," Salzberg says.

They believe what they're doing is important, but they don't want fans to hang their hopes on Diaspora; they want people to remember that they're not programming savants. "At the end of the day, we're just four dudes out of New York University trying to see if we can solve a problem," Salzberg says.

Yet in a way, they're perhaps the best kind of dudes to take on the challenge of creating a viable alternative to Facebook. Starting a social networking business today without honed programming skills and a clear moneymaking scheme is a big risk. But because the Diaspora guys are young and have little to lose, it's a risk they can afford. "Worst-case scenario is we put it on our résumés and go get jobs at Google," Salzberg says.

Best-case scenario is they make something that changes the world.

The guys didn't set out to dethrone Facebook. They first started hanging out together in the fall of 2009 as student members of the Association for Computing Machinery. Like most ACM college chapter offices, NYU's was a place to stay up late eating pizza and hacking things. In February 2010, they had just finished assembling a MakerBot Thing-O-Matic 3-D printer from a kit when Eben Moglen came to campus.

Moglen, a professor at Columbia Law School and a staunch advocate of free, open-source software, had come to NYU to give a lecture he titled "Freedom in the Cloud." Addressing a packed auditorium, he laid out the argument that cloud-based services built on proprietary software, such as Google Docs and Yahoo Mail, were an assault on user privacy and, from a programmer's perspective, utterly unnecessary. As for Facebook, Moglen chided, the company offers its users "a very poor deal, namely, 'I will give you free Web-hosting and some PHP doodads and you get spying for free all the time.'….This is bad. I'm not suggesting it should be illegal. It should be obsolete! We as technologists should fix it!"

Moglen proposed what he called a "freedom box," which he described as "a really good Web server that you can put in your pocket and plug in anyplace." It would talk to your friends' servers, host all your data, and keep track of everything you did and everyone you met on the Web. And you wouldn't have to share that information if you didn't want to. Most important, the software that ran the box would be free and open source, so anyone could download it and tailor it to his or her needs.

"We thought this idea was really rad," Grippi says. "We decided, well, let's take a shot at making the social networking aspect of it 'cause we were getting a little fed up with Facebook."

While the guys admire Facebook and believe, as Salzberg says, that founder and CEO Mark Zuckerberg "sort of blew the lid open on what people are willing to do online," they find many things about his company unnerving, even maddening. They don't like that Facebook owns the data they share through the site and can mine or sell it to advertisers at will. They don't like that if they disagree with the company's privacy policy and choose to cancel their accounts and sign up with a different networking service or make their own, they can no longer socialize with friends who still use Facebook. "It sucks," Grippi says.

Above all, they don't like that most ordinary people and many Web engineers have come to believe that seven-year-old Facebook represents the be-all and end-all of everything online socializing will ever be. "Facebook is about distilling yourself always to the simplest thing," Salzberg says. "To think that we've arrived at the pinnacle of human communication, it's just…"—he pauses and lets his shoulders slump—"a bummer." Social networking is still a young technology, he argues; there's a lot left to be invented, and one company can't do it all. "Sending little messages and pictures between your friends isn't some convoluted technical masterpiece. So why isn't there an alternative?"

One night late last April in NYU's ACM office, the guys recorded a video pitch for their project and posted it on Kickstarter, a website that crowdsources funding. In the video, they sit shoulder to shoulder in front of a sliding chalkboard, facing the camera. Sofaer begins, his voice full and cadenced, like a preacher who truly believes his own sermon: "In real life we talk to each other. We don't need to hand our messages to a hub and have them hand it to our friends. Our virtual lives should work the same way."

They couldn't have timed their pitch better. Five days earlier, Facebook had yet again changed its privacy policy, this time making the site's default settings such that anyone browsing the Internet could see a user's status updates, photos, and most of the user's profile information. Two days after that, Zuckerberg announced a newly developed protocol he called Open Graph. It lets other sites, such as Yelp, Pandora, and CNN, access Facebook users' public profile data—name, gender, hobbies, friends, songs they like, songs their friends like, articles they recommend, restaurants they've tried—and use the data to customize the users' experiences on those sites without making them create new accounts.

Many Facebook members were outraged that the photos and updates they thought they had shared only with friends were now being handed over to companies. Donations for Diaspora came rolling in. A reporter from The New York Times called and interviewed the guys. The CEO of Pivotal Labs, a software consulting company in San Francisco, invited them to use its office space for free. Even Mark Zuckerberg pitched in. (He told a reporter that he identified with the guys' ambition to change the world and thought Diaspora was a "cool idea," although he wouldn't say how much money he had donated.)

The guys had given themselves 39 days to raise $10 000 for a summer project. When the fund-raiser ended on 1 June—before they'd written a single line of code—they had $200 641 from 6474 donors.

That day, they caught a plane to California to begin the most ambitious hacker project any of them had ever tackled.

Pivotal Labs occupies the third floor of a drab, gray building sandwiched between a CVS drug store and a fitness club on Market Street in downtown San Francisco. The office is open and airy. There are rows of tables cluttered with big-screen monitors, a couple of Ping-Pong tables, and a kitchen stocked with coffee, cereal, granola bars, fruit in wicker baskets, and Mexican Coca-Cola. Dozens of programmers are usually coding in pairs or troubleshooting problems in small groups, and so the room is always humming, like a café in the afternoon.

The Diaspora guys sit side by side at one of the tables, surrounded by boxes overflowing with Diaspora T-shirts and paper plates piled with pistachio shells.

When they arrived at Pivotal last June, they knew their project would be less a summer experiment than a full-time job. They created a corporation, partly because they needed a place to keep their money that wasn't Salzberg's checking account, and partly because they knew that eventually the money would run out and they'd have to solicit investments from venture capitalists.

Their plan was to code all summer, get the false starts and embarrassing oversights out of the way, then release what they had on the code-hosting site GitHub in mid-September. They would license the software under the Affero General Public License (AGPL), which would ensure that the Diaspora code remained free to anyone to use and play with and that whoever changed it or added to it had to release the code under AGPL, too. That's the beauty (and some say the curse) of free, open-source software. It didn't matter that Diaspora was just four dudes out of college, because now there was a whole world filled with smart programmers who could help them. Today more than 100 people have contributed to the Diaspora project, and the code has been translated into 40 different languages, including Swedish and simplified Chinese.

When at last the guys sat down to type the first lines of code, they knew that several veteran programmers, including some at Google, were already experimenting with many of the protocols needed to build a federated social network. A few programmers were even leading their own open-source projects, among them StatusNet, BuddyPress, OneSocialWeb, and the Appleseed Project.

The pieces of the puzzle were out there. The guys just had to fit them together.

The first thing users of any online social network need to be able to do is meet one another. On a centralized network such as Facebook, finding new friends is easy. In fact, Facebook often does this for you by recommending people who share your affiliations or who know your friends. If you're looking for a particular person, you simply type in the name. Facebook's system queries its database and voilà—up pops a picture along with a list of the person's friends. Unlike browsing blogs or sending e-mail, socializing on Facebook lets you check people's identities to help determine whether they're the kind of people you want to introduce yourself to.

Meeting people on a federated network of many separate servers with many separate databases is trickier. To let users identify one another as they would on Facebook, Diaspora's software follows a sparsely used protocol known as WebFinger. It was developed in 2009 by a couple of engineers at Google and modeled after an old Unix protocol called Finger. Users of a Unix network could type in the command "finger username@hostname" and receive information about Mr. User Name, such as a phone number and whether the person was logged into the network. WebFinger does something similar by using the language of the Web (HTTP), rather than simply the language of the network (TCP/IP).

For example, suppose I have an account on the Diaspora-powered pod My-seed.com, and you have an account on Diasp.org, and that "me@my-seed.com" would like to find "you@diasp.org." (The guys call such identifiers "handles.") I type your handle into a search box on my stylized user interface and the Diaspora software on my pod routes my query to a file on your pod that contains your WebFinger profile—basic stuff about you, such as your full name and your picture. Diaspora's software then extracts that information from the file and dumps it into an interface template, which displays it on my screen, much like a public Facebook profile.

The downside to the WebFinger system is that if I don't know your handle, I can't find you any other way. I can't search for you by your name or by your job title, for example—unless, of course, you are already friends with someone else on My-seed.com, which means its server cache knows you exist and has stored your WebFinger profile.

The ability to let users search its database is perhaps the biggest advantage Facebook has over a federated network such as Diaspora and the biggest threat the company poses to Google. Because Facebook encourages its users to list their hobbies, affiliations, likes and dislikes, and tag photos, videos, and events, the site has become a sizable search index. Users can search for everyone who collects Raytheon CK722 transistors, pull up photos of all of them, and poll other Facebook users about whether they prefer iPhones or Android phones before buying one.

"Search is a huge problem," Salzberg concedes. "Everybody's always saying, 'Oh, my God! Facebook is awesome because I can search for anyone!' And you can't do that on Diaspora. But we're not going to quit trying to solve the distributed networking problem just because we don't know how to do search. That's something some company the size and scale of Google will need to solve." In the meantime, there are other, more basic things Diaspora's distributed system must do to "mask the distributedness," as Salzberg says, "and make it look and feel the same as a centralized website."

Suppose now that I, at me@my-seed.com, have used your handle, you@diasp.org, to call up your WebFinger profile. Next, I need to be able to communicate with you. In the Diaspora system, any personal messages I send or private updates I post are encrypted with standard RSA cryptographic keys and "pushed" from my pod to your pod using HTTP. "Push" technologies, which are also used in instant chat and for forwarding e-mails from an online account to a smartphone, are distinct from "pull" technologies, used for RSS feeds and browser requests, in which the servers receiving content constantly ask the servers generating content for updates.

Because most conversations on social networks happen nearly in real time, Diaspora's message distribution system must be as efficient as possible. As you can imagine, a push system distributes messages more efficiently than a pull system, but it also makes a lot of demands on the server doing the pushing. The Diaspora guys are therefore trying out a new protocol being developed by Google engineers for dispersing public posts, which don't need to be encrypted and will likely be sent to more people than private posts.

The protocol, named PubSubHubbub, calls for an intermediary server, or hub, and assigns it the task of handing out updates from a publishing server to its subscribers. Using a cloud-based hub, such as the one Google runs, lets you host data on your own small, cheap server while the hub takes care of publishing your updates for you. Diaspora's software then packages the updates using an open-formatting protocol called Activity Streams—developed by engineers now employed at Google, Microsoft, Socialcast, and elsewhere—to add useful identifying information to each update, such as who wrote it, when it was written, and what kind of message it is (a status post? an article link? a new location? a preference?). The software run by subscribing Diaspora customers then uses that information to display the updates in an aesthetic, engaging way, much like an RSS or Atom feed reader.

The final element of what Diaspora's Zhitomirskiy calls the "social networking atomic units"—meeting people, sending private messages, publishing public updates—is commenting. For this, the guys adopted a protocol called Salmon, developed by Google engineer John Panzer. If a Diaspora user comments on a post, the comment gets pushed "upstream" to the person who wrote the post. The post author then pushes the comment out to his friends.

Last 15 September, as planned, the guys made their code public. Since then, they have been adding features and little by little "improving the user experience" as more and more fans ask for invitations to join the network. "It's kind of like building an airplane as you're flying it," Salzberg says.

Critics of Diaspora say that assembling a social network on the fly probably isn't a good strategy. They predict that Diaspora's missteps—the security holes, the broken features, the lost accounts, the misrouted messages, the delays, the crashes, the inconsistencies—will drive users away before the company can even get off the ground.

Some programmers who are designing their own federated networking systems argue that a better approach may be to first create standard protocols and libraries that established networks, such as Myspace, China's Renren, and Russia's VKontakte can use to interoperate as if they were Diaspora pods. "The problem with Diaspora right now is it's not designed to work with other providers out of the box," says Ben Zhao, a network security expert at the University of California, Santa Barbara [listen to an interview with Zhao]. Zhao is developing a mobile social network architecture called Polaris that, if widely adopted, he says, would assemble existing social networking services into a single federated system. To be fair, the Diaspora guys are working toward linking up their pod with other networks, starting with StatusNet, which targets businesses and organizations, and Identi.ca, an open Twitter-like service built on StatusNet software.

"I would argue there's also a lot of value in putting all the pieces together in a nice package with a nice name," says Smarr, the Google social Web engineer. "There's nothing like building an end product to figure out where the holes are in the building blocks."

Of course, he adds, "agreeing on the building blocks and getting the big, established players to see the benefit of being part of these technologies" won't be easy. Not for Diaspora, and certainly not for Google, agrees tech analyst Enderle, "especially if the larger, established networks see Google as a competitor."

It's most likely then, that if federated social networking does pervade the Web, it will take over slowly, like ivy enveloping the brick halls of Harvard University. At first, open-source projects such as Diaspora will grow steadily and haphazardly, all the while tweaking their technologies, working out standards, and syncing with each other. Then, when they attract enough users to seem like they're onto something cutting-edge and lucrative, perhaps other niche networks—the Quoras, the Goodreads, the Foursquares—will join the federation. Developers will create new start-ups and invent features and experiences that Facebook can't offer: private networks, anonymous networks, exclusive networks, ephemeral networks.

"If you take all of Facebook's competitors and tie them all together into a large web of choices," Zhao predicts, "then when the next big Facebook privacy gaffe happens, people will actually jump ship because there will be a good alternative." At last, Facebook will face a real killer.

And it just might look an awful lot like Google.

This article originally appeared in print as "The Anti-Facebook."

This is part of IEEE Spectrum's special report on the battle for the future of the social Web.