reddit
Slashdot
Digg
StumbleUpon
delicious
Facebook
Twitter
Oregonian/Landov [bottom]
Google also keeps server frills to a minimum. Like Facebook, it buys commodity-level computing hardware and just fixes the many pieces that break, instead of purchasing high-end systems that are less prone to failure but also much more expensive. Economics, if nothing else, drove engineers at both companies to similar conclusions here. Fit and finish might count if you're buying one server or even a hundred, but not when you're shopping for tens of thousands at a time. And striving for high reliability is a little pointless at this scale, where failure is not only an option, it's a daily fact of life.
Facebook's Michael explains that he helped design three basic types of servers for running the Facebook application. The top layer of hardware, connected most directly with Facebook's many users, consists of outward-facing Web servers. They don't require much disk space—just enough for the operating system (Linux), the basic Web-server software (which until recently was Apache), the code needed to assemble Facebook pages (written in PHP, a scripting language), some log files, and a few other bits and pieces. Those machines are connected to a deeper layer of servers stuffed with hard disks and flash-based solid-state drives, which provide persistent storage for the giant MySQL databases that hold Facebook users' photos, videos, comments, and friend lists, among other things. In between are RAM-heavy servers that run a memcached system to provide fast access to the most frequently used content.
Alpha geeks will recognize that these pieces of software—Linux, Apache, PHP, MySQL, memcached—all hail from the open-source community. Facebook's programmers have modified these and other open-source packages to suit their needs, but at the most basic level, they are doing exactly what countless Web developers have done: building their site on an open-source foundation.
Not so at Google. Programmers there have written most of their company's impressive software from scratch—with the exception of the Linux running on its servers. Most prominent are the Google File System (or GFS, a large-scale distributed file system), Bigtable (a low-overhead database), and MapReduce (which provides a mechanism for carrying out various kinds of computations in a massively parallel fashion). What's more, Google's programmers have rewritten the company's main search code more than once.
Speaking two years ago at the Second ACM International Conference on Web Search and Data Mining, Jeff Dean, a Google Fellow working in the company's system infrastructure group, said that over the years his company has made seven significant revisions to the way it implements Web search. However, outsiders don't realize that, because, as Dean explained, "you can replace the entire back end without anyone really noticing."
How are we to interpret the difference between Google's and Facebook's engineering cultures with respect to the use of open-source code? Part of the answer may just be that Google, having started earlier, had no choice but to develop its own software, because open-source alternatives weren't yet available. But Steve Lacy, who worked as a software engineer for Google from 2005 to 2010, thinks otherwise. In a recent blog post, he argues that Google just suffers from a bad case of not-invented-here syndrome. Many open-source packages "put Google infrastructure to shame when it comes to ease of use and product focus," writes Lacy. "[Nevertheless, Google] engineers are discouraged from using these systems, to the point where they're chastised for even thinking of using anything other than Bigtable/Spanner and GFS/Colossus for their products."
Might Google's or Facebook's infrastructure yet crack under their ever-increasing loads? Facebook's regular user base has mushroomed to more than half a billion people, and it continues to add more than 20 million users a month. And Google must devote vast computing resources to keep up with the 34 000 searches it performs each second, while running ad auctions, translating languages, handling Gmail traffic, hosting YouTube videos, and more. Can all this just go on and on without end?
It seems it can. While the costs are enormous, these companies appear to be handling the computing burden with relative ease. But maybe that shouldn't be surprising. After all, if they don't have adequate horsepower, they can always delay the introduction of whatever resource-intensive service they're working on—and they both roll out such features regularly enough.
Take Google Instant, an instant-search function that the company introduced last September. "The point was to increase the delight factor," says Ben Gomes, who headed the Google Instant team. Instant looks at the first letters you key into a search query and offers a page of results based on what it anticipates you intend to type. So for even a simple query, multiple searches must now go on in parallel, and further calculations must be carried out to choose which results to show. Would Google have made such a change if its engineers had any doubts about the ability of their systems to take the punishment?
Similarly, Facebook can certainly control how heavily its users tax its system. As of last year, for instance, Facebook users had uploaded 50 billion pictures to the site. And yet, despite the enormous amount of bandwidth and storage space eaten up by all those photos, Facebook has periodically boosted the resolution of the images users can save. Would it have done that if its operations engineers felt their computing infrastructure was in danger of collapse?
My point is that neither Google nor Facebook is likely to falter in scaling up their systems to match demand. Sure, there will be glitches and slowdowns from time to time. But it seems unlikely that either company will suffer from a long-term lack of computing oomph as they continue to shape the way we run our online lives. Just how quickly they'll have to build new data centers, and what new kinds of energy-saving technology those centers will contain, is anyone's guess. But one thing's for sure: Their servers will always be ugly.
This is part of IEEE Spectrum's special report on the battle for the future of the social Web.
