The Game-Frame Guild

“This arrowy thing is a Barracuda spaceship, and these little spheres are asteroids,” says Tarqüínio “Tarq” Teles, pointing to a jumble of wire-frame outlines floating on a computer screen. “This is how our server sees the world. It keeps track of planets, ships, stations, everything. Shut it down and you shut down the universe.”

This is the virtual universe of Taikodom, a science-fiction online game that Hoplon Infotainment, a small company in Brazil, plans to launch next month. Teles, Hoplon’s hyperenergetic 37-year-old CEO, says that Taikodom will allow tens of thousands of people to play together in a sprawling virtual galaxy. The game is Brazil’s first incursion into the rapidly expanding market of massively multiplayer online games, or MMOGs. It’s a genre made famous by powerful franchises such as Sony Online Entertainment’s EverQuest and Blizzard Entertainment’s World of Warcraft. But Teles and his team are not intimidated.

“Our secret weapon is fun,” he says, “and we Brazilians know a thing or two about fun, right?”

A visit to Hoplon’s office helps confirm Teles’s claim. The start-up is based in a lush island city named Florianópolis that’s just off Brazil’s southern coast and only an hour’s flight from São Paulo, the country’s business center. With pristine beaches and a bustling town, Floripa—as the locals call it—is known for its laid-back surfing scene and vibrant nightlife. If you’re lucky, you may run into supermodel Gisele Bündchen in one of the trendy local clubs.

On this scorching November afternoon, sipping a caipirinha on the popular Brava Beach is an enticing idea. But over at Hoplon, nestled atop a wooded hill on the island’s north bay, the window shades are down, and the only tan the T-shirt-clad staffers are getting is from their computer screens. They hunker down in cubicles cluttered with Star Wars bric-a-brac and Frank Herbert novels, churning out Java code and three-dimensional models in hopes of meeting their tight launch schedule.

Just four years ago, when Teles and three friends—Tiago Luz Pinto, Carlos Eduardo Knippschild, and Cristóvão “Cristo” Buzzarello—started developing Taikodom, they worked out of Teles’s one-bedroom apartment. Now, after recruiting 72 employees and raising over US $10 million in venture capital funding, they’re hoping to capture a fraction of the booming MMOG market, which reached $3 billion last year and should double by 2012, according to game industry research firm DFC Intelligence, in San Diego.

But if there’s one factor that really sets Hoplon apart from other independent game developers, it’s the unusual game server the company adopted. Clusters of PCs are the workhorses behind most MMOGs. Hoplon, however, plans to run Taikodom on an IBM System z machine. For the uninitiated, that’s a mainframe. The Brazilians have partnered with Big Blue to develop a server optimized for large-scale multiplayer games by fitting a System z with Cell processors, the nine-core parallel-processing chips that power Sony’s PlayStation 3. IBM calls this bit-crunching beast the gameframe.

The two companies claim that their machine can do a better job than conventional servers at handling the two most demanding computing tasks in an MMOG: transactions and simulations. In Taikodom, the first job involves things like keeping track of each user’s spaceships, weapons, and virtual money. The second involves things like calculating the trajectory of objects and checking for collisions. Today’s typical game clusters handle both types of tasks, and they can host a few thousand users at a time. Add more and the servers slow down, and users experience lags in the game.

Games with lots of players like World of Warcraft have gotten around this problem by splitting the work among multiple clusters, creating duplicate worlds that don’t communicate. That means one subscriber can’t rendezvous with another unless they arrange to play on the same group of servers. IBM and Hoplon think they have a better idea: the gameframe’s hybrid hardware can divide the workload more effectively and hold all the users in a single universe.

“For Hoplon, it’s like the perfect machine,” says Joe Clabby of IT consultancy Clabby Analytics, in Yarmouth, Maine. “Mainframes are the best transaction engines, and when combined with the Cell to handle math-intensive tasks, the result is probably the sleekest, densest, most scalable architecture they could ever get.”

Hoplon and IBM are also creating a middleware layer called bitverse, a software platform that makes the underlying hardware transparent for programmers. For IBM, the gameframe and bitverse are products it can offer to customers interested not only in MMOGs but also in scientific simulations, virtual worlds, and a future 3-D Internet. It’s a market that has caught the attention of other big companies like Microsoft, Sony, and Sun Microsystems, as well as start-ups like BigWorld and Forterra.

For the four Brazilian gamers, the partnership with IBM is a dream deal: they get unusual access to high-end technology, and they save money on equipment because they rent all their mainframe resources from IBM and can request upgrades as their subscriber base grows. Their ultimate ambition, of course, is to see Taikodom, which is currently undergoing a public beta test, competing with the big titles—a goal they know will require much more than heavy-duty hardware.

It’s Friday, 1:30 p.m. , and a dozen Hoplon staffers cram inside a tiny conference room for the daily 15-minute status meeting. There are no chairs or even a table. Everyone has to stand. A board filled with yellow Postâ’’its hangs on one of the walls.

“The Post-its are our task-management system,” says Knippschild, a collected, detail-oriented 29-year-old in shorts and sneakers. “It’s based on a method called Scrum, used for agile software development.” The term scrum comes from rugby, he explains, and refers to a play in which several team members try as a group to gain control of the ball. The idea is to break from project management models that function as relay races, in which members work separately and in sequence.

Knippschild is in charge of keeping Taikodom’s development on track. Call him the Scrum Master. In the meeting, representatives from Hoplon’s art, software, and other departments report the tasks they have finished and those they plan to do next. They control the flow of assignments—fixing a bug with the game’s weapon interface, creating a new space station—using the colored notes, so that everyone can keep track of things at a glance.

After the meeting, Knippschild shows me how some of the groups work. In the game-design department, where a team of nine conceives the mechanics and visual elements of Taikodom, two long-haired guys dressed all in black are locked in a heated argument that has something to do with a “portable cyclotron.” At the other end of the office, in a room decorated with drawings of insectlike spaceships and bright movie posters, art director Samara Sena is busy generating, as she puts it, “textures and explosions.”

But what catches my attention is a quieter bunch nearby, their headphones on, lines of code flashing on their screens. The software team, Hoplon’s largest, is led by Pinto, the company’s chief technology officer. The bespectacled 31-year-old wrote his first Basic program at age 9, discovered Smalltalk as a teenager, and became an engineering student before dropping out to start Hoplon.

Pinto says he can’t show me the gameframe; it resides at an IBM data center in Hortolândia, outside São Paulo. But he offers to show me Hoplon’s server farm, where his team tests the software before uploading it to the remote site. The servers are in a wooden closet with sliding doors, packed with PCs of different sizes and colors. “We call it our server barn,” he says.

Like so many other business deals, the partnership with IBM began at a bar. Back in 2003, Teles and Buzzarello attended an IBM briefing on grid computing in São Paulo. Later that day, waiting at the airport for a flight back to Florianópolis, they met the IBM engineer, Marlon Machado, who had just given the briefing. With time to kill, the three sat at a bar, ordered beer, and began to sketch out an MMOG system architecture on a napkin.

They kept in touch, and in 2005, when they met again, Teles told Machado that they were having problems with their databases: whenever many users tried to log on at the same time, all attempting to retrieve masses of data with which to generate their view of the universe, the databases would crash.

“That’s when the idea of using a mainframe—designed for concurrent transactions—came about,” Machado recalls. He told Teles he would see what he could do. Two days later he called with good news. “Tarq, I got a server for you,” he said. “It’s a z9.”

With an architecture descended from the famed IBM System/390 of the 1980s, the z9, a refrigerator-size black monolith, is one of the world’s most powerful mainframes. It can have up to 512 gigabytes of memory and 54 central processors—some general purpose, others dedicated to cryptography or Java applications.

But the greatest thing about this high-powered, supersecure machine, Machado says, is that you can create hundreds of logical partitions, each with its own operating system and a share of the processors, memory, and storage. These partitions work as virtual servers, exchanging data directly through the mainframe’s memory at much faster rates than if they were connected through conventional networking interfaces.

“The mainframe saves you from having to have massive farms of x86 servers, consuming lots of power and generating lots of heat,” he says. “It’s a cluster in a box.”

Pacing around the office, Teles looks tired after returning from a weeklong game conference. But when I ask about Taikodom’s fictional universe, he becomes animated. He says the game is different from other space-themed MMOGs, including EVE Online, which hit the market in 2003 and now has 230 000 subscribers. Not only will players be able to form alliances and participate in large-scale battles, they’ll also be using social networking tools to create new kinds of interactions. One user may coach other players to help them obtain fighter-pilot certificates; another may use an in-game camera to broadcast battles.

“You can be a pilot, patroller, trader, miner, merchant. Or a spy, pirate, mercenary—maybe even a journalist,” Teles tells me.

The story, he continues, takes place in the 23rd century. Humanity has been living in space ever since a mysterious energy shield engulfed Earth. Nations don’t exist, and societies center around megacorporations. Some seek to build large industrial empires. Others dedicate themselves to hedonistic pursuits, like making beer. Human communities have a genetic pool that artificial intelligences use to generate new breeds. These AIs raise the kids as well. “Sex still exists, but no one wants to have babies,” Teles says. “The only ones who want to are the Atavic people, who believe in the Galactic Spirit. They are a crazy sect!”

The idea for Taikodom began to take shape when Teles met Pinto and Buzzarello at the Federal University of Santa Catarina, in Florianópolis. They enjoyed playing multiuser dungeon games, or MUDs, the early computer role-playing games based on text, running on the university’s mainframe. In the late 1990s, when they saw EverQuest, with its 3-D graphics and thousands of users around the world, they thought the market for such games was going to explode and decided they wanted to be part of it.

Their idea was to create a game without using the concept of levels and skills typical of existing MMOGs. “That’s when you choose a character—a dwarf, wizard, warrior—with certain skills, and you need to kill a monster to increase your level so you can kill a bigger monster and so on,” Teles says. “It’s called the treadmill model.” They wanted to give users more freedom and emphasize social interactions over mechanical, repetitive game play.

In early 2000, the trio put together a business plan and went after investors. But they came knocking just as the dot-com bubble was bursting, and all the doors closed in their faces. To bring in some cash, they took small jobs writing software for the Web and for scientific simulations. Just when the group was starting to run out of money, Knippschild, then finishing his engineering degree, joined as a Java programmer and mentioned that his uncle, Erich Muschellack, might be able to help.

“The greatest thing I saw was the opportunity of developing not only a game but an industry of games in Brazil,” say Muschellack, the founder of venture capital firm Idee Tecnologia, in São Paulo. “Their plan is very ambitious. But that’s good: they know they have to work 30 hours a day.”

In January 2004, after eight months of negotiating, Hoplon and Idee closed a first round of funding. The friends became partners—and their game, business.

When Teles told his staff that Taikodom was going to run on a mainframe, they were worried they’d have to learn PL/I, BAL, JCL, or some other esoteric mainframe programming language. But Teles told them that the mainframe ran not only specialized software but also Linux and Java. For the programmers, that was like handing them the keys to a Ferrari.

And not just any Ferrari. The z9 easily handled the transactional workload of the game but didn’t do so well with the floating-point math operations. So IBM decided to bring in its calculation turbocharger, the Cell processor.

“The technology vision here is to create a hybrid that has the best of both worlds,” says Roland Seiffert, a senior engineer at IBM’s Böblingen lab, in Germany, which designed the System z architecture and ported Linux to run on the mainframes. “It puts computations where they belong.”

Seiffert says that today’s multicore x86 processors perform parallel calculations by chopping a stream of instructions into threads. But the Cell beats the x86 because in addition to multiple threads, it has parallelism inside the threads. For example, to determine if a missile has hit a spaceship, you must compute the coordinates of the missile over time and check to see whether they coincide with those of the spaceship. You take a matrix containing the coordinates of the missile and use kinematics equations to calculate the future coordinates. Whereas an x86’s threads calculate the new coordinates sequentially, the Cell’s take sets of coordinates and perform mathematical operations on them all at once.

IBM used a Gigabit Ethernet interface to connect the z9 mainframe to the Cell chips, which are housed in blade servers, thin computers that sit on a rack like books on a shelf. Hoplon now has 10 virtual servers running SUSE Linux on the z9. In one test, the system was able to handle 700 spaceships in one section of the universe. That’s still very far from the company’s goal of supporting some 40 000 users at a time. But the Brazilians say the solution—more mainframe resources and faster networking equipment—is on its way.

IBM is replacing the z9 with the recently unveiled z10, a system the company says is 50 percent faster than its predecessor and provides the computing power of a cluster of 1500 x86 servers. And Hoplon, which had been sharing a mainframe with other IBM customers, will now have the new machine entirely to itself. The two companies are also relocating the z10-based gameframe to another data center with network switches and backbone connections better tuned to keep latency in the tens of milliseconds, as online games demand.

Still, if IBM wants to make the case that the gameframe is the ideal platform for MMOGs and virtual worlds, it has yet to demonstrate how it compares with other approaches. One alternative, also developed by IBM, is the system running EVE Online—a cluster of IBM blade servers with AMD processors and arrays of solid-state disks—which the game’s developer, CCP Games, says can host 41 000 simultaneous users in a single universe.

Another approach that challenges the gameframe concept head-on is Sun’s Project Darkstar, a Java-based open-source game server platform. Chris Melissinos, chief gaming officer at Sun, says that IBM is “throwing hardware at a software problem.” The gameframe, he says, forces developers to conform their games to its hybrid hardware. By contrast, Sun is building a game-agnostic platform that handles all computing tasks using conventional servers. “To the developer, it looks like he’s talking to one CPU on the back end,” says Melissinos. “In reality, it could be thousands of CPUs.”

Hoplon’s answer is that its bitverse middleware is a key piece of the gameframe and works much like Darkstar, except that it has more powerful hardware at its disposal. “The Cell can give MMOGs more realism, more advanced physics and AI, which has long been a challenge,” Machado of IBM says.

It’s Friday night in Florianópolis. I’m still peppering Teles and Pinto with questions about Taikodom, so we head out to Por do Sol Açoriano, one of the best seafood joints in town, where Hoplon’s founders have delivered many pitches to investors, partners, and of course, journalists. “The only reason Hoplon has survived to this day is because of the quality of life here,” Pinto says, gesturing from the spacious veranda to the sea.

A key factor that will distinguish Taikodom from other MMOGs, Teles says, is its elaborate story. Although the four founders had their ideas about creating fictional universes, they decided to get professional help, by hiring Gerson Lodi-Ribeiro, a famous Brazilian sci-fi author. The idea is that in Taikodom, players’ actions will have an impact on the story. “A single player may affect the universe’s fate,” Teles says.

Within the Brazilian game community, opinions are divided. Some say that Taikodom will show the world what Brazil can do with enough resources. But others have accused Hoplon of hyping its game, some even calling it vaporware. Flávia Gasi, a game journalist, says the problem is that Hoplon has announced the game’s launch on several occasions, only to pull back each time. “It seems that whenever the game is ready, the market has already advanced, so they start all over again,” she says.

Outside Brazil, Taikodom is still barely known. A small number of English-speaking players are testing the beta version. Some of them have banded together within the game to fight against Brazilian alliances. This could be just friendly rivalry, or it may be a sign of a bigger problem: a massive Brazilian presence could pose an entry barrier for non-Portuguese speakers.

I ask Teles why it’s taking so long to release the game. He sighs. “Just because we have an investor and a partner like IBM, people think we live in paradise,” he says. “They have no idea what we’re going through. It’s been work, work, work.”

Challenges, he says, have varied from expected hurdles—difficulty in finding good game programmers—to surprising snags, like the flame war that erupted in Taikodom ’s discussion forum after a player reported seeing a gigantic bee flying in outer space (it was a software glitch), or the time a water pipe burst in the office, soaking computers and destroying drawings.

The next month will be crucial for Hoplon, because this may be its only chance to get it right. The company plans to publicize Taikodom in Brazil and eventually amass thousands of subscribers. Then it will seek partners in North America and Europe for a late 2009 release.

“As they say, if you knew how hard it was going to be, you wouldn’t have decided to do it in the first place,” Teles says. “Then again, you wouldn’t have had a shot at making something successful.”