“Engineering productivity is hard to measure,” said Peter Seibel, the tech lead of Twitter’s engineering effectiveness group. “But we certainly can harm it.”
Seibel was speaking at @Scale, a conference hosted this week by Facebook that brought together 1,800 software engineers from some 400 companies—all building applications that will potentially be used by millions or billions of people. Seibel told the story of the evolution of software at Twitter—a Babel of different programming languages including Ruby, Java, and Scala that made it hard for different groups of engineers to work together, but was eventually (mostly) fixed.
“As an industry we know how to scale up software,” he said. “We also know how to scale up organizations, to put in management that lets thousands of people work together.”
“But we don’t have a handle on how to scale up that intersection between engineering and human organization. And maybe we don’t understand the importance of that.”
“We massively underinvest in this kind of work,” he said.
Seibel believes that groups of software engineers can be more effective if they take the right number of people away from actually working on the product and assign them to supporting those engineers. “For 10 people,” he says, “don’t devote anyone to engineering effectiveness.” People will figure out what they need on their own. “In 100 engineers, devote two people” to making tools and other support better, he says, “and they’ll be as productive as 101 engineers.”
In a group of 1000, he suggests, 255 engineers need to support the rest, and they’ll be as effective as more than 1400 engineers. And in a group of 10,000 (the size of Facebook’s engineering team), one-third of the engineers need to be working on engineering effectiveness.
Those effectiveness engineers can do a host of little things that chip away at productivity problems, and every one percent improvement, in a large organization, adds up. Reducing compile time just five minutes a day, for example gives engineers 1 percent more real working time. Reducing the number of times tools break—even if each incident just causes an interruption of a minute or two—can bring about huge productivity improvements. Seibel explains that each interruption takes an engineer out of “flow,” and studies show that it typically takes 15 minutes to enter a flow state. Good tools are also important, Seibel said, because they are simply fun to work with. “We should provide good tools for the same reason that we provide good food: they make work more enjoyable.”
Engineering effectiveness teams can also work to reduce what Seibel calls Tech Debt, that is, problems that have been dealt with in a less than optimal fashion in order to allow engineers to move forward, though they recognize that the solution will have to be cleaned up later. “Tech debt compounds over time,” he said. “A little bit of tech debt when there are ten engineers kills you when you get to 1000 engineers.”
Seibel admits he has yet to solve the problem of tech debt at Twitter. “My dream is having a team of senior engineers,” he says, “who can drop in on some code, make it better, and then get out of town. There could be huge engineering gains if you had people who could go in and fix some of those things you never have time to do.”
Tekla S. Perry is a senior editor at IEEE Spectrum. Based in Palo Alto, Calif., she's been covering the people, companies, and technology that make Silicon Valley a special place for more than 40 years. An IEEE member, she holds a bachelor's degree in journalism from Michigan State University.
