Apollo 13, We Have a Solution: Part 2
With The Lunar Module's life support systems coming on line, the immediate threat of death to the crew had been suspended, and it was time to start thinking about how to get the astronauts home.
Jerry Bostick was the chief of the flight dynamics branch, the part of mission control that looks after a spacecraft's trajectory--where it is, where it's going, where it should be, and how to get it there. The controllers of the flight dynamics branch sat in the front row of mission control, which they had proudly dubbed "the Trench." As they listened to the crew in space and the systems controllers in the row behind them struggle with the explosion's aftermath, "we went into the mode of okay, well, can we come back home immediately?" remembers Bostick. The Trench soon calculated that if the crew used the Odyssey's main engine and burned every last drop of fuel, they could turn around and come straight back to Earth, in a procedure known as a direct abort.
But the main engine was in the service module, and who knew what damage had been done to it? It might malfunction: in the worst case, firing it up could result in another explosion and kill the crew instantly. The other option was to let Apollo 13, carried forward by its momentum and the moon's gravity, go around the moon. There, gravity would pull Apollo 13 around the back side of the moon, accelerate it, and sling the spacecraft back toward Earth. This journey would take several days, however, and the lunar module was intended to support only two men for two days--not three men for four. If the crew didn't get home fast, they could run out of power and die.
Kranz says this was his toughest call on Apollo 13. "My team was pretty much split down the middle. Many of my systems controllers wanted to get home in the fastest fashion possible. The trajectory team did not want to execute a direct abort because it had to be executed perfectly. If we didn't get the full maneuver, more than likely we would crash into the moon," he explains, "I was of the frame of mind that said, 'Hey, we don't understand what happened here...and if we execute a direct abort, we're not going to have much time to think about it...We needed to buy some time so that when we did make a move, it would be the proper move.' "
Weighing the concern that the Aquarius wouldn't cut it on a longer return journey, Kranz told Spectrum he had "a lot of confidence in my lunar module team." Apollo 13 was Kranz's fourth mission involving a lunar module. "I knew it was a very substantial spacecraft...I was pretty much betting that this control team could pull me out of the woods once we decided to go around the moon."
Kranz made his decision. The main engine was out. Apollo 13 was going around the moon.
There was, of course, a fly in the ointment. During earlier Apollo missions, the outgoing trajectory of the spacecraft had been selected so that if the service module's main engine failed for any reason, the slingshot effect would aim the command and service module perfectly at Earth, a so-called free-return trajectory. But this trajectory put very tight constraints on the mission timeline, and for Apollo 13, it had been abandoned.
"We were on a non-free-return trajectory. If we did nothing, we'd whip back towards the Earth but miss it by several thousand miles," the Trench's Bostick explains.
As the question of trajectory was being decided a shift change was going on at mission control. When the explosion occurred, Kranz and his controllers--collectively known as the White Team--had been about an hour away from the end of their shift. As was common, most of the next shift--the Black Team, led by Glynn Lunney--had already shown up, so as to be able to take over running the mission seamlessly from their predecessors, and they had been on hand throughout the crisis.
As Kranz's team gathered up to leave mission control, Bostick went to speak to the incoming flight director, Lunney. By good fortune, Kranz and Lunney were perfectly matched to the different phases of the crisis they would be faced with. Kranz was a systems guy--he knew the internals of the spacecraft better than any other flight director, the ideal person to cope with the second-by-second equipment failures and reconfigurations triggered by the explosion. Lunney had come up through the flight dynamics branch, making him ideally suited to get the spacecraft headed in the right direction.
>"Kranz was there at the right time to make the decisions that had to be made rapidly, and then, when Lunney took over he brought a calmness to the control center to do the right things once they had gotten stabilized...They turned out to be a wonderful pair," says their boss at the time, Kraft.
So Bostick speaking to the perfect audience when he voiced his concerns. "We need to get this thing back to a free-return trajectory," Bostick told Lunney. Lunney instantly agreed, but this left Bostick with a problem. Getting Apollo 13 onto a free-return trajectory required a solid push from a big engine. With the Odyssey and Aquarius docked together and the main service module engine out, that left only the engine attached to the lunar module's descent stage, designed to be used only for the relatively short period of time needed to land the Aquarius on the moon. "It was a problem, because we didn't have capability in the control center to calculate the result of a docked maneuver" using the descent engine, remembers Bostick.
During a mission, controllers called on a bank of mainframe computers in a Manned Spacecraft Center facility set up and maintained by IBM, known as the Real Time Computer Complex (RTCC), to calculate the length and direction of engine burns needed to produce a given trajectory. To do these calculations, the mainframes were programmed with information about the spacecraft, such as their mass, center of gravity, how much thrust the engine produced, and so on. Unfortunately for Apollo 13, the program to calculate how the conjoined command and lunar module could be maneuvered using just the descent engine simply didn't exist.
"So the first thing we did was call our computer guys and say 'Hey, call all the IBM guys in and start writing some software!" says Bostick with a laugh. As a backup, the mission planners who originally put together the Apollo 13 mission were called in to double-check the RTCC's results. "In 2 or 3 hours we were able to come up with a free-return maneuver. I think it made everybody feel a lot better--including the astronauts." Bostick remembers talking to the crew after the mission. "When we executed the free-return burn it made them feel that they might get out of this thing alive," he says.
Kranz's Team Hadn't gone home after its shift. The White Team now formed the nucleus of a new Tiger Team, dedicated to figuring out the fastest way possible to get the crew home, given that the spacecraft was going around the moon. They also had to work out how to stretch the lunar module's consumables to last the entire trip and how to get the command module reactivated and configured to survive a re-entry--the astronauts' only way to get home alive.
Arnie Aldrich, the CSM branch chief, had joined the Tiger Team, along with another EECOM, John Aaron. An hour before, Aaron had been at home, standing in front of the mirror shaving, preparing to come in for his shift, when his wife brought him the phone, saying his boss, Aldrich, was on the line. Recalls Aaron, "He said 'John, I need to ask you some questions. There's something significant that's happened out here and these guys can't quite figure it out. It's not going well.' "
Aldrich called Aaron for a couple of reasons. One was that Aaron was an expert on the command and service module's instrumentation system. The other was that Aaron was one of the best mission controllers in NASA.
Four months earlier, Aaron had saved the Apollo 12 mission when, during launch, the rocket was struck by lightning--twice. The second strike knocked the CSM's fuel cells off line, sent the guidance system spinning, and scrambled telemetry to the ground. With warning lights blazing and alarms sounding, it looked like the crew would have to abort the mission, scant seconds after liftoff.
Aaron was in the EECOM's seat for the launch, and as he watched the scrambled data ripple across his console, he was suddenly reminded of a ground test he had seen a year earlier where an electrical malfunction had caused a similar problem. The crazy pattern of the data on his console "was a pattern that I remembered," says Aaron. And, thanks to hours of research he'd put in after the ground test, he knew how to fix it. He uttered the terse command, "Set S.C.E. to Aux," to his flight director, Jerry Griffin. Griffin, like everyone else in mission control, had no clue what that meant. Nevertheless, trusting in his EECOM, Griffin ordered the command to be passed up to the crew immediately. The corresponding switch was flipped onboard and valid telemetry was restored. With valid data, Aaron could see that the fuel cells were off line, and with a second command to reset the cells, Apollo 12 was on its way to moon. The incident cemented Aaron's reputation as a "steely-eyed missile man."
So, when Apollo 13 ran into trouble, Aaron was Aldrich's go-to guy. "I had a very good group of people working for me at the time of the explosion, but we were scratching our heads, and the very best person I had was John Aaron," says Aldrich.
After the explosion, Aldrich had moved into the spacecraft analysis, or SPAN, room, located across from mission control. The SPAN room was fitted out with more consoles and acted as a bridge between the flight controllers and the army of engineers who had actually designed and built the spacecraft. "In there were supervisors like me and executives from the engineering organizations in NASA and the manufacturers, and this group would sit together and monitor the flights," says Aldrich. The SPAN room had come into being because "we learned during Mercury that we wanted immediate access to the manufacturers, that we needed clear and unfiltered data very rapidly," says Kranz.
Over the phone, Aaron asked Aldrich to walk around behind the consoles in the SPAN room and describe what he saw. "I started asking him: tell me what this measurement says, tell me what that measurement says. And that went on for about ten minutes," says Aaron.
In the data Aldrich read to Aaron, Aaron was looking for a pattern that would map to failures in the instrumentation system onboard the Odyssey, but he was coming up empty. "I told Arnie, 'Well, I'll be right there. In the meantime tell those guys they've got a real problem on their hands,' " says Aaron.
As the lunar module controllers raced to power up the Aquarius, Aaron had made it in to mission control. "When I walked in the room, I intentionally did not put a headset on because I could see each of the flight controllers had zoomed in and were trying to sort the problem out from the perspective of their individual subsystem," he says. He walked behind the controllers, looked at their data, and listened to what they were saying to the back rooms. Finally he sat down beside the embattled command and service module controller Liebergot and plugged his headset in. "I said, "Sy, we've got to power the command module down," recalls Aaron.
Aaron didn't just want the command module powered down to minimal systems only. He meant powered down as in off. No guidance system, no heaters to keep back the cold of space, no telemetry to help controllers diagnose the problem. Nothing. Aaron was concerned that even a minimal power draw from the batteries would leave them with nothing for re-entry.
Aaron remembers debating with Gary Coen, one of the controllers with responsibility for Odyssey's guidance system. "He was pleading with me to leave the heater circuit on in the inertial platform in the CM," says Aaron. The inertial platform, which gave the computer raw data about which way the spacecraft was pointing, was never designed to handle extreme cold. "He said 'John, [the heater] only takes 0.4 amps..if we turn it off, the platform may never work again.' And I said, 'Well, Gary, just do the math. 0.4 amps times 48 hours--we gotta turn it off. If it doesn't work again, we'll just have to figure out how to get home without it.' "
But Without The Odyssey's Guidance System telling the crew precisely which way they were pointing in space, how would they be able to align the spacecraft correctly to perform the free-return trajectory maneuver?
The answer was to rely on the lunar module's guidance system, which had at its heart an identical computer to the one in the Odyssey's guidance system. However, the lunar module's guidance system had been powered off for most of the way to the moon--it had no clue as to which way it was pointing. The crew would have to transfer the alignment information manually from the command module's computer to the lunar module's computer before pulling the plug in the Odyssey.
Doing so would require some good old-fashioned arithmetic. "You could read the angles out of one computer and type them into the other, but you had to invert them," because the Odyssey and Aquarius were docked head to head, and therefore pointed in opposite directions, explains Aaron. The job fell to Lovell onboard the Aquarius, but "because I had made mistakes in the arithmetic several times during sims..I asked the ground to confirm my math," said the commander afterwards. The Trench broke out pencil and paper and confirmed the angles.
As soon as possible after the crew aligned the lunar module's guidance system for the free-return trajectory maneuver, they shut down the command module completely. In the end, the inertial platform heater circuit breaker "was the last circuit breaker we pulled," says Aaron.
Now Aaron And The Other Members Of The Tiger Team were gathered in a room near mission control. Kranz soon arrived and looked around the crowded space. The controllers were subdued and shaken--they had failed to contain the crisis, and the crew was still in extreme danger. But the last thing the astronauts needed was for controllers to begin second-guessing themselves.
Confidence was part of the bedrock upon which mission control was built. When prospective controllers joined NASA, often fresh out of college, they started out by being sent to contractors to collect blueprints and documents, which they then digested into information that mission controllers could use during a mission, such as the wiring diagrams the lunar module controllers had used to figure out how to power up the Aquarius. After that, the proto-flight controllers started participating in simulations. The principal problem NASA had with these neophytes was "one of self-confidence," explains Kranz. "We really worked to develop the confidence of the controllers so they could stand up and make these real-time decisions. Some people, no matter how hard we worked, never developed the confidence necessary for the job." Those not suited for mission control were generally washed out within a year.
Now Kranz feared his controllers, battered by the events of the last hour, would lose their nerve. What happened next was a spectacular moment of leadership. "It was a question of convincing the people that we were smart enough, sharp enough, fast enough, that as a team we could take an impossible situation and recover from it," says Kranz. He went to the front of the room and started speaking. His message was simple. "I said this crew is coming home. You have to believe it. Your people have to believe it. And we must make it happen," recalls Kranz.
In the Ron Howard movie, this speech was "simplified into 'Failure is not an option,' " chuckles Kranz, who never actually uttered the now famous phrase during the Apollo 13 mission. Still, Kranz liked it so much, because it so perfectly reflected the attitude of mission control, that he used it as the title of his 2000 autobiography.
Kranz's speech electrified the room. "Everybody started talking and throwing ideas around," remembers Aaron.
Kranz appointed three flight controllers as his key lieutenants. Aldrich was put in charge of assembling the master checklist for powering the command module and other re-entry procedures. A lunar module controller, William Peters, was ordered to make sure the Aquarius lasted long enough to get the crew close to Earth. And Aaron was put in charge of devising how electrical and other life support systems would be used so that as the crew turned on the command module again prior to re-entry, they'd be able to get it up and running and complete the descent through the Earth's atmosphere before the batteries were exhausted.
Aaron's main problem was that, as with the Aquarius, powering up the command module was a complex procedure, made even more difficult by the fact that, unlike the lunar module, the Odyssey was never supposed to be powered down at any point during the mission. "The only power-up sequence we knew was the one that started two days before launch," Aaron remembers. But judging by what was left in the Odyssey's batteries, "we had just a couple of hours at full power," he says.
Aaron listened to the hubbub of ideas on how to get the command module going and decided it was time to step in. "I started throwing some ideas out as to how the power-up sequence could be altered," he told Spectrum. Controllers immediately started to object, explaining why it was vital that one aspect or another of the sequence remain untouched.
Aaron decided to chuck them all out of the room--with the exception of Jim Kelly, a backroom command and service module controller who specialized in the electrical power system--to give himself a chance to think. "I said 'Go get some coffee and come back here in 45 minutes, and Jim and I will have a timeline of what we can turn on and when for a rudimentary re-entry sequence."
Aaron and Kelly took some paper and started sketching out a timeline, blocking out how much power each system in the command module would use as it was brought on line. "We didn't have any computer programs to do this," says Aaron. But, thanks to the simulations, the pair had been trained in "all kinds of situations where power failures happened. Mostly we were just sketching the timeline out from memory and what we had learned from training," says Aaron.
The other controllers returned to find a big block diagram drawn on the blackboard. "They came back in, and I started describing" the timeline, remembers Aaron, "That started the brokering process, because every controller still wanted their favorite piece of equipment on and the earlier the better."
The brokering process, with Aaron acting as the final arbiter, would continue for another two or three days, refining the timeline and fleshing it out until the sequence was finally ready. Aaron's work would raise his stock among his colleagues even higher. He "just had a knack for the job...He was always thinking ahead, always capable of making the best of a tough situation and getting us out of it," remembers Kraft.
Integrating the power-up sequence with other tasks that would have to be done before entry into a set of procedures that could be read up to the crew was Aldrich's job. The result, for a time the most precious document in the U.S. space program, started out as a typewritten document, but as it was revised over and over, it was "updated in pen and pencil...It was five pages long," says Aldrich, who still has the final checklist in his possession. "I haven't looked at it in quite a long time. I know where it is, but it's buried!" protested Aldrich when pressed for more details, revealing that he has kept quite a scrapbook from his 46 years and counting in the space business.
Kranz's Tiger Team worked closely with the inhabitants of the Mission Evaluation Room (MER), who were located in the building next to mission control. While the SPAN room was designed to act as a communications conduit between mission control and the engineers who had actually built and designed the spacecraft, the MER was where the problems posed by mission control actually began to be solved.
The MER was established during the Mercury program. In the early days of the program, the same people who built the spacecraft would staff the consoles in mission control. But it turned out that "people didn't have time to be responsible for the engineering and also put all the time in learning how to operate missions...so there was a split," says Aldrich. In mission control was a "mission team, which really knew how the flight was to be executed, what the possible trouble spots might be, and was prepared to deal with things that came up, but it frequently would need more engineering help," to deal with questions that cropped up concerning one piece of equipment or the other, Aldrich explains. In the MER would be "an engineering team that was pretty well informed, but which wasn't directly engaged with the flight on a first-hand basis," says Aldrich.
The MER was big enough to house dozens of engineers and if a problem couldn't be solved by those present, they could call on engineers throughout NASA's nationwide network of RandD centers as well as the engineers of the contractors who built the spacecraft. North American Aviation, based in Downey, Calif., (now part of Boeing Co., Chicago), built Apollo's command and service modules, while Grumman Aerospace, based in Bethpage, N.Y., (now part of Northrop Grumman, Corp., Los Angeles), built the lunar module.
As procedures for powering up the Odyssey or stretching the Aquarius's life support system were developed in mission control, hundreds of engineers in California and New York would test them out in the same factories where the spacecraft were built.
"In Apollo 13 movie, you see Grumman," trying to hedge its support for some of the risky tactics being employed by mission control, and "that did not happen," remembers Kranz, who is otherwise "very pleased" with Ron Howard's movie. [To see what the other controllers thought of the movie, see sidebar, "Mission Control at the Movies"]."The contractor support was absolutely superb," says Kranz determinedly. "The contractors knew what was at risk for every mission. If we had a problem and we turned to them, they gave us everything we needed."
After Apollo 13 Performed the free-return trajectory maneuver using the lunar module's descent engine, the debate went on about the fastest way to get the crew home. If no changes were made to the trajectory, the crew would splash down in the Indian Ocean in about four days. But there were no recovery forces to pick up the command module if it ended up in that part of the globe.
Bostick and his flight dynamics controllers immediately began working on how to shave some time off the return journey and have the splashdown happen in the Pacific, where all the recovery forces had already been deployed. "We concluded we could do that fairly easily and speed up [the splashdown] by about 12 hours, but we had also worked up an option that would get back to the Pacific and speed it up by 36 hours," says Bostick. But the 36-hour option would have involved jettisoning the service module immediately, exposing the all-important re-entry heat shield to space for a long time, and required nearly every drop of fuel left in the Aquarius's descent stage. Neither of these actions sounded appealing.
In any case "by then the systems guys had really done a bang-up job of squeezing the consumables" in the lunar module, says Bostick. They had done this principally by turning off nearly every system in the Aquarius except for guidance, communications, and a water/glycol cooling system that was needed to stop certain systems from overheating.
"Most of the water [onboard] was used for cooling; it was our most critical resource," explains Legler, who was the lead controller responsible for managing the Aquarius's power and water usage. The two consumables were interrelated; the fewer systems that were turned on and drawing power, the less water would be needed for cooling. Normally, fully powered up, the Aquarius's systems drew 50 to 75 amperes, and by dint of hard work, "we powered it down to about 12 amps," says Legler. Twelve amps is about as much power as a vacuum cleaner uses. Unfortunately for the crew there was no power in the budget to run heaters to keep the crew warm, and temperatures inside the spacecraft began to drop sharply.
With the Aquarius now expected to go the distance, the risky 36-hour option wasn't needed, and the 12-hour maneuver was chosen.
This required another burn from the lunar module's descent engine, one that would take place 2 hours after Apollo 13's closest approach to the moon. The point of closest approach was known aspericynthion, or PC for short, and so the trajectory adjustment was called the "PC+2 burn."
The PC+2 burn needed to happen exactly right, and the Trench insisted the lunar module's computer be used to control it. But the lunar module's guidance system used a lot of power, and the Trench agreed that if they could use it for the PC+2 burn, they wouldn't ask for it again. Almost exactly 24 hours after the oxygen tank explosion, the crew completed the burn and shut down the navigation system. From here on out, the astronauts would be flying by the seat of their pants.
This article is presented in three parts. For the final installment, click here.
Or go directly to Part 1 here.