Moon Lander: How We Developed the Apollo Lunar Module (16 page)

After some general discussion of the responsibility for preparing and using reliability estimates, Shea drew the meeting to a close. “Well,” he said, “I think this meeting has accomplished its purpose. Joe Gavin, is there any doubt in your mind that your people were wrong?”

“None whatsoever. I’m sorry we gave everyone so much trouble.” Gavin was ashen-faced.

MIT had blown Grumman’s analysis out the water, showing that we had not dug deeply enough to properly understand and interpret the GE reliability data. We left the meeting with our heads hung low, figuratively beating our breasts.
¹²

This incident had a lasting negative effect on Grumman’s reputation on the Apollo program, and for a time it did not do my own personal reputation any good, either. Shea and others saw Grumman as a loose cannon meddling outside its jurisdiction—definitely not team players. They thought we could not be trusted and were particularly upset by what they saw as Grumman’s “holier-than-thou” attitude, ready to point an accusatory finger at the alleged transgressions of others. For years afterward we endured strained relations with MIT; not until we were jointly supporting flight missions with them did feelings improve. This incident reinforced NASA’s growing belief that Grumman required close oversight and constant detailed direction.

Combining Innovation with Discipline

As 1963 progressed Rathke and I realized that our initial headlong approach to LM design must become more disciplined and systematic. At first I was enthralled by the thrilling feeling that we were inventing something totally new and historic. “No one knows what LM should look like; no one’s ever done this before,” I told my staff at more than one early project meeting. We were simultaneously awed by the challenge and stimulated with the possibilities for achievement. But the ever-growing complexity of the design and the need to schedule and control activities and determine personnel skill requirements and assignments demanded a more rigorous technique of engineering leadership.

First we established a regular series of LM Engineering meetings. The weekly project engineering meeting was chaired by Bill Rathke, or me in his absence, and was attended by the LM System and Subsystem managers, the assistant project engineers for Systems and Subsystems (Whitaker and Carbee), and direct staff reports, about thirty people in all. At this meeting we reviewed items of general interest, including the previous week’s accomplishments and plans for the coming week. We discussed those problems or activities that required major emphasis, making key personnel assignments as needed. NASA’s latest directives, trends, and planned activities were also reviewed. The meeting was planned not to exceed one hour and a summary weekly report was published, emphasizing items discussed at the meeting but also including more routine reports from each Engineering group.

I chaired a daily series of technical staff meetings (TSMs) in my conference room, beginning at eight o’clock each morning. These meetings, which generally involved eight to twelve engineers from specific System or Subsystem groups, were progress review, problem-solving, and management-direction meetings, often focused on a single problem or issue. Carbee and Whitaker attended most of these daily meetings, and Rathke was there if the subject warranted. The appropriate system or subsystem managers and their deputies participated on their scheduled days, and Erick Stern or others from Systems Analysis and Integration attended most meetings. The TSM schedule covered every LM Engineering group at least once every two weeks.

These TSMs were scheduled for one hour but took longer if I thought the discussions were productive. I tried to keep the first two hours of the day, until ten o’clock, clear for these meetings. Conducting the TSMs every weekday was arduous and time-consuming but well worth the trouble. Many of the successful approaches to LM design were thrashed out at these working sessions. I tried to publish an agenda with the subjects to be covered at each TSM one cycle in advance, but these were often changed or modified by the press of daily events. The weekly program meeting, which Joe Gavin held at ten o’clock on Tuesdays, did not conflict with the TSMs, but NASA reviews, visits, and meetings frequently did, making TSM postponements and rescheduling commonplace. I clung doggedly to the TSM discipline throughout the four years that I served as LM project engineer and subsequently Engineering director, and I believe that this simple technique allowed me to provide “hands-on” guidance and leadership, and to pull the many diverse threads of the LM design and development together into a coherent whole. I viewed the LM Engineering staff as a marvelous symphony orchestra in which each section (system or subsystem discipline) must excel with its instrument (knowledge and skill), and where each would have a turn to solo (solve a problem or achieve a key milestone) upon which the success of the entire ensemble would depend. When their moments in the spotlight came, no LM Engineering group was ever found wanting; they all gave “star quality” performances.

Another vital technique for LM Engineering was the configuration control process. NASA had adopted a strict, rigorous specification and handbook for configuration control, which had originally been developed in the Department of Defense for the air force and navy ballistic missile programs. Although these requirements were new to Grumman, we were convinced they were necessary to achieve the quality and reliability levels demanded by the Apollo program, and we sent a number of key supervisors to be trained by NASA in the new professional specialty of configuration control. We formed Configuration Control to implement and interpret these requirements, but all Engineering groups were affected by this area.

Besides documenting and controlling the detailed elements of the design, Configuration Control formalized the systems engineering process. All system performance and functional requirements, starting with the very top level description of the Apollo program’s mission and objectives, were progressively broken down (partitioned) into more detailed design and performance specifications at lower and lower levels of the system. Finally this process reached the level of procurement and design specifications and drawings, from which parts, components, and assemblies were procured and built. This also was a new discipline to us that we implemented vigorously.

Key to this process and its documentation was a defined set of engineering drawing levels. For the LM spacecraft, level 1 was a single large drawing that showed, as interconnected boxes with annotated inputs and outputs, all the systems and subsystems that made up the LM and their predominant interactions. The level 2 functional diagrams took each system or subsystem and indicated their major elements and components and the inputs and outputs from each on a single large drawing.
¹³
Level 3 took the system or subsystem down to each functional element with detailed inputs and outputs and reference to performance specifications. Levels 4, 5, and 6 were the more familiar engineering design drawings used to manufacture and assemble the parts at progressively lower levels of detail. These were referenced upward to the higher level drawings.

The Level 2 functional diagrams were especially useful during the first two years of the LM program, when the design was maturing and we were making major choices in design approaches and functional redundancy. These diagrams were a principal tool in assessing the relative analytically determined reliability of alternative system configurations, and they served to document the selected result. The level 2 diagrams were prepared by Erick Stern and his Systems Analysis and Integration Group. As project engineer I had the final approval signature, which I applied after a review meeting with Carbee, Whitaker, Stern, and the appropriate system or subsystem managers.

One day in late November 1963 I was holding such a review on the rendezvous radar subsystem with a group that included Grumman’s LM radar experts and Frank Gardiner of RCA. We pored over the drawing and discussed
it for about an hour and a half, at which point I considered it ready to approve once a few minor changes were made. I then joined Joe Gavin, Bob Mullaney, Bill Rathke, and others at a meeting in Joe Gavin’s conference room where Gavin and Mullaney were relaying to us the results of a NASA Apollo management meeting they had attended in Houston the previous day. Suddenly the door burst open and a stricken-faced man announced, “The president’s been shot! In Texas.”

In the shocked silence that followed, Joe Gavin, breaking his usual reserve, said heatedly, “That’s not true! I just heard the news on the way over here, and it’s not true.”

Then Joe hurriedly retreated to his office to make calls, as did several others, leaving the rest of us stunned and anxious. One by one they filtered back into the meeting, their doleful features confirming the worst. Joe returned and dejectedly said he was wrong, the president had indeed been shot and was dead. Gloom took over. The meeting was adjourned as no one any longer had the heart for it.

When I reached my office, Erick was waiting with the corrected rendezvous radar level 2 diagram. One look told me that he had heard the bad news, but he said that because President Kennedy himself set the schedule deadline for this project, he would want us to keep pushing ahead.

Alone together in the office we quietly verified that the agreed changes had been made, and I signed the diagram and affixed the fateful date: 22 November 1963. When I finished the Engineering floor was almost empty; the news had come late in the afternoon and most people’s reaction was to go home, seeking the solace of private grief. The martyrdom of the young president who had started the Apollo program was a powerful motivator to everyone to overcome all obstacles and reach the goal he had established for us.

Other LM Design Changes

There were other significant changes in the LM design in late 1964 and into 1965. The circular forward hatch with cylindrical tunnel and docking ring was changed to a larger rectangular hatch without a tunnel or docking provisions. This change was made after the M-5 full-scale metal mockup review in October 1964 demonstrated that a spacesuited astronaut wearing his backpack could not get through the circular hatch without undue effort. Abandoning docking redundancy on the LM was rationalized because the command module only had one docking hatch, which was just as likely to fail as a lunar module hatch, in which case EVA would be required anyway. The rectangular shape and absence of the tunnel made forward hatch ingress and egress much easier and allowed us to provide a small platform at the top of the ladder. This made getting to and from the surface safer and easier.

The LM control weight was increased again in November 1964 to thirty-two thousand pounds at Earth launch (without crew), and the propellant tanks were resized accordingly. This increase was made possible by NASA’s further refinement of the cislunar (Earth/Moon) trajectories and a decrease in the assumed propellant allowance to the CSM for rendezvous. The LM descent-stage hover time was also reduced by one minute, to ninety seconds. This LM weight increase was the limit that could be squeezed out of the Saturn 5. The LM’s weight growth continued unabated each month, casting the specter of a difficult, costly, and schedule-busting redesign effort before us. Within a few months it would become my overwhelming preoccupation.

Additionally, in February 1965, after months of design studies, NASA approved the switch of the LM electrical power system from fuel cells to batteries. This was a change NASA had championed, as they became increasingly concerned about the growing complexity of the hydrogen-oxygen fuel-cell systems under development for both the LM and the command and service modules. The original LM proposal used fuel cells, and after contract go-ahead Grumman held a competition that was won by Pratt and Whitney, which was also building the fuel-cell system for the CSM. The LM fuel-cell system was under development at Pratt and Whitney for two years when it was canceled in favor of the “all battery” EPS.

Fuel cells generate electricity by reverse electrolysis, combining gaseous hydrogen and oxygen in a catalytic reactor containing fine-mesh nickel screens to produce electric power with water as a byproduct. Such a system was attractive for Apollo because it was relatively lightweight, producing more kilowatt-hours per pound than comparable battery systems. The longer the mission duration and higher the total kilowatt-hours of electrical power required, the greater was the fuel cell’s weight advantage over batteries. The waste water from the fuel cells could be used for cooling and possibly as drinking water, an additional advantage.

The main disadvantage of fuel cells was their relative complexity. They required tanks for hydrogen and oxygen, the reactor chambers (cells) and associated plumbing, valves, pressure regulators, controls, and instrumentation. Batteries, on the other hand, were self-contained “black boxes” that required only monitoring instrumentation and recharge circuitry to be integrated into the EPS.

Owen Maynard, NASA’s LM Engineering manager, was skeptical of the LM’s need for fuel cells from the beginning of the LM program. He asked me why we had proposed fuel cells rather than batteries in one of our first meetings during the contract negotiations in Houston. I justified our choice based on the weight advantage—more than 450 pounds—offered by fuel cells. A year and a half later, as the fuel-cell assembly (FCA) weight began to increase as the design progressed, Maynard asked me to revisit the FCA-versus-batteries
tradeoff study. While FCA weight was increasing, advances in non-rechargeable battery technology showed improved kilowatt-hours per pound and reduced weight.

Grumman and NASA jointly worked the tradeoff studies for more than six months, looking for ways to make the simpler battery system more competitive with respect to weight. We were able to reduce the total LM power requirements for the mission by minimizing the LM’s active equipment during the translunar coast phase and eliminating the need for the LM EPS to recharge the batteries in the crew’s backpacks. We simplified the system further by determining that the batteries in the descent stage could be passively cooled and by repackaging from five descent-stage batteries to four. Comparative reliability analyses confirmed our guess that batteries provided a substantial relative improvement in reliability.