Moon Lander: How We Developed the Apollo Lunar Module (26 page)

Combustion instability can occur because the rate of energy release in the combustion in the chamber is extremely high, and many physical and chemical variables interact there, each capable of influencing the others. Combustion chemistry, chamber pressure and temperature, injector flow patterns, propellant flow rates, chemical and acoustic energy can all perturb one another in the chamber’s roaring inferno. Geometrical tolerances in the injector, bubbles in the propellant, variations in propellant supply pressure, and acoustic and thermal shocks at startup can further act to initiate or sustain instability. Acoustic pressure pulses bouncing back from the chamber walls can bend some of the liquid streaming from the injector orifices, causing more or less rapid combustion and generating new waves of pressure. Instability results in large amplitude oscillations in chamber pressure and heat transfer into the chamber and nozzle walls, which can increase uncontrollably until the engine explodes or ruptures. Typically instability does not occur every time a rocket engine is fired—it’s a statistical phenomenon, with instability failures occurring on average once every X number of starts.

Combustion instability has been a problem in rocketry since von Braun and his people encountered it at Peenemünde during World War II. It was usually fixed by making ad hoc changes to the injector and the combustion-chamber geometry, changes arrived at by “cut and try” on each new engine design, an approach that did not provide any reliable rules for the next design. When it was encountered on the mighty F-1 engine for the Saturn S-1C first stage, which stood fifteen feet tall and produced 1.5 million pounds of thrust, it received unprecedented attention. An F-1 engine destroyed itself due to combustion instability in a test stand at Edwards Air Force Base on 28 June 1962, setting off a feverish round of design changes and tests led by Jerry Thomson of NASA Marshall and Paul Castenholtz of Rocketdyne, the F-1’s builder.

At first the effort was frustratingly slow, partly because the random nature of the instability did not produce the problem often enough or in any predictable manner. When it did occur, the instability destroyed a huge F-1 engine—an expensive way to learn you still have the problem. After losing two
more engines in early 1963, Thomson and Castenholtz devised a technique of exploding a small bomb (like a blasting cap) inside the chamber of a firing engine and observing how quickly the pressure oscillations triggered by the bomb damped out. Arbitrarily they decided the engine would be considered stable if the oscillations damped out within 400 milliseconds, that is, .4 seconds. The bomb test technique allowed the engineers, not nature and statistics, to initiate instability on the test stand. They were ready to shut the engine down quickly if the oscillations diverged rather than damping out, thus avoiding the loss of an engine in their tests.

The heroic efforts of Thomson and Castenholtz in slaying the dragon of instability on the F-1 is another story well worth reading.
⁴
They did not succeed until early 1965, when the F-1 engine was qualified after more than two and a half years of struggling with this intractable technical problem. In their wake they left the Apollo program management extremely sensitive to the gravity of rocket engine instability, and they also left a well-developed bomb test technique.

Although we and our ascent engine subcontractor, Bell Aerosystems, were aware of instability problems on the F-1 and other contemporary engines, we thought our engine’s small size and simple injector pattern might render it immune to that affliction. The initial test firings at Bell in 1963 and early 1964 went well, until NASA realized that Grumman had not imposed bomb stability test requirements on Bell. The ascent engine was derived from the successful Agena engine, designed for the air force’s unmanned spy satellite program, for which there were no bomb stability requirements. NASA said this was not good enough for a manned spacecraft engine and we somewhat shamefacedly agreed—it had been an oversight on Grumman’s part.

The first bomb stability tests in mid–1964 showed a problem—the combustion chamber pressure oscillations triggered by the bomb did not damp out. They did not diverge either, just continued at constant amplitude for the duration of the firing. The bomb did not always cause undamped oscillations, and even when it did the engine seemed to be no worse for the wear after the test. Manning Dandridge, our LM Propulsion Section head, and Dave Feld, ascent engine program manager at Bell, were puzzled by this uncommon engine behavior, and they consulted widely with combustion instability experts in NASA, the air force, and industry. No one had seen exactly this instability signature before (steady, undamped oscillations), but all agreed it was unacceptable and must be eliminated.

For the next two years Bell tried every instability cure that they, we, or NASA could think of, to no avail. Dandridge’s usual cheerful optimism wore thin; he grew increasingly anxious and frustrated. At first supremely confident that the next baffle design or injector spray pattern modification would solve the problem, he spent long hours with his engineers working up additional
variations to test. Dandridge trusted in objective analyses to solve engineering problems, yet the complexity and number of variables associated with combustion instability thwarted all efforts at mathematical modeling of the phenomenon.

There were other ascent engine development problems, primarily with welding and fabricating the injector, and with localized gouging of the ablative throat and nozzle in long duration firings. There were overpressure “spikes” during startup at altitude, when the engine exhausted onto a plate in close proximity, simulating the descent stage from which it would launch on the Moon. These restricted the options available to the designers for stopping instability. The Bell engine’s ablative nozzle was made of thermoplastic material similar to the command module’s heat shield. It resisted the extremely high temperatures (three thousand degrees Fahrenheit) inside the engine by its excellent thermal-insulation properties and by charring when overheated, leaving in place a firm, charred residual material that largely preserved the nozzle’s original shape. Irregularities in the injector spray pattern could cause local hot streaks within the chamber that produced cuts and gouges in the nozzle throat, the location within the engine where the rate of heat transfer to the walls was the greatest. Baffles on the injector face improved the engine’s stability, but usually made nozzle gouging and erosion worse, so the designers found themselves between the proverbial “rock and a hard place.”

Two failures in the fall of 1966 gave added impetus to the quest for a solution. The first spontaneous instability, without bomb detonation in the chamber, occurred during an altitude test of the engine at White Sands; this was with a flat-faced injector. Soon afterward a baffled injector configuration failed a bomb test at Bell. More engineers, design variations, and tests were added to the schedule, but results were elusive. At one point a configuration successfully passed ten bomb tests but failed the eleventh. We had set a criteria of twelve successful tests before declaring a design stable; after that, we increased the hurdle to twenty tests.

In the midst of this activity Dandridge and Bob Thompson of Grumman Propulsion were flying from La Guardia Airport to Buffalo on one of their many visits to Bell. Seated together, they discussed engine tests and bombs intently for most of the flight. As they disembarked, four burly men in dark suits confronted them, flashed FBI shields, and hustled them into an anteroom. For the next two hours the agents questioned them severely, demanding to know what was behind all this talk of bombs that the flight attendant had overheard. It took much explaining by Dandridge and Thompson, and telephone calls to Grumman security and Bob Carbee, before the FBI was convinced that their implausible story was true.

The cut-and-try fixes dragged on without conclusive results, and by mid-1967 NASA was very concerned; they saw this problem as a potential Apollo
program “show stopper.” NASA hired Rocketdyne to develop an alternative injector that could be installed into the Bell engine. As soon as Rocketdyne had some injector hardware and began to obtain test results, there was a continuous series of high level reviews and visits between Rocketdyne at Canoga Park, California, and Bell at Niagara Falls, New York. General Phillips, Apollo program director, George Low, Apollo spacecraft director, and NASA propulsion expert Guy Thibideaux were involved, as was I, Joe Gavin, and Dan-dridge and his group for Grumman. Bill Wilson of NASA at Rocketdyne, Steve Domokos, Rocketdyne ascent engine program manager, and Dave Feld of Bell maintained close communications and a cooperative attitude, transforming what could easily have been a very sticky contractual situation into close teamwork across companies and organizations.

Rocketdyne succeeded in producing a stable injector, but the engine they designed had other problems, including hard starts, rough running and manufacturing and assembly difficulties. Bell was unable to show a positive fix on their injector despite repeated tries. At one point, in a combination of jest and frustration, Dave Feld declared, “Perhaps combustion instability is an East Coast phenomenon. Let’s send our engine to Rocketdyne and have them test it out there.”

To Feld’s surprise, we quickly accepted his suggestion. The tests were repeated in the Rocketdyne facility at Santa Susanna, and the bomb-induced instability was still present. With the Bell engine in hand, Rocketdyne was then directed to install their injector and combustion chamber into it, retaining the Bell nozzle, valves, and mounting hardware. After some adjustments to the injector spray pattern to minimize throat erosion, this combination of the two companies’ engines performed very well. Bomb-induced oscillations damped out in less that four hundred milliseconds and thrust and specific impulse
⁵
were within specifications.

In May 1968 Phillips and Gavin visited Bell and Rocketdyne and reviewed the latest design options and the encouraging test data. They presented three design and contracting options to Low: (1) Bell engine and Bell injector (still not positively stable), (2) Rocketdyne injectors installed in Bell engines at Rocketdyne, or (3) Rocketdyne injectors installed in Bell engines at Bell. Low chose option two, with the entire assembly put together and furnished by Rocketdyne. By June 1968 this design had passed fifty-three bomb tests without instability, and it was completely qualified in August.
⁶
The long ordeal of ascent engine development was over and a major item was removed from the show stoppers list. Not a moment too soon, because the heavy LM flight schedule of 1969 was almost upon us. LM had already lost its berth on Apollo 8, which was to have been the first manned LM flight with the CSM in Earth orbit. (LM-3 wasn’t ready when needed to fly in late 1968.) Instead, Apollo 8 became a manned lunar orbit mission with CSM only at Christmas 1968. The original Apollo 8 mission was slipped to Apollo 9, flown in March 1969.

Stress Corrosion

The Super Weight Improvement Program resulted in the widespread use of chemical milling for structural parts on LM that rendered aluminum alloy parts more susceptible to stress corrosion, because it left a relatively rough and open-pored surface finish. Even without chem milling, certain aluminum alloys were vulnerable to this phenomenon, which in the 1960s was just being encountered and understood. Stress corrosion is intergranular corrosion occurring at the metal grain boundaries, usually visible only through a microscope looking at an etched and polished surface. A combination of steady stress and moisture or humidity is required to produce this corrosion. The stress levels can be relatively low—well below the recommended working stress of the material. Cracks and failures from stress corrosion typically do not develop until months or years after the original stress is applied—and this became a hidden time bomb for the LM structure, surfacing widely in the schedule-critical years of 1967 and 1968 and finding its way to the show stoppers list.

The most common applied stress causing this problem was “fit-up” stress, which resulted when parts did not fit or nest together exactly. When the fasteners were tightened, the parts were deflected until they made complete contact and the stress resulting from the deflection was locked into the material. Some joints, particularly the rod ends inserted into LM structural tubes, required a press fit to provide a solid, immovable connection. In such cases a predetermined stress level was purposely locked in upon assembly.

Minimizing fit-up stress required parts that fit together well or were carefully shimmed upon assembly to eliminate deflections when the fasteners were tightened. Training of assembly mechanics was revised to emphasize the importance of proper parts fit up and techniques for accomplishing it. We also reviewed the engineering tolerances on parts assemblies, particularly press fit joints.

Some stress corrosion of thin tabs was noted on LTA-1 as early as 1964, and a stress corrosion inspection and survey was performed on all LMs then under construction. The problem continued at a low level, with an occasional crack found, until a rash of cracked parts was discovered in mid-1967, beginning with LM-1. The cracks were mostly in the press fit ends of structural tubes, although some thin tabs were found cracked also. Mueller was furious. At that late date the Apollo schedule was threatened by this insidious problem—it could exist on almost any part, anywhere within the LM. The day after an inspection, a new crack could develop, because the nature of the phenomenon was chronic and progressive. LM stress corrosion was branded a show stopper.

Heavily pressured from above, I went to general quarters. Led by Bob Carbee, our best structural design and materials troubleshooters, Len Paulsrud,
Will Bischoff, and Frank Drum assembled a team from Engineering, Manufacturing, and Quality Control to inspect all the accessible structure on the LMs under construction. Visual inspection was conducted with flashlights and magnifying glasses. Suspect areas were brushed with Zy-glo dye penetrant, which glowed under ultraviolet light and enhanced the visibility of tiny cracks.