Saturday, September 19, 2020

Steve Bales' crib sheet that saved the lunar landing

        You have to be a real space nut (like me) to fully appreciate the historic significance of this single sheet of paper, posted recently in a Space History FB group:

         It's the list of alarm conditions that could be generated by the Apollo Lunar Module computer. It was provided by Jack Garman, a 24-year-old backroom computer guru in Houston during the Apollo 11 mission. Guidance Officer (GUIDO) Steve Bales had it on his desk during the landing, and it was this that gave him confidence to make the call that the 1201 and 1202 executive overload program alarms were not serious enough to abort the landing. Bales received special commendation by President Nixon as he presented a Group Achievment Award to the whole Mission Management team, but perhaps it was Garman who should have had the recognition — or rather, should ALSO have had recognition. After all, it was Bales on the hot seat as Neil Armstrong radioed "Give us a reading on the 1202 program alarm".

Rendezvous radar
        Naturally, there was a great deal of post-mortem analysis of that hairy but ultimately triumphant situation, and a story became current that the computer was overloaded because LM pilot Buzz Aldrin had accidentally turned on the rendezvous radar. So the computer was overloaded because the RVR was uselessly pinging a non-existent target. I plead guilty to having spread that false story myself in something I wrote in 1979, for the tenth anniversary.

        The fact is that the RVR was on but NOT by accident. It was standard and correct procedure. The thought was that it was a good idea to have it functioning and warmed up, one less thing to worry about in case of a real abort.

        The true cause of the overload was much trickier, and more difficult to diagnose. I'll quote from an article in Ars Technica by Lee Hutchinson, dateline 5th July 2019:

"The LM’s rendezvous radar contained a collection of electronics called the Attitude, Translation, and Control Assembly, or ATCA. The ATCA was responsible for providing an electrical interface whereby the LM’s guidance computer could control the radar’s hardware, and the ATCA was powered by 800Hz, 28-volt alternating current. The guidance computer in turn used a piece of equipment called a Coupling Data Unit, or CDU, to read the orientation of the radar’s antenna (its shaft and trunnion angles) so that the guidance computer could keep track of where the radar was pointed. The CDUs—there were actually two of them—were also powered by a separate 800Hz, 28-volt AC reference signal."

        As Hutchinson goes on to explain, the two separate AC reference sources were not constrained to be in phase with each other, as they really needed to be for the computer to make sense of the data. Their phase relationship depended on the precise instant at which the LM pilot turned on the RVR and set the mode switch to SLEW. 

Rendezvous radar controls in the LM

        Most of the time, during simulations, the two reference sources were sufficiently close to phase-lock that no problems arose. As ill luck would have it, Aldrin set that mode switch at the exact moment when the ATCA and CDU signals were 90° out of phase—the worst possible condition. The  CDU interpreted the data it was getting as far out of the normal range, and it issued a series of interrupts to the guidance computer, consuming 15% of the computer's time. 

         Jack Garman didn't know any of this as he wrote the crib-sheet for Steve Bales. He just knew that the 1201 and 1202 alarms would simply force the computer to re-prioritize its task list, and as long as those alarms didn't keep on coming faster and faster, all would be OK. And by the way, Jack did get recognition.

Garman receiving an award from Chris Kraft, Deputy Director of MSFC

        Even more technical detail on the alarms can be read in Tales from the Lunar Module Guidance Computer by Don Eyles, an M.I.T. programmer who designed most of the system.


Trevor Gale said...

I am surprised that even in 1969 the *power supply* phase was so important, indeed even the voltage regulation wouldn't be of great importance on the primary side (generation side). I'd have expected to see such supply noise (phase noise / voltage deviations) rejected by regulator / reference circuits in front of the electronics concerned, especially where various sensors are concerned.
I know that space was tight and that heat dissipation ws also a severe issue, both of which would have an impact on the amount of power regulation electronics but I am nevertheless surprised: I'd like to know if there's any more detail concerning these aspects of the power supply sensitivity.

Trevor Gale said...

With reference to my earlier comment: I stand corrected. I had wrongly assumed that the phrases "the ATCA was powered by 800Hz, 28-volt alternating current" and a similar for the CDU referred to a power *supply*. I was wrong, as these two units received a *reference* signal which had been defined accurately regarding its frequency (800Hz) but for these two subsystems to work together the required that the two reference signals also had to be *in phase* - more or less. Naturally, the switching on of any oscillator or like signal source will not define any phase relationship whatsoever with another signal on-board; for this to be the case there must be locking circuitry between the two (or more) signals such that in a short time they synchronize with each other. The design specification, in the form of the interface control document, called for *frequency locking* of the two signals but did not call for *phase* locking. Therefore both signals could run at exactly 800Hz but at any number of degrees of phase difference between themselves. Looking at the block diagram in the arstechnica document (see it is clear that there's no phase-lock ciruitry present and that both the CDU and ATCA need to be in synchronisation in order to gather data correctly. My apologies for any confusion which I may have caused.