Background
As the Space Task Group (STG) began to relocate to Houston, Texas, from Lan-gley Field, Virginia, in 1962, the last of the Mercury flights were still being supported from the Mercury Control Center (MCC) at the Cape Canaveral Air Force Station in Florida. In those early days, dramatic change was underway throughout the manned space program. Galvanized by President Kennedy’s Moon challenge in 1961, Gemini and Apollo began to take form in the engineering, design, and manufacturing organizations throughout the country. The learning curves were very steep in all of the disciplines. This was equally true in the control of flight operations, as they were facilitated in the first control center in Florida and at key remote sites around the globe.
The flight operations function (becoming known as flight control) faced new changes for the upcoming programs—missions were much more complex due to rendezvous, extra vehicular activity (EVA), and lunar challenges. Telemetry and command systems were changing from analog to digital, reliable global communications were enabling a more centralized control center with fewer remote sites, and the schedule demanded extensive training with concurrent training for one flight while actively performing another simultaneously. All of these considerations and others led to the conclusion that the new Mission Control Center must be colocated with the flight control team in Houston.
Beginnings—Operating Concept
Discussion of flight control in the MCC begins with an operating concept of the way missions will be planned and managed by the team. Flight control was, and still is, part of the overall planning effort. In general, the program office defined the objectives and selected details for each of the flights. The definition by the program office evolved over the years from rather simple statements of objectives to more formal documents, outlining objectives, flight content, and some of the sequencing of events. The flight control organization has always seen itself as responding to these program office requirements, while participating in their development. In addition to the flight control element, another analytical organization has always existed within the overall operations organization. In general, its role was the mission planning task, again in response to program objectives. It developed the software analysis tools, the options, and the recommendations in areas such as launch phase design, abort mode definition, rendezvous and entry techniques, lunar trajectory simulations, lunar navigation methods, use of propulsion systems and guidance for lunar landing, and some consumables analysis. To this overall process, the flight control team brought a detailed knowledge of how to use all of the spacecraft systems to accomplish these objectives and techniques and how to monitor the basic health of the spacecraft systems as they are performing the assigned tasks. Another part of the flight control team, specialists also in trajectory, guidance, and abort planning, took the techniques defined by the planning organization and determined how to display, monitor, and control all of the necessary real-time steps in performing these techniques.
Early in Mercury, the role and authority of the flight control team in the MCC was established and integrated with the program office, the flight crew, and the vehicle engineering organizations. That is stated rather simply now but was a major learning experience at the time for all participants. Resolution was greatly enhanced by the early leaders of the STG, some of whom had considerable aircraft flight-test experience and reputations.
During Mercury, there were also arrangements with STG and other engineering organizations to participate as flight controllers. This worked better in some cases than others, but the eventual demands of time and travel for simulations, launch site test support, and real-time flight control led to the decision to use dedicated personnel assigned full time to the flight control organization.
The flight control team in Mercury had another component that gradually reduced in size to zero as the 1960s progressed. Figure 1 is a useful reference to explain how this component came to exist.
The figure displays the ground tracks resulting from consecutive spacecraft orbits around the globe. The figure also portrays the telemetry coverage of the spacecraft available from various locations which were referred to as remote sites. These sites were in contact with the spacecraft for up to 5 minutes but realtime data was not quickly or reliably routed back to the MCC in Florida. Crew voice was generally relayed to the MCC in real time, but those communication links were also not as reliable as desired. Therefore, small flight control teams of three people were sent on station at key remote sites. They conducted air-to-ground communications with the flight crew and evaluated spacecraft systems in real time with their local displays—much like a short-term, surrogate control center. They also sent teletype snapshots of spacecraft systems data back to MCC after the spacecraft passed over their site. This worked well enough but required a significant number of people. The coordination between MCC and the multiple remote site teams also created the necessity to keep many people well informed and synchronized in mission awareness.
Figure 1. A reconstruction of the Mercury Control Center at Kennedy Space Center shows the world map with the ground tracks and station coverage.
This component (active flight control teams at remote sites) was carried over into Gemini, the successor flight program to Mercury. But as global communications improved, the number of teams deployed was gradually reduced. There was an average of 13 remote flight control teams for Mercury. Seven teams with six flight controllers were still deployed early in Gemini but gradually phased out after the unmanned LM-1 flight in early 1968. Even as it phased out, the experience gained by so many young engineers, two to three years out of college, by leading one of these teams was heady stuff and accelerated their learning and growth dramatically.
The guiding principle of the Mission Control Center (MCC), flight control team was, and is today, straightforward. (Note: After the Mercury missions were completed, the term MCC was changed to mean Mission Control Center.) The team was responsible for making any decision or taking any action to conduct the mission safely and successfully. This responsibility carried the requirement to make any necessary decisions within seconds. This criterion and the temperament to accept this responsibility became the driver for selecting people. The required understanding of spacecraft systems and trajectory techniques, as well as understanding how to use all of these in varying mission circumstances was also very important. This real-time response requirement drove the need for training and simulations and the need to establish documented mission rules and procedures by which to guide real-time decision making. Spacecraft systems and trajectory/guidance disciplines were the core skills of this team. Systems team members developed and published the spacecraft system handbooks which were functional schematics of the spacecraft systems, including the placement of telemetry measurements. Not only was this a powerful learning tool, it also became the reference for all of the operations team; they were also used extensively by the engineering and industry teams. Spurred by the false heat shield deploy telemetry indication on John Glenn’s MA-6 flight, the systems controllers performed a very valuable program service in critically assessing and improving spacecraft telemetry instrumentation on all future spacecraft.
Another element of the operating concept was the focus on training. This was and is a phased process, starting in the classroom and proceeding through individual systems training in special facilities and interactive computer-based training. The final step in flight preparation involved closed-loop simulations with the entire flight control team in MCC and the flight crews in simulators. This final step tested and verified all of the planning and mission rules for each of the major phases of the mission. Later on, members of the flight control team also observed geology training for the crews, EVA training in the large water tank, and even flew jump seat in the Shuttle training aircraft as an adjunct to their training.
All of these and other considerations led to the definition of the flight control team structure. The leader and final decision maker was the Flight Director, and the rest of the team reported to this position. An astronaut was always assigned as the voice interface between the flight crew and the MCC. The designation was Cap Com from the original Mercury terminology—Capsule Communicator—and represented the crew’s point of view to the MCC team. The rest of the team was populated by specialists who had console call signs such as
*GNC—control and propulsion systems (command service module) *EECOM—environmental and electrical (command service module) (com was reassigned) *INCO—spacecraft/ground communications *CONTROL—control and propulsion systems (lunar module) *TELMU—environmental and electrical (lunar module) *AFD—assistant to the flight director *PROCEDURES—coordination with remote sites *FIDO—trajectory monitoring and control *GUIDO—onboard guidance systems *RETRO — abort and return to earth planning *BOOSTER—launch vehicle systems *FAO—flight planning and crew checklist *SURGEON—flight surgeon *NETWORK—interface with the network *M & O—interface with MCC systems
Each one of these specialists was supported by additional team members located in staff support rooms (SSRs), often called “back rooms.” The number of SSR participants varied for each discipline and by the phase of the mission. This feature provided in-depth support to the Mission Operations Control Room (MOCR) console operators and an extra resource to pursue resolutions of multiple problems as they occurred. The full MOCR teams numbered about 16 per shift. SSR support averaged about 106 personnel on each shift. Besides this core team, there were several other consoles for management, public affairs, and other involved agencies. The flight control team also had the responsibility to define all of its requirements for data displays, commands, and voice loops for each of the respective positions in MCC, preparing them to use these capabilities.
During the preparation time leading up to a flight, the flight control team engaged the program office to ensure understanding of objectives and to influence specific details. Very importantly, controllers were also active participants in the planning process, helping to define the specifics of the mission, flight plan, checklists, and their own procedures. The Flight Director and his flight control team also updated the mission rules for the unique aspects of a particular mission after detailed consultation with the flight crew.
To illustrate the relationships between the various documents, the mission plan was the source material for trajectory design, usually in the form of multiple reports. The flight plan was a detailed time line of flight activities prepared for the flight crews to use but monitored and followed by all parties. Mission rules governed the criteria for deviating from the nominal plan and selecting alternative courses of action up to and including early mission termination. Checklists and procedures were also referenced in the flight plan and governed detailed switch by switch scenarios for standard activities such as guidance platform alignments and EVA suit donning. Spacecraft malfunction procedures were developed by astronauts and flight controllers. Procedures were also documented for each console operator, outlining their detailed actions for using the capabilities at their consoles. Systems handbooks were references for troubleshooting.
The flight control team defined the interface with the launch site team, the range safety office, and any other mission support functions. These other functions might include payload centers operating within the MCC or at remote locations such as Goddard Space Flight Center, (GSFC), Jet Propulsion Laboratory (JPL), Marshall Space Flight Center (MSFC), and the Air Force Space Flight Control Center at Sunnyvale, California (only for the Shuttle). This interface definition included requirements for voice and data transfers between these locations and the procedures governing any transactions. Procedures also included hand-over definitions such as the transfer of control from the launch site to the MCC at clearance of the launch tower. The flight control team then conducted simulations with these other organizations to prepare for the flight. At the end of this process, the teams performed their flight support roles during the mission.
Although the type and number of experts varied with time or specific mission requirements, this operating concept endures until today. It has served the programs and NASA very well during the past decades and will continue to do so.
The Control Center is the instrument by which this team performs its task. Based on the Mercury experience and the vision of what Gemini and Apollo would require, the Mission Control Center (MCC) in Houston began to be defined starting in 1962. This definition recognized that the MCC would evolve in equipment technology and the scope of participation. The MCC was located in Bldg. 30 at the Manned Spacecraft Center (MSC, later renamed as the Johnson Space Center—JSC) complex. Building 30 included an office wing and the Mission Control Center element. The MCC wing was a three-tier facility of 118,500 square feet; the computing and communications equipment are on the first floor, topped by two floors of very similar design. Each has a central Mission Operations Control Room (MOCR), later called Flight Control Room (FCR) for the Shuttle, a viewing room behind the MOCR, and multiple staff support rooms (SSR—later called MPSR’s, multipurpose support rooms for the Shuttle) which provided in-depth support to the flight controllers in the MOCR. Additional space on the operational floors was assigned to the simulation team, the recovery team, and other support functions. Figure 2 is an overview of the MCC layout.
Figure 3 is a layout of the MOCR that has consoles for all of the discipline specialists in the room. The MCC in Houston was implemented under a 1962 contract with Philco-Ford Western Development Laboratories (later moved to Houston as Philco Houston Operations—PHO) for the design, development, and maintenance operations of the overall facility. NASA also contracted with IBM in 1962 for the Real-Time Computer Complex (RTCC). These contracts were modified, extended, and/or recompeted until a major change occurred in 1986 (to be discussed later).
As the concept of MCC as the central hub of ground based flight control continued to evolve for Apollo, floor space and equipment was dedicated to representatives of the spacecraft program office who orchestrated support to each mission from JSC engineering and vehicle contractors. The role for flight control remained the same. All decisions and actions were taken by this team especially for near real-time decisions, that is, within seconds or minutes. When an unanticipated technical condition occurred and “time-to-decide” was measured in hours or days, there was an opportunity to consult the program office/engineering/industry team for help in the resolution. This arrangement was not very strongly implemented for Gemini, perhaps because the program office did not have a test and evaluation organization as in Apollo and also because much of the JSC engineering was focused on Apollo and not Gemini. For Apollo, this concept greatly added to the strength and problem-solving capabilities of the MCC team. The spacecraft program office support function (including flight control and spacecraft industry representatives) came to be known as SPAN (Spacecraft Analysis room). This room was tied into other facilities outside the MCC that housed JSC and industry engineering teams. These teams followed the missions with telemetry and voice available from all of the key communication loops. This interface (SPAN) tied the rest of the program team into the MCC in a very controlled and orderly way. The arrangement ensured high confidence in dealing with anomalous technical conditions and a unified, well-understood methodology for engaging the program office team of JSC engineering and vehicle contractors.
Additionally the MCC provided a facility for the interface with the payload customers, principal investigators, and scientific teams that sponsored individual payloads. Again, this methodology ensured the highest confidence in dealing with any deviations to the planned payload time line or use of that equipment.
Figure 2. Building 30 MCC at the Johnson Space Center has three floors; the upper two floors are nearly identical.
Figure 3. The MOCR (FCR) layout at the Johnson Space Center included the Flight Control Team in the front three rows and a back row for management and administration.
Figure 4 is an illustration of MCC as the central hub, tying in various organizations for decision making on ongoing flight activity.
Gemini Era—1962-1966 (1)
In the Gemini program, the mission planners, the flight control teams in MCC, and the flight crews established the operational experience base and confidence for Apollo. Gemini was conceived and conducted to test as many of the Apollo mission techniques as possible in Earth orbit. As the reliability of global communications grew, the Gemini operations began to reduce the number of remote-site, flight control teams and centralize decision making in the new MCC. Gemini was a fast moving program that had multiple technical challenges. Ten manned Gemini flights were planned and flown in a 20-month period. From the first short-duration flight of Gemini 3, Gemini 4 accomplished an EVA and experienced the first difficulty in attempting to fly formation, using the last stage of the Titan launch vehicle. A long duration was the emphasis of Gemini 5 for almost 8 days. After an Agena failure, two Gemini spacecraft on 7/6 completed the first rendezvous and station keeping and achieved a nearly 14-day flight by the Gemini 7 crew. Gemini 8 docked with an Agena stage, but a short in the Gemini control system led to an emergency undocking and early mission termination in the Pacific contingency recovery area. Gemini 9-12 experimented with multiple types of rendezvous, docked, and tethered Agena maneuvers. These final flights also concluded the learning process for orbital EVAs. Once adequate crew positioning aids (hand- and footholds) and water tank training were made available in Gemini 12 spacecraft, the crews had a much more controlled way to perform assigned EVA tasks. With all of these new challenges, many missions also experienced spacecraft problems especially with the fuel cells and control systems thrusters. All of this provided even more experience in real-time problem solving, while continuing to satisfy mission objectives.
Figure 4. The MCC Flight Control Team is the hub of all ground support functions during missions.
The software capability to handle these new operational challenges was embedded in the MCC systems (2). These systems can be technically described around the three major functions of MCC architecture: the Communications Interface System (CIS), the Real Time Computer Complex (RTCC), and the Display and Control System (DCS). The CIS provides internal communications and connects the MCC to the spacecraft communications links via the network and remote sites. The RTCC provides the mainframe computational and display processing, and the DCS provides the information to the human operators.
The communications system elements consisted of a UNIVAC 490 (the communications processor), a Lockheed PCM-102 ground station, and some custom hardware, called the Master Digital Command System (MDCS). The Gemini launch data systems collected data on 2.0-kbps lines from the Eastern Test Range (ETR), as well as on 40.8-kbps lines where the ground station did the frame-sync and decommutation functions. Each console had an intercom panel for the major loops such as air to ground, flight director, and launch side coordination and a set of special loops unique to each console.
The mainframe processors consisted of five IBM 7094s, using a customized IBM operating system within the RTCC. The machine which was designated as the mission operations computer (MOC) software would process the data from the communications processor and perform the telemetry, trajectory, and command functions. A concept of a dynamic standby computer (DSC) was established. The DSC processed the same inputs with the same software, but the results were not used. The computer operators would compare the results from the two machines to ensure that they were synchronized. Should the MOC fail, the DSC would be brought on line within seconds to become a new MOC.
The DCS element that created video displays and presented them to the operator was the digital to television (DTV) subsystem. The MOC data was shipped to the DTV subsystem across a channel buffer. The DTV subsystem then used the MOC data to produce 48 channels of display. The background format consisted of fixed display formats which were determined premission and did not change with telemetry processing. The foreground data elements were the real data resulting from real-time telemetry. This system provided numerical outputs on black and white CRTs with essentially no graphics. Figure 5 is a typical Gemini era console.
The projection plotter display equipment was the subsystem that produced the large screen display of the spacecraft position and ground track on the world map, as well as other launch and landing graphics (2). Other MCC-unique systems were developed in this time frame, including the video hard copy system. If an operator needed a printout of his video display he pressed a hard copy request button which activated a camera that took a 35-mm film image of the same video channel that the operator was viewing. The film was automatically processed, printed on paper, and dried in fifteen seconds. The hard copy equipment operator then placed the print into the pneumatic tube (P tube) system, and the canister delivered the processed print to the console operator (2).
In the original Mercury Control Center, the computing center was managed by a team from the NASA-Langley Research Center destined to join what became the Goddard Space Flight Center (GSFC) in Greenbelt, Maryland. Early in the software development, the computing facility was actually housed in an IBM building on Pennsylvania Avenue in Washington, D.C., but the actual flight support to the Mercury Control Center was in a computer complex at GSFC. Its role was limited to launch tracking, orbit determination (navigation), retrofire,and landing calculations because it did not process telemetry. Telemetry and command functions were processed through separate equipment in the Mercury Control Center in Florida and the remote sites. Because this computing function was so vital, new, and so geographically separated, a backup capability for ret-rofire calculations and other trajectory support was established in a facility called the Auxiliary Computing Room (ACR). The ACR employed the same programs used for premission planning and the institutional computers at STG in Virginia. This backup capability was carried over into the new MCC in Houston using institutional mainframe machines in the JSC complex. During Gemini, additional premission planning programs in the ACR were used to back up rendezvous maneuver calculations and entry planning. This capability was eventually phased out after Apollo 12.
Figure 5. A typical console for Gemini and Apollo featured two CRT displays, intercom panel, event and status lights, and command switches.
In Gemini, the simulation task became much more sophisticated than in Mercury. Mercury basically used an open-loop simulation system, whereas Gemini had to be closed loop between MCC and the Gemini simulator because of more variations in potential actions by the crew or the MCC. The Gemini simulator became the major training platform for the flight crews. The simulations were mechanized and conducted in closed-loop fashion, so that all actions at either the simulator or the MCC were correctly reflected in the ongoing simulation. A simulation control area was located on the second floor of the MCC, where the simulation team managed the complex process of establishing realistic and challenging exercises. During this period, some flight control teams were still sent to selected remote sites before adequate data communications were available. The simulations area also had a replicate of the remote-site control rooms for training the remote-site teams.
In March 1965, the MCC-H monitored the Gemini 3 mission, and primary control responsibility rested with the old Mercury Control Center at Cape Canaveral. Although not all systems were completely tested and operational, the MCC-H came on line in such a manner as to convince NASA that it was ready to control Gemini 4. In June 1965, NASA manned the new MCC in Houston to control the Gemini 4 mission with MCC in Florida as backup. In the flight of Gemini 4 that included the first space walk by an American astronaut, the MCC in Houston was declared operational and the old control center at Cape Canaveral was deactivated. The H was dropped from the name and Houston became the control center for all subsequent manned spaceflights (2).
The Gemini experience base in operations from the MCC provided a powerful level of confidence as the planners, flight control teams, and flight crews moved to the Apollo program.
Apollo Era—1966-1972 (1)
While Gemini was underway, unmanned Apollo flights were scheduled to test the new launch vehicle (Saturn 1B) and the Apollo command and service module (CSM). During 1966, two successful flights (AS 201 and AS 202) were conducted with high-speed entries to test the CSM heat protection system. These flights also provided an opportunity for the flight control team to understand the Apollo spacecraft. After the last Gemini flight in November 1966, Apollo was ready to move into the manned flight phase.
Then, in January 1967, tragedy struck Apollo with the fire on the launch pad. Three crew members perished. The program went into an agonizing reappraisal. Externally, some questioned whether the program should continue. But internally, the recovery process and spacecraft modifications moved into high gear. The country and Congress reaffirmed their support for Apollo. The unmanned flights resumed with the first two flights of the Saturn V and reentry of the CSM at speeds equal to lunar return velocities. The second Saturn flight produced a major scare when two engines shut down late in the second stage due to a center engine pogo condition and a miswiring of the shutdown circuitry. But it did not significantly affect the mission. An unmanned lunar module (LM) was also successfully tested in low Earth orbit after it was launched on a Saturn IB. These unmanned flights added specific Apollo knowledge to the Gemini experience of the flight control team.
This operating capability enabled the Apollo program to fly 10 days on the first manned orbital mission (Apollo 7) and to go to lunar orbit on the second manned Apollo mission (Apollo 8). Two more major missions, Apollo 9 and Apollo 10, tested the CSM and LM in Earth orbit and then in lunar orbit. Apollo 11, the fifth manned mission in the program, landed on the moon and returned safely. During the powered descent of the lunar module (LM), the flight control team evaluated computer program alarms, and correctly assessed them as acceptable to proceed. This schedule represented a very aggressive sequence of missions through Apollo 11, which would have been unlikely without the Gemini operating foundation. Apollo 12 was struck by lightning soon after liftoff and still reached a nominal Earth orbit, but the CSM guidance platform was lost; the CSM guidance was restored in orbit. MCC evaluation of the CSM and the Saturn IV B permitted a “GO” for translunar injection to the Moon. With the aid of MCC lunar navigation, Apollo 12 was vectored so that the onboard LM systems and the crew guided the landing to the surveyor spacecraft, which had landed on the Moon years earlier. Subsequent precision landings allowed specific planning and execution of scientific exploration of the lunar surface on five landings after Apollo 11. The successful Apollo 13 return was the ultimate test and demonstration of capability by the flight control team, ably assisted by the engineering/ industry teams around the country, for problem resolution with a long ”time to decide.” All of the CSM power-up and entry procedures were tested in the simulators by astronauts. This success was the result of a decade-long improvement in the flight control team in solving complicated problems in real time and the on-call engineering support provided by the program office team. Apollo 14 was the recovery flight after the Apollo 13 mission. Apollos 15, 16, and 17 extended lunar surface exploration with longer lunar stays, more EVAs, and the lunar rover for surface mobility. The CSM also had a service module bay full of scientific instruments for mapping the Moon and its environment while in lunar orbit. Apollo was also the first time the flight control team enjoyed the luxury of full-time coverage from the Deep Space Network (DSN) once the vehicle left low Earth orbit and while it remained on the front side of the Moon.
Any discussion of the MCC system would be incomplete without recognition of the management processes involved. Because the mainframe computers used in the MCC were always fully taken up with the telemetry, command, and trajectory planning requirements, there was very high priority in establishing a highly reliable MCC system. This introduced very real tension between the flight controller’s desire for new and improved capabilities (requirements) and the implementer’s desire to develop and freeze a system (implementation) which would be stable and reliable for each flight. This tension resulted in major and painful ”scrubs” of requirements to fit within the system capacity and the program schedule. In spite of the pain, the process worked well due to the guidance and dedication of the flight organization management team. Their skills generally resulted in the most capability available for the flight control team at any given time, and maintained rigorous control of the implementation process and the delivery of a reliable system.
To elaborate further on the software implementation, it was not an art, as some considered software development at that time, but a well-practiced discipline. The management process consisted of three basic elements: planning, execution, and feedback. The planning phase for the large software system development consisted of the following steps: (1) define the goals and objectives, assign responsibilities, and organize; (2) define the system requirements; (3) design the system; and (4) estimate the required resources. The execution phase was divided into the following steps: (1) implementation or programming, (2) testing, (3) verification, (4) operation, and (5) maintenance. The feedback phase consisted of reviewing progress, incorporating changes, and updating plans. These three phases were defined somewhat arbitrarily for presentation and, in most cases, were not sequential. However, all of the steps did exist and had to be reckoned with, especially because the software had to accommodate changes reasonably (3).
There were no cases where MCC did not have adequate capability for each flight or where the MCC was not ready to support. Sometimes, the desired stability was reached uncomfortably close to the scheduled time for the system’s use. But in the end, MCC was always there and ready. This performance was a major tribute to the managers of the both requirements and implementation processes.
During the Apollo project, the CIS communication processors were upgraded from Univac 490s to 494s. The extra processing power allowed the 494s to perform some of the PCM ground station functions and all of the previous MDCS functions. The new merged system was called the Communications Command and Telemetry Subsystem (CCATS). The CCATS in the MCC provided the capability of simultaneously performing the functions of digital communication, data handling, telemetry data, decommutation and distribution, and verification and control.
Similarly, the RTCC processing systems outgrew the IBM 7094s, and they were replaced by five 360/75s. Some of these were the first units off the IBM production line. The concept of using a dynamic standby computer which shadowed every operation of the MOC was carried over from the Gemini design. The operating system of the 360s was modified to support the performance needs of the MCC, and the new operating system was named The Real Time Operating System (RTOS), later upgraded to the Extended Operating System (EOS) (2).
The increasing performance needs were primarily due to the increased complexity of the trajectory functions of a lunar mission, but there were also new changes in the type of data which was being processed. There was an increased emphasis on flight planning aids and decision-making support.
Looking back on those days, it is very interesting to note that the MCC supported the lunar landing missions with a total computer capacity comparable to one modern work station and the new MCC has hundreds of these work stations.
As part of the continuing provision for payload support, a data system was developed in the MCC to support the Apollo Lunar Surface Experiment Package (ALSEP); the first of these was deployed on Apollo 12 (2). A simpler, solar-powered version was deployed on Apollo 11. These packages had a complex of scientific instruments, including seismic detectors, and were designed to remain on the surface and take measurements for years. An operational control room was implemented on the MCC second floor using standard consoles and displays for this package. The ALSEP data system was deactivated in September 1977 after approximately 8 years of continuous and successful support of the five operational ALSEP packages left on the Moon on each landing mission (2).
Skylab, Apollo-Soyuz Test Program (ASTP)—1973-1975 (1)
Skylab, the first U.S. space station, was designed to conduct solar, Earth resource, biomedical, and a subset of corollary experiments. It was also the platform for long-duration in-orbit experience for U.S. astronauts. The initial unmanned launch of the Skylab station in May 1973 resulted in the loss of one of two solar wings and the meteorite shielding during the launch phase. The vehicle came close to becoming a complete failure once it reached orbit because of the increasing temperature within the living module caused by the loss of the shielding. These conditions were repaired by the first Skylab crew, using a manually deployed thermal shield (like a parasol) through one of the Skylab’s airlocks from inside the module. An EVA was used to restore full power from the remaining solar wing. On the second flight, the crews replaced the initial thermal shield with a more permanent design, and the third manned mission culminated in an 84-day record flight. In-orbit operations were a real test of the flight crews, the flight control team, the scientific experiment teams, and the MCC itself. The self-imposed demands for maximum return on the time spent in Skylab caused some amount of work overload on the crew and on the MCC operators and occasionally resulted in frustration.
In July 1975, the first joint American/Soviet Earth orbital mission was conducted. In a test of rendezvous and docking compatibility, the ability of the two major space-faring nations to cooperate with each other was also tested. This experience gave the country a foundation on which later decisions were made in the 1990s to cooperate with Russia on the International Space Station. ASTP also had the communication advantage of using one high-altitude NASA communication satellite that provided coverage across half the globe.
In preparation for these missions, one of the major changes in the MCC systems was the replacement of the D/TV system with digital television equipment (DTE). A prototype DTE system was installed in July 1970. The operational DTE system supported Skylab, ASTP, and later Space Shuttle missions. Besides the DCS changes, other major modifications included implementing a data compression system on the network and the necessary processing changes in the
MCC to handle this compressed telemetry data. A major data storage subsystem was also added, called the Mass Data Storage Facility (MSDF). The MSDF consisted of two Control Data Corporations (CDC) computers, a Cyber 73 and a Cyber 74 and associated peripherals. The MCC also provided some scientific data processing in the form of an Earth Resources Preprocessing Production System (PPS). The system accepted raw Earth resource data obtained from both aircraft and Skylab before use for data application purposes. The system accepted various data formats and produced preprocessed data for distribution to Principal Investigators (PI) where final processing was performed. Skylab postmission processing continued in the MCC for years after the last Skylab mission in special scientific data systems like this one (2).
Space Shuttle—1977-1986 (1)
The Space Shuttle was the next major program in human spaceflight. National policy dictated the Space Shuttle as the single launch vehicle system for all U.S. payloads and as the carrier for any international payloads which the United States would launch. By this policy, all other U.S. launch vehicles would be phased out during the 1980s. There were also strong commitments that the Shuttle cost and price to customers would be significantly reduced from current launch vehicle systems. These two conditions—service all type of payloads and achieve major cost reductions—became significant drivers to the program. The subsequent failure to achieve expected cost reductions led to many criticisms and a lack of acceptance on the part of some of the U.S. payload community. Even without these conditions, the development of the Shuttle was a very major technical challenge. Ultimately, the challenges were met, and the Shuttle is now a major element of the U.S. launch vehicle stable.
As part of that development, there was a series of five flights of the orbiter, dropped from the Shuttle carrier aircraft at Edwards Air Force Base during the second half of 1977. These were the first Shuttle program flights, and they were supported by the flight control team. Starting in 1981, the flight control team supported the next 25 Shuttle flights, including the loss of the Challenger and crew in January 1986. Until that event, major progress was made in transiting the Shuttle system into a very versatile payload carrier and mission platform. Highlights included four orbiter flight-test missions, the launch of numerous commercial communication satellites, the first American woman in space, Space Lab scientific missions, an untethered spacewalk with a manned maneuvering unit, a U.S. senator and congressman as passengers, several international passengers, retrieval and repair (or recovery) of satellites on three separate missions, two classified DOD missions, and numerous technical advances within the Shuttle system itself. However, this trend changed abruptly with the Challenger accident in January 1986. U.S. policy was changed to restrict the use of the Shuttle to NASA and certain selected missions. The commercial community of communications satellites was not permitted to fly on the Shuttle. The DOD chose to return to expendable launch vehicles, although there were several DOD flights on the Shuttle in the early 1990s. The Shuttle payload traffic then contracted to essentially NASA and international scientific missions.
MCC supported all of these Shuttle missions. A special system was developed for the 1977 approach and landing test (ALT) flight while the total Shuttle system was still in development. The ALT system consisted of a 9.6-kbps line from Edwards to the MCC. Displays were available through the DTE system. An early version of the telemetry program was used as part of the software system. The primary operations support room was converted from the Apollo era recovery support room. Standard consoles along with X/Y plot boards were used for displays (2).
The communications interface subsystem added some major new equipment to support the Space Shuttle program. The Tracking and Data Relay Satellite Systems (TDRSS) network multiplexer-demultiplexer system came on line for STS 3 to set the stage for spacecraft communications in the TDRSS era. The first TDRSS satellite was launched on STS 6, and with later launches of TDRSS, became the system by which Shuttle data, voice, and video were communicated to the MCC (2).
Telemetry processing computers (TPC) were Interdata 832 computer systems and included the template for the current downlink telemetry format. They identified and transmitted the data across an interface to the MCC. Additionally, the onboard Payload Data Interleaver (PDI) placed payload data in “windows” in the telemetry stream, and the TPC routed the PDI traffic for delivery to local or remotely located payload customers. The TPC also drove selected analog and event displays and strip-chart recorders.
For the RTCC, the IBM 360/75s were replaced by four 370/168s. These computers performed many other functions in addition to the MOC and DSC functions. The machines supported engineering analysis, software development, vehicle trend analysis, MCC system reconfiguration, mission planning, and other functions. In the display and control system, some new custom design work on the DTE functions allowed expanding the capability. This expansion boosted the number of video displays on consoles from 80 to 104 channels (2).
Because the Shuttle was defined as the primary U.S. system for all payload communities, NASA provided special accommodations to serve their respective requirements. These accommodations were for DOD, scientific payloads, and commercial customers.
Early in the Shuttle program, the DOD identified the need to support classified flights in the MCC. In response to this requirement, implementation of a ”control mode” for secure operations on the third floor was begun in 1979. Certification was approved in May 1983. Secure television was established. The mainframe computers could be switched and isolated to support classified operations. Cable trays were separated into classified and unclassified. Likewise the grounding system in the MCC was modified. Physical controls were established for separation and access. Card readers were installed, and tourist traffic was rerouted only to the second floor viewing room of the MCC.
The Payload Operations Control Center (POCC) facility provided Spacelab experimenters with a highly capable data evaluation processing and planning facility. This facility was implemented during 1981-1983 to support Space Lab flights (see Fig. 6).
Primarily for the commercial satellite customers, the general purpose Customer Support Room (CSR) was essentially a minicontrol center with communications, displays, a management meeting area, and even a viewing room (see Fig. 7).
Figure 6. The Payload Operations Control Center (POCC) supported primarily the experiment teams for Space Lab missions.
Much of this service provided in the MCC for customers came to a halt or was redirected after the Challenger accident.
Figure 7. The Customer Support Room supported primarily the commercial community, especially the Communication Satellite Teams.
Challenger Recovery Through 1990 (1)
The recovery from the Challenger accident was again a painful process, reminiscent of the Apollo fire, for all elements of the manned spaceflight program. All of the questions asked after the Apollo fire were reengaged. Although there was an extraordinary number of detailed subjects, they were all aimed at the questions, How to recover and how to prevent another accident.
Slowly, the entire program team worked through their respective areas searching for better answers. Many answers became standard practice and resulted in more checks and constraints on the program. Safety again became the watchword for this assessment period and beyond. Eventually, the Shuttle returned to flight on STS 26 in September 1988. Then, a series of STS flights followed with significant achievements, and the flight control team contributed its share to these successes.
Shuttle flight highlights reflect the deployment of major payloads that were delayed by the accident but still scheduled for launch on Shuttle for technical or program reasons. STS 26 was the return to flight voyage of Shuttle and deployed the TDRSS 2 spacecraft. Five classified DOD missions were flown in this period. TDRSS 3 was also deployed, followed by three interplanetary missions—Magellan to map Venus, Galileo to explore Jupiter, and Ulysses to explore the polar regions of the Sun. Another Spacelab mission (Astro 1) was conducted, and STS 31 successfully deployed the Hubble Space Telescope (HST), but the optics problems encountered set the stage for a later repair mission.
In the MCC systems, the 370/168 mainframes were replaced by 3083JXs during 1986 and later updated in 1990 to 3083KXs, which doubled the CPU capacity (2). In 1986, the first real change of the major MCC contractors (PHO and IBM) since 1962 occurred when the Shuttle Transportation Systems Operations Contract (STSOC) including all MCC functions was awarded to Rockwell. During this period, two additional significant initiatives were pressed by the Mission Operations Directorate which was now the consolidated organization for planning, training, and controlling the missions, plus all the development and operations of the MCC and simulator complexes. The flight control team had been considering and experimenting with work stations in support of consoles. This was motivated by a desire to exploit new technologies and to reduce the lead time and inflexibility of the highly controlled mainframe software. This initiative was strengthened during this period by the addition of a contractor (Ford Aerospace) devoted primarily to new developments, which could be phased into the MCC, which was then separately sustained and operated by Rockwell. This accelerated the prototyping of new ideas in a test facility called the Transition Flight Control Room (TFCR), featuring workstations with color displays and graphics. The second major initiative was the early recognition of the facility needs of the emerging Space Station program. As a result of this assessment, a new wing called Building 30 South (30S) was approved, and construction began in 1990. It was planned to house the flight control team and other associated flight support activities for Space Station operations. This new wing was attached to the original MCC wing of Building 30, and provided the additional floor space deemed necessary for the task of Space Station flight control. Figure 8 portrays the five floor layout of this 103,400 sq.ft. facility.
Figure 8. Building 30S MCC has five floors and two control rooms.
Shuttle and Space Station Preparations—1991-1999 (1)
For mission operations, the decade of the 1990s can be characterized by continuing support of Shuttle missions mostly with NASA payloads and planning for a newly redesigned International Space Station (ISS), including the new MCC. The 1993 decision to include the Russians led to an ISS configuration, in which the Russians would provide about 10% of the mass of the in-orbit Station complex. The substantial contribution of Russian hardware created the need for the flight control team to integrate their operations with the MCC in Moscow.
Although this was not the original intent for 30S, it became clear by 1993 that all Shuttle and Space Station mission support should be located in the single new MCC in 30S, resulting in more simplifications and efficiencies.
The Shuttle program continued to launch and service individual projects. In the 9-year period from 1991 to the end of 1999, the Shuttle program completed 58 missions at an average rate of 6.5 per year. The Shuttle flight program logged many achievements. Selected highlights of this period were three more TDRSS flights, another DOD mission, and multiple scientific missions aimed at atmospheric, life sciences, and microgravity studies, including some sponsored by other countries. The Compton Gamma Ray observatory and the Chandra X-ray telescope were also launched. There were two tether flights, a reboost of the stranded Intelsat communications satellite with the real-time decision to use three crew members (instead of the usual two) on the EVA to capture Instelsat, Hubble Telescope repair and reservice missions, and a reflight of Senator John Glenn. The Shuttle also flew one rendezvous and station-keeping mission with MIR and nine docking missions to MIR. The United States also launched its first major space station element—the node—late in 1998 and a later flight serviced the initial ISS configuration.
In 1994, the concept of a number of test Shuttle missions to MIR was established. Recognizing the huge construction task which the ISS represented and the complex coordination with Russia, it was decided to use the current Russian Space Station called MIR as a test platform to gain the operational experience of conducting joint missions and the early flight of some of the payload equipment planned for the ISS. In a sense, the Shuttle/Mir project was conceived to contribute to the ISS program what Gemini contributed to Apollo, that is, an early operational foundation.
The Shuttle/Mir project was an aggressive flight sequence over three plus years from early 1995 to mid-1998. STS 63 was the first rendezvous and station-keeping test, but without docking because the provisions were not yet available. There followed nine Shuttle docking missions, rotating crew members, and bringing supplies to and returning equipment from the MIR. Seven American astronauts lived and worked on MIR for a cumulative total of approximately 908 days, and the Shuttle brought up about 20 tons of supplies and returned 9 tons to Earth. The flight sequence continued, even in the face of major and minor MIR difficulties. The major problems were an onboard fire and depressurization of one of the MIR modules. The NASA/Russia team perservered through these difficulties and gained robust experience in jointly managing and conducting the Shuttle/MIR project.
As ISS development matured, the initial deployment of ISS hardware to Earth orbit followed. The ZARYA Russian module was launched in November 1998, followed by the American node UNITY on STS 88 in December 1998. This initial configuration was serviced on a 1999 Shuttle flight. The Russian ZVEZDA service module (containing life support and propulsion systems) was long delayed but was finally launched in July 2000. This service module was the key element, enabling continuation of ISS construction.
The design of the new MCC provided the floor layout of a Flight Control Room (FCR), as shown in Fig. 9; a typical console is also displayed in this figure. There has been a continuing interface with the MCC in Moscow since the first launch in 1995. This interface was supported by controllers within the MCC in Houston periodically and by a small team of U.S. resident flight controllers in the MCC in Moscow. Figure 10 portrays the new MCC as the central hub for the ISS, serving the requirements of both the Space Shuttle and the Space Station.
The transition to workstations in the MCC began with the early deployment of 66 workstations in 1991 that grew to more than 400 DEC Alpha 9000 workstations in operation in 2000. A high level version of the architectures of both MCCs is displayed in Fig. 11, workstations added considerable local console processing compared to the original mainframes that delivered data in fixed formats to consoles.
Figure 9. The Control Room in Building 30S layout is functionally very similar to the earlier MCC (shown here for Shuttle operations). The Flight Director’s console is typical of the new MCC, and it features CRT displays with color graphics and a digital intercom control pane.
Figure 10. The MCC Flight Control Hub has new interfaces with the Space Station Program Office Team and the MCC in Moscow.
The new capability in 30S was gradually phased into operation by carefully progressing from limited flight following Shuttle flights as early as 1993 to primary use of the new MCC for in-orbit control of STS 70 in July 1995. Shuttle landing control was exercised on STS 73 later in 1995. In May 1996, all Shuttle phases were controlled from the new MCC on STS 77. (The one specific function of launch trajectory tracking and determination was still performed in a MOC on the first floor of the early MCC but displayed to operators in the new facility. This last residual function will be phased to 30S by 2002.)
Figure 11. The evolution of MCC data architecture that now uses servers, local area networks (LANS), and workstations.
Figure 12. The history of major contractors and contracts for the MCC is shown across almost four decades.
The latest step in contracting for MCC services also occurred in 1998, when NASA awarded the Consolidated Space Operations Contract (CSOC) to Lockheed, which had acquired Loral, which had earlier acquired Ford Aerospace. To complete the contracting history (see Fig. 12), the Rockwell contract for STSOC had been reassigned to a joint venture of Rockwell (later Boeing) and Lockheed known as United Space Alliance (USA) before the MCC sustaining and operating segments of that contract were transferred to the new CSOC.
Perspective
For the past 40 years, the operating concept of a ground-based flight control team has been very effective. The concept envisioned the team responding to program objectives and staffed with all of the necessary disciplines for real-time decision making. The concept has been driven by the need for rapid and critical response to flight situations, within seconds when necessary. This has led to careful selection of personnel and a widely scoped training regimen from classrooms to fully integrated simulations with the flight crews in simulators. An early emphasis was established for flight controllers to develop as many of their own tools as possible, from functional schematic systems handbooks, to mission rules and procedures, and to the requirements for all of the processing, displays, and controls on their respective consoles. In-depth support to each console operator from staff support rooms has also been a constant feature. With variations for program content and mission phases, the concept and the makeup of the flight control team has been remarkably consistent for four decades.
In terms of interfaces with other operational units, the flight control team defined the technical requirements (voice, data, and other) for any transactions with these remote units and the procedures governing same. This has led to the flight control team in the MCC acting as the central hub on the ground for mission support with interface connections to units such as the launch team, range safety, network and MCC systems, DOD support, and payload organizations in their control centers at GSFC, JPL, MSFC, the Air Force Sunnyvale center, or in other facilities. A very strong interface with the spacecraft program office team of NASA/JSC engineering and industry teams has also evolved. This interface assured maximum confidence in resolving technical problems and a controlled methodology for engaging the program office team. Because the realtime flight control team has been augmented by and connected to all the necessary organizations, it has been a vital element in all of the achievements and successes of human spaceflight.
The MCC is the instrument by which this operating concept has been mechanized. Its fundamental architecture has three elements—front-end communication processing, a computation complex (now servers), and a display and control element (now work stations with local processing added). It has evolved over the years in equipment and configurations but has provided the same fundamental core functions.
After nearly perfect support to 114 missions by the original MCC, its third floor FCR is now maintained as a national historical monument commemorating Apollo 11. The second floor has been modified to support life science payloads. The original Building 30 MCC had performed nearly flawlessly as NASA achieved its human space goals during the last four decades. It is now retired with honor from its original mission, and the new MCC is poised to write its own chapter in human space history.
