What operating system(s) were used in the space shuttle?

Question

Is there any possibility to learn and use the operating system languages used in the space shuttle?

Answer 1

I was a NASA contractor working on the same contract, but with 3 different companies, writing GN&C code for the space shuttles from 1989 until 1995. Initially, it was Ford Aerospace, then Loral, then Lockheed-Martin. IBM was the prime most of that time. It has been a long time since I did any work like that, so don't hold what I write below as gospel. Memories of a middle-aged guy.

I have some time this morning, so I'll spew some thoughts ...

The Shuttle OS isn't general purpose. It is purpose built, designed to be loaded with 1 program, instruction pointer set to 0 and then started running. I honestly don't know how that worked, since we never touched any of the flight computers. I've only seen photos and recently saw one inside a glass case at a museum. When I worked there, we didn't have access to the computers directly. Because the computing hardware is modified IBM 360-based systems, that is the machine language used by all the main systems. There were 5 GPCs - general processing computers. During ascent and landing, 4 ran the PASS - Primary Avionics System Software and 1 ran the BFS (backup flight software). The BFS was written and maintained by a different contractor, Rockwell. I know nothing about the BFS except it was never used. Heard rumors that eventually that contract was canceled because the PASS was of such high quality there wasn't any need for the BFS. Perhaps someone from Rockwell can answer?

The vast majority of programming for the GPCs was written in HAL/S. This is a compiled language similar to PL/1, but with matrix calculation extensions and a multi-priority scheduling system. Basically, it supported 254 threads, but we only used 3. HFE, MFE, LFE - Hi, Mid, Low Frequency Executives. The priority was H, M, L as well. HFE ran at 25Hz. MFE ran at 12.5Hz and Low ran at 0.5Hz. Mid-instruction, a higher priority could interrupt a lower one. It was important to leave time so that lower priority tasks would have sufficient time to run. Some complex guidance code would overrun the allotted time and had to be highly optimized. Since these were real-time systems, a late answer is just a bad as a wrong answer.

The PASS software ran in a redundant set with break points along the code paths (managed outside our application code). Periodically, each computer would get to a checkpoint and compare with the other computers in the "redundant set". Basically, they would vote. Majority rule. Any computers outside the majority would be dropped and all future values from that one would be ignored. If only 2 systems are in the redundant set and they disagree, the computer which arrived first was deemed correct. I don't believe they ever had fewer than 3 GPCs running. Basically, only 1 was dropped over code path disagreements (1996 and earlier). I don't know anything after 1996 when I left the NASA area.

Due to the poor performance of passing parameters on the stack, we used global variables. There was a strict naming convention which said which priority the variable could be used on, which subsystem it was part of and what type of variable it was (int, scalar, double, bit, hex). I think we were limited to 32-character variable names, but don't quote me, it wasn't an issue. Dynamic memory wasn't allowed to be used. Used pointers once (called "NAMED" variables in HAL/S) and needed to get a variance approved. Since many of the programmers were trained in engineering (including me), not CS, pointers were not well understood. I was taking advantage of pointer overruns to access contiguous memory variables which were beyond the allow size supported by the compiler for arrays. Basically, I declared multiple arrays back to back and used a pointer to treat their memory as 1 variable. Nothing fancy, but it meant having to ensure that the compiler didn't optimize storage in unexpected ways. The compilers and linkers were specific to the shuttle program and not bug-proof. They were fairly high quality, however. Some of our infrastructure guys would perform compiler audits looking for odd issues. One guy, JY, had so much experience and knowledge that asking him questions was frustrating. 10 yrs later, I became just like him at my job and understood that simple questions don't always get simple answers. "It depends" is a valid answer and more info is required to answer.

Packing bits was common since memory was limited. Much of my code was binary logic based on bit values to decide which calculation would be performed under the vastly different situations possible.

On orbit, programs were overlayed just like the old MS-DOS "overlay" methods. I didn't work on the Vehicle Utility code, VU, but it handled the arm controls and other things outside of flight control (HiFE), life support (LoFE) and guidance (MedFE).

If you are interested in the people doing the work, there have been a few articles about us written over the decades. A coworker left the project to work for AMD. After a year, he returned because AMD was seeking "good enough" engineering, not perfect engineering. His story is in "Fast Company" - "The Write the Right Stuff" https://www.fastcompany.com/28121/they-write-right-stuff

Some other "languages used" onboard weren't really languages. DFG - Display Format Generator (or something like that). It was the language that drove the DEUs for the astronauts to see status and manage the computers. It was extremely rudimentary with only BE (branch equal) and BNE (branch not-equal) type comparison operations. No label jumping - it only used counts of instructions. Basically, if X is true, jump 15 instructions and continue. It was very simple, but hard to get a display correct the first time. We had no interactive ways to see the screen output based on our code. Had to wait for printouts of the batch test runs to see the drawings.

For initial values of the tens of thousands of parameters, there were 2 different pre-processors - I-Load and K-Loads. I never touched any K-Load stuff. The I-load stuff specified initial values which were either copied to a memory location or had calculations performed based on other inputs which were then copied into a memory location. These locations could be into the RAM or onto tape storage which was loaded during major "OPS" changes. OPS 1/6 for launch. OPS 3 for Landings. OPS 2/8 for on-orbit stuff. I-loads happened PRE-mission. The i-load operator precedence wasn't the normal one taught and used world-wide in math classes everywhere. PMDAS - Parenthesis, Multiplication, Division, Addition then Subtraction. Usually, multiplication or division are of the same priority, but not with the i-load tool. Same for addition and subtraction. Found a bunch of bugs in my mentor's code when I worked there just a few months because of this. I was reading the i-load manual and was new enough NOT to assume anything. My mentor, DC, had been working there at least 5 yrs, perhaps 10. He was EXTREMELY smart. They had a formal code review which I missed, wasn't prepared. A few days later, I had finished reviewing his code and asked about why those statements were ok, referencing the I-load user's guide. He read it carefully, then said I'd found a number of bugs that everyone else missed. The formal review at IBM (we were a sub to them) was delayed and he fixed the issues, re-reviewed it in-house, and sent over new review at IBM. BTW, finding errors in code reviews was extremely rare. Everyone worked extremely hard to help each other out. After the first few months working in the group, most of the programmers wouldn't have any bugs found in their code even at our in-house reviews. A few of us would have 1 or 2 errors found per year in-house. It was extremely odd for IBM reviews to ever find any errors. We used Deming's statistical process control methods to tweak our internal processes as needed to address any errors found. When I left the team, 1 bug was being found every other year in the code. It was generally believed that zero SEV-1 bugs remained in the software. Jim Orr created a report in 2010-ish which showed there were over 100 SEV-1 bugs in the code in 1994 which would be found later in the FSW program. A SEV-1 bug is loss of crew/vehicle, I believe. http://www.slideshare.net/JamesOrr4/space-shuttle-flight-software-pass-loss-of-crew-errors-jk-orr-20150827-52150515

We used MVS-based mainframes on the ground to build all the software running TSO/ISPF. Version control was managed by an IMS database. JCL was used to submit compile, link, testing, printing tasks to the system. We had a number of different tools to help research the code - basically different menus written in ISPF macro languages or REXX (after I moved on) were used to search the different code bases. Every flight had different code, but most were based off Operational Increments, OIs. I worked on OI8D-OI24 and a little planning on OI25. My favorite tool was called HALSTAT. It provided reports about all the variables used in a module (file) throughout the rest of the code. References, assignments and both had different codes. I think the codes were 2, 4, and 6, respectively. I miss HALSTAT in all my coding since then, though c-tags and etags are better than nothing.

Some systems I worked on in those years were:

• Nosewheel steering redesign (JP and JV)
• 2-Engine Out Abort (Don't recall the complete team)
• 3-Engine Out Abort (SM, JP, LP were primary, but EVERYONE helped).
• Mir Docking (BW and PD)
• Drag chute Deployment (PD primary)
• Main Landing Improvements - basically they were blowing out tires, so smoother landings were needed. If you watch landings after 1992, they are all VERY, VERY, smooth thanks to this code.

Just to be clear, I introduced 1 DR in my time there (1 bug that got passed all the internal and IBM reviews) - on the Main Gear Landing Improvements. It was discovered about a year later in level 6/7 testing. I had lots of bugs found in-house by my peers in my first year. After 1 really bad review with about 10 bugs, my boss asked if I was happy with the work. I had read the requirements differently than everyone else based on some indentation. Really it was 1 error, but I was consistent. However, in the code, it was about 10 errors. It is humbling when others catch your mistakes. We were very close members of the same team. Our signatures on review sheets meant something important. We never forgot that lives were at stake and national pride.

There were many, many, other projects and fixes.

I do not miss wearing ties. ;) I do miss the mostly 8-5 hours.

The mission control software and payload operations centers around the world are completely different too. Worked on the team that replaced the Apollo era control centers with the Star Trek NG-like systems running mainly DEC Alpha systems with some SunOS, HP-UX, AIX and Irix systems in the back rooms. For reference there were about 600 Alphas and less than 20 other systems. When we deployed them, they ran OSF/1, a Unix variant, but that name changed to Digital Unix ... this was before Compaq bought DEC. Visited the MSFC a few years ago and saw their POC for space station support. Couldn't see which OS was running and the tour guide didn't know. The console furniture was all different than what we deployed in the early-mid 1990s.

I don't know of any way to run the PASS HAL/S software or use the hardware today (in 2017). The OS isn't interactive. It communicates only with specific hardware which nobody will have or can get at home. Without a DEU and a flight key entry system, control sticks and hundreds of avionics switches/knobs, I don't see how to make this software usable. Someone would need to make an emulation layer for all of that. Suppose it could be done.

Please recall that all my direct knowledge ended in the mid-1990s. Things were probably updated later in the program which could have changed many things. For example, I think the DEUs were replaced with LCD screens somehow. Any code changes to support that, if they didn't just put in an emulation layer for DEUs is unknown to me.

Back in the 1990s, I did find an IBM 360/370 ASM emulator for Intel CPUs. At the time, it wasn't good enough even to run my beginner-level IBM 360 ASM homework on. It is hard to explain how complex the software development processing was. Everything was cross-compiled and cross-linked from the MVS system for the AP101/s computers.

Looked through the public NASA code archives today and a search didn't find any "shuttle" related code. Would love to see how GPE (a module name) was changed over the years.

Read elsewhere that the early HAL language guide was available, but the HAL/S language is different. For example, I don't recall seeing any way to "print" anything in HAL/S - doesn't mean that instruction was removed, but there wasn't any device to "print" at. Inputs came from HW devices and were stored into variables. Outputs were written to HW devices. By the time I came along, all the I/O stuff was long written and working, so never needed to learn how it actually happened.

Also, I'm not knowledgeable about any specific missions. My daily work was far removed from current missions. We didn't stop to watch launches/landings during the work day. Our code was generally written a year or two prior to any mission use. I suspect IBM did all the closer per-mission code changes. Around 1995/96, the IBM group was sold to Loral and those two groups were merged into a single team. Some people transferred inside IBM but others loved the work and stayed.

A basic HAL language would probably be written with a translation layer in almost any of the popular languages today. Perl, Python, Ruby, Java, C++ could handle it. At least for simple, 1 module, programs. Perhaps similar to the way C++ began as a C preprocessor.

Creating a translation layer that could handle the full multi-priority, interrupt-enabled, shuttle software would be a vastly different, huge, project.

Sorry for going off topic. Hopefully someone will find this slightly interesting.

Answer 2

For the flight control computers, aka General Purpose Computers (GPCs) there is a good writeup on this topic in the ever-useful Shuttle Crew Operations Manual page 2.6-20.

DPS in this quote stands for Data Processing System

DPS software is divided into two major groups, system software and applications software. The two groups are combined to form a memory configuration for a specific mission phase. The programs are written in HAL/S (high-order assembly language/shuttle) specifically developed for real-time space flight applications.

System software is the GPC operating system software that controls the interfaces among the computers and the rest of the DPS. It is loaded into the computer when it is first initialized. It always resides in the GPC main memory and is common to all memory configurations. The system software controls GPC input and output, loads new memory configurations, keeps time, monitors discretes into the GPCs, and performs many other DPS operational functions. The system software consists of three sets of programs. The flight computer operating system (FCOS) (the executive) controls the processors, monitors key system parameters, allocates computer resources, provides for orderly program interrupts for higher priority activities, and updates computer memory. The user interface programs provide instructions for processing flight crew commands or requests. The system control program initializes each GPC and arranges for multi-GPC operation during flight-critical phases.

One of the system software functions is to manage the GPC input and output operations, which includes assigning computers as com- manders and listeners on the data buses and exercising the logic involved in sending commands to these data buses at specified rates and upon request from the applications software.

The applications software performs the functions required to fly and operate the vehicle. To conserve main memory, the applications software is divided into three major functions: Guidance, navigation, and control (GNC), Systems management (SM), and Payload (PL).

(I summarized the last sentence in the last paragraph, it is not a direct quote. Also, the bolding is mine.)

Answer 3

Reread the question title "used in the space shuttle" again, can add a different answer for the astronaut laptops in the mid-1990s.

The astronaut laptops were IBM thinkpads (it was 1996-ish), never knew thinkpad models. They were running some sort of Windows, definitely NOT win95 which was too new for flight use at the time. Could have been Windows for Workgroups 3.12 or NT. Not certain. Nothing really odd about the OS from what I remember. The laptops had extra batteries attached under the normal laptop.

Was part of a team writing a document management system, Hyperman, for all the flight control rooms, and payload operations centers computers as well as shuttle and space station laptops.

It was written in C++ for SunOS, Solaris, HP-UX, AIX, Irix, OSF/1 to start, but also ported the code to Windows and MacOS.

Borland C++ was used to compile the code for Windows using the win32s libraries just like any other Windows programmer of the day would for that platform if they wanted 32-bit software. Don't recall the version used. So, if you'd like to learn that, just learn Windows programming. Borland's old compiler is a free download these days.

Hyperman was built using commercially available cross-platform libraries, Faircom C-Tree and Visix Galaxy. We had a requirement for solar-system-wide, royalty free, distribution. ;) Galaxy was an amazing tool - crazy great, but very expensive. Looks to be about 10x less expensive now.

Don't know what happened to Hyperman. Think NASA dropped using it after paper documentation inside the FCRs and POCs was demanded by the flight controllers.

Answer 4

While this answer addresses several issues, I'd like to clarify a point raised there:

A basic HAL language would probably be written with a translation layer in almost any of the popular languages today. Perl, Python, Ruby, Java, C++ could handle it. At least for simple, 1 module, programs. Perhaps similar to the way C++ began as a C preprocessor."

This does not seem to be the case. The Mars rover (MER) software is written in C (2000).

According to Glenn E. Reeves, MER Flight Software Architect at JPL/CalTech:

The Flight Software is coded primarily in ANSI C, with some targeted assembly code and some C++. The size of the system, in source lines of code (SLOC), is [300K] but this value does not include the operating system.

Although many aspects of the design are objected-oriented, the features of the language incorporating inheritance and polymorphism are not exploited. We have found that when implemented to their fullest extent in C++, these constructs result in detrimental code size and add the potential for non-deterministic behavior.

I'm currently looking for the original source for this...

Answer 5

I am not SE-seasoned enough to add a comment, but that first answer was very thorough, but as someone pointed out, the BFS was in the Shuttle all the way to the end. There were at least 2 reasons for the BFS. You hinted at one, but had the wrong answer, as far as I know. If the PASS got down to only 2 strings, it is hard for them to vote. First-in was not the answer, the BFS Engagement was the real answer from NASA procedure. The other reason was in one sense similar. If the PASS had a generic SW bug, that would necessarily involve all 4 boxes, the BFS would be engaged. (We had the luxury of actually working with the GPC. We had a lab procedure to get us loaded and running with a new build as we went into HW/SW and full load requirements testing.) I must confess that we, Rockwell, goofed up on our first delivery on the improved NW steering to NASA, where we had a pourly documented feature of the IO SW which was writing the HW NW steering command from 2 different application SW locations, one of which had a value of 0.0. Because of that type of error, we couldn’t find it in our testing, which was application SW to SW. NASA, of course, saw it right off the bat as the NW was commanded to a correct value and then toggled to zero at a lower processing rate.

Answer 6

It seems that none of the other answers mention this, but there was a Space Shuttle Cockpit Avionics Upgrade project in ~2004, part of which was a selection of a new OS for the Space Shuttle.

The decision process was fascinating in and of itself because it used Bayesian networks to choose the best system, you can see the details in this document.

The chosen system was VxWorks by Wind River Systems, closely followed by Integrity by Green Hills Software. However, despite all the potential benefits, the project was eventually cancelled and a case study released by NASA.