2 Votes
Also worked at JSC
I worked at JSC as a contractor for 7 yrs on real-time fight control systems and mission control center code used around the world, on the shuttles and space station.
While I can’t comment specifically on the CRAP system or any other that I didn’t work, I can assure you that for the 4 systems on which I worked with teams of 3 to over 500 developers:
a) code maintainability was as a critical aspect
b) efficiency was an issue, almost always (imagine running on 15+ yr old hardware and still adding code to it with the old RAM still inside)
c) reformatting code just to make it pretty was against the coding standards. We didn’t touch a line if we didn’t have clear authority to change it for functional reasons. We did make it pretty while working on it, just not when checked in.
d) self documenting code is a good idea, provided the coders aren’t small minded and can step back and think like the next coder will. That is a rare skill, in my 20+ yrs of coding. Commenting based on what the next guy wants to know isn’t easy. Some compilers had stupid limitations on variable lengths.
e) nobody should be allowed to write new code until they’ve been a maintenance programmer for 2 years. You learn a ton on how not to code that way. It doesn’t really matter what language.
f) for systems that allowed dynamic memory allocation, a memory leak was considered a Sev 1 bug. Real-time FC systems don’t usually allow memory allocation.
g) unclear code comments were considered minor bugs, including spelling errors. Had to be corrected before the code was submitted.
h) every line of every program was reviewed by at least 2 other people who physically signed a piece of paper saying they didn’t find any errors. When you sign your name to a review and hand it in, you tend to actually perform a careful review. That paperwork with my signature can still be found, I’m certain.
i) if someone assigned to review design, code, or test work wasn’t ready, we held up the review until they were. PERIOD. If they didn’t consistently review and catch errors that other reviews found, they were encouraged to “find someplace else” to spend their days via peer pressure. This wasn’t needed very often. We all counted on each other to catch mistakes that could get people killed or destroy billion dollar vehicles.
For the GN&C software I developed, only 2 errors made it outside our group in over 5 years were attributed to me. I introduced many errors, but the team caught almost all of them. I caught my share of errors in other peoples’ code too. When I left that team, all 11 remaining developers introduced less than 1 error every 2 years combined.
Last I heard, a few of the guys now work at pace maker companies writing software. My mother has a pace maker and I’m very happy these guys probably ensured it works perfectly, EVER TIME.
RE: Also worked at JSC by Jackson
Sounds like you were level A software. Sadly, since we are level C, the level of care was considerably less than in some other systems like FEPS or DataAcq or MissionControl. One source of my lamentations is that I work in a place that has good code and good programming standards, and I just happened to get stuck on the red-headed stepchild that doesn’t.
c)reformatting code just to make it pretty was against the coding standards. We didn’t touch a line if we didn’t have clear authority to change it for functional reasons. We did make it pretty while working on it, just not when checked in.
This is why so much of our code still looks the way it does. Whenever we get an Anomaly Report we try to clean the code around the problem up so the reviewers will have an easier time understanding what it did and now what it does (svn really has helped with this with its svn diff command, rcs, while workable, sucked when reviewing code). Then again, we aren’t a critical system.
d) self documenting code is a good idea, provided the coders aren’t small minded and can step back and think like the next coder will. That is a rare skill, in my 20+ yrs of coding. Commenting based on what the next guy wants to know isn’t easy. Some compilers had stupid limitations on variable lengths.
This is true, but self-documenting names usually don’t hurt. They at least give a place to start. I haven’t run into any variable length limitations since I have been here.
e) nobody should be allowed to write new code until they’ve been a maintenance programmer for 2 years. You learn a ton on how not to code that way. It doesn’t really matter what language.
Ah, yes. Now when you say maintenance programmer, you do include that that person will write new code to fix old problems, right? You just mean brand-stinking-new code? That would be nice, but with the amount of work to do and the lack of supply in experienced maintenance programmers, this is not followed in my section of 3 (where the most experienced member has 18 months on the system, and he isn’t the lead).
f) for systems that allowed dynamic memory allocation, a memory leak was considered a Sev 1 bug. Real-time FC systems don’t usually allow memory allocation.
g) unclear code comments were considered minor bugs, including spelling errors. Had to be corrected before the code was submitted.
h) every line of every program was reviewed by at least 2 other people who physically signed a piece of paper saying they didn’t find any errors. When you sign your name to a review and hand it in, you tend to actually perform a careful review. That paperwork with my signature can still be found, I’m certain.
i) if someone assigned to review design, code, or test work wasn’t ready, we held up the review until they were. PERIOD. If they didn’t consistently review and catch errors that other reviews found, they were encouraged to “find someplace else” to spend their days via peer pressure.
These are all still true for level A software (critical) and to some extent for B and C (my system). These are the procedures that I read and was so impressed with. Our system allocates memory like its going out of style (of course, we have to for our system’s function). Of course, half the time, we find that the original code doesn’t take into account overrunning allocated array length or even where pointers are pointing in some cases.
We actually had a bug a few months ago where a fixed length array that held a ton of data was searched to process its information. As we iterated through the array, the current index was checked to see if it was greater than the size of the array. Great right? Except they did the check before incrementing the index that would be used to extract array information. End result was a lot of funky data in a program that called a program that called the code in question. Best part? No comments.
RE: Also worked at JSC by Anonymous
I used to work on some C code(~100MB) once which contained 50% dead code, and my task was to document the functionality. People write bad code to keep their jobs. Have seen it so many times in the industry, and it has tremendously worked. Is it right/wrong. I used to feel at one point that it was wrong.