P2K: Where are we, and what goes on here?
Art Thompson: This is the system test complex for the Mars Exploration Rovers ATLO, "Assembly, Test, and Launch Operations." This is where we will do most of our functional testing, electrical integration, and the system testing of the spacecraft itself. We will be testing the lander, the rover, and the cruise stage, so we'll be testing three spacecraft all in this area.
P2K: What are those wooden boxes doing?
Art Thompson: These are actually bookcases. What we'll do is we run procedures: everything we do in this building will be done executing procedures, and every day we'll have a stack of procedures that the test conductors and test operators are going to be accomplishing for the day. And once they've run the procedure they will actually start populating these bookcases, and by the time the mission's over, all of these bookcases will be filled and then some with the "as-run" procedures that we've completed.
P2K: Just so I have it for the record, give me your name and title.
Art Thompson: My name is Art Thompson, and I'm the ATLO Systems Lead. I'm responsible for making sure that we've got the test complexes set up right, we've got our test personnel down here trained, and we've identified what they're going to do every day, prioritized their task list, and make sure that they've got the resources that they need to actually come down and test the spacecraft. So we're... (where) the action happens. We come down and we run tests, and we actually issue the commands to the spacecraft using this equipment, and this is the first time we see the subsystems coming together, to see the spacecraft actually be born. And she'll develop a personality in here. We'll learn how to deal with her and eventually we'll take her back to Florida, put her on the rocket and launch her.
P2K: You guys have got to figure out a name...
Art Thompson: It's actually MER-1 or MER-2 until she's launched. Then it becomes MER-A and MER-B. So the first spacecraft launch will be MER-A, although by the time it launches we'll probably have an official name. But right now it's MER-1 while she's on the ground.
P2K: Just how big a deal is ATLO in the life of any mission?
Art Thompson: In my opinion it's by far the most important part. Of course, everything leading up to it is important because if you never get to the starting gate you never run the race. But at this point, all of the subsystems have designed their equipment, they've built it in "breadboard" and they've tested it, they've gone through a flight build and they've tested that in a standalone subsystem testing. And then they'll deliver it down here, and it comes down basically just like parts of your computer. If you open up your PC you'll see all these little boards. You take all these boards out, each board works standalone but it really doesn't do a whole lot. You bring it down here and we go through in the High Bay out there, just the other side of that glass, and we actually start putting it together, and that's when you see the computer starting to talk to the radio, and the radio actually being able to talk to the ground, and all of the instruments coming together and talking and it really becomes a viable entity instead of just basically a single cell amoeba.
P2K: You make it sound like a "no-brainer", so what is your job, what do you have to do over the next few months?
Art Thompson: Oh, it's really anything but a no-brainer. What we've found on almost every spacecraft, any time you build something, you design the interfaces, you have interface agreements, you come up with how things should work but when you plug them together for the first time you realize you didn't quite get it right. And so there's a lot of debugging in real time. In fact, one of the things that we will do down here is we'll really crawl along and test every signal on every line to make sure that we've got it right. Not only did we get it right, it's within the tolerances that we expect.
Once we've done that then we start to see how they play together, and things often times play very well alone, like little kids, and you put them together, sometimes two or three kids play great, and then you put four or five together, and then you get a bully in there and then they don't play well together. And we got to figure out which one's are the bullies and which ones don't want to play, and we've got to make them play. Once we get them all playing together, we then have to take them up and put them in a space-flight simulation.
We simulate what it's like to actually launch: it's very rocky, very rough, we vibrate it, we put it in an acoustic chamber, and basically we turn up the speakers real loud because when you launch on a rocket there's a really loud acoustical environment you have to survive. And then we put it in a "thermal vac" chamber and we actually put it out in space on Earth. So we vacuum... we suck out the air, put it in a vacuum, and we heat it up so it's exposed to the sunlight, or we cool it down so it's not, and the spacecraft has to be able to survive and not only survive but thrive in that kind of environment.
P2K: So what is going on behind you?
Art Thompson: Right now the activity going on outside... Okay, what we've got is the MTM or the Mechanical Test Model. This is a mockup, a flight-like mockup, of the spacecraft itself, the rover. The gold chassis on the bottom is the WEB, the Warm Electronics Box. The silver chassis on top is actually the REM, the Rover Electronics Module. That's a mass (ed: weight) mockup. It looks very similar to the flight unit, it weighs the same, has cable harnesses on it, and they're actually rehearsing loading the REM into the WEB. This is, when they're done with this next week that's going to go into some environmental testing where we're actually going to put pyrotechnic devices on it, and we're going to be firing off the pyrotechnic devices to see what kind of shock that structure can withstand and exactly how much force will be on that structure when the different pyrotechnic devices fire off.
P2K: Why do you need to rehearse putting a rack into a box?
Art Thompson: Because this is very, very tight. If you look out there you can see that we barely fit inside the box, and the WEB is very delicate. The components on the WEB, the aerogel, the coating of the WEB itself, you don't want to tear it, you don't really want to scratch it, and you've got to make sure that you really know how things fit together. And quite honestly, this is probably the first time that we've put stuff together, and oftentimes they'll find out that a tab is sticking out too far on the REM and it interferes as it's dropping down into the WEB, and you want to make sure that when you've got flight hardware that cost a lot of time and money to build, that you don't drop it, you don't break it as you're handling it. So they're actually using handling fixtures out there that are specially designed to lift up this equipment and maneuver it in position and lower it in place. And we're using this opportunity to test on non-flight hardware so you get the experience when the flight hardware comes in, you're much more able to handle the flight hardware with high confidence.
Another point you might consider. You can see the pipe sticking out on the back of the REM, the silver pipe that comes up in kind of a "U", an inverted "U". That's the thermal cooling and you've got to make sure that the cabling doesn't snag on the WEB as you're dropping it in, that you line up all the tabs, the cooling pipes basically come out the feedthrough as planned. You know, you've designed this on paper, but the first time you put it together...
P2K: How to assemble something is part of the overall design of the mission, right?
Art Thompson: Absolutely. Those are deliverables, in fact. We spend a lot of time identifying what kind of lift fixtures, what kind of support fixtures. In fact, on the left side of the high bay out there is the design for Mars Pathfinder, but we're going to use that same fixture to support this spacecraft. So there's all sorts of different handling fixtures that we require.
In fact, you might want to look over here, which is kind of an empty REM, WEB. This is a really raw mockup of a WEB, and you can see a raw mockup of the REM inside. And the fidelity of this is much lower than the fidelity of the MTM that you were just filming out in the High Bay, and they use this so they can get a feel for exactly how things are going to come in. In fact, when you are looking down, there are struts that fit down from the REM to the WEB, and they're designing how they actually mount this in and how they actually bolt it in. If you can look over here you can see the shock absorbent struts and basically all the space in here will all be used up or taken up by other instruments, and in fact over here is a cable mockup of the REM itself.
From the top you can see where all of the individual computer boards are. In fact, this is a bulkhead of connectors, and you can see how many connectors are involved there. And on the sides we have our RF subsystem on the outside, on both the front and the back, along with batteries and RHUs.
Here's a mockup of the mini-TES electronics itself, which will be sitting up in the front of the WEB. So we spend a lot of time and effort designing mockups to see that everything does fit together and that we can route cabling, and how we actually route the cabling through the inside of the WEB.
And it eventually matures into something like a full-scale cable mockup here, where you can actually see this has six wheels and it has the rocker-bogey suspension on it. In fact, we have the Rover Equipment Deck, or the "RED," which is a mockup of the solar panels, and inside there this is actually the PMA, the Pancam Mast Assembly, as it's stowed with a UHF antenna. In front here you can see we've got one of our external cable bulkheads, and these things are basically arrayed almost all the way around the rover. We have cameras, camera mockups down low, we have our instrument arm down low in a stowed position. And as you walk around you'll just see more and more of these external bulkheads that allow us to access the internal workings of the vehicle and interface to devices on the outside.
P2K: How many of these kinds of mockups would you end up making?
Art Thompson: I can't tell you the exact number, but I know that we've got about five or six of them right now. I don't know what the total number is; that's not really my area of expertise.
This model, in fact, was used for some of our thermal vac prototype testing. We took one of these up into a ten-foot chamber just to see if it would fit. We've got some pictures of us taking this vehicle and actually putting it inside the chamber and walking around inside to see we had enough room. Because, if you're familiar with the workings of this vehicle, we've got the Instrument Deployment Arm (IDD), which swings out and tries to come out and find a rock and places three or four different instruments against the rock. When we're in a "thermal vac" chamber, or doing a surface thermal balance test, we want to be able to deploy this thing, and inside a ten foot chamber we really don't have enough room to deploy the IDD and have it really do anything meaningful. So we took basically this type of model up, because it's a full-scale actual mockup of the vehicle, and put it in a thermal chamber and saw what we were able to do by moving the stuff around inside the chamber.
P2K: What's in these bookcases behind us here?
Art Thompson: These bookcases are used for the "as-run" procedures. As we execute procedures here we take all of the documentation and we put them in these bookcases. By the time we're done, all of these bookcases and more will be completely full, and wherever the spacecraft goes-this is just for one of the two spacecraft-wherever the spacecraft goes, these documents accompany the vehicle up until the time we launch it. Then these are basically our "Bible" for the spacecraft. It tells us everything we've done, every test we've performed, all the telemetry that's been gathered on it, and we need this information to tell us basically calibration parameters, using potentiometer values for how far we can manipulate the spacecraft, actual readings from temperature transducers. All of that information is very valuable to us. We can't just forget it because when we're actually using the spacecraft during the mission, we refer to the mission to know exactly what the temperature is. When we were performing "thermal vac" testing we calibrated all of the transducers to the actual mission temperatures, and all of that information will be captured in the "as-run" procedures. And should something fail, we can actually go back and do a history on the spacecraft and see if there was an indication that parts were getting stressed or were "going south" on us during the test. Hopefully we can catch that before we launch and we can do a trend analysis and say, "Gee, that part's not looking too good, perhaps we should swap a board out" or, go in and take some action to make sure that the spacecraft stays healthy.
P2K: You've just got two lonely binders....What are they?
Art Thompson: Those are actually an artifact of an ATLO readiness test that we performed. We actually, it was two weeks before the start of ATLO. We got all of our system test complex operators down here. We went through a series of training through the morning, and then in the afternoon we sat at the stations and we ran two procedures. These are real procedures that we will be running when we have the spacecraft, but right now we do not have the spacecraft, so we went through mock execution of those procedures, and they actually gathered mock data from the test bed and they filled those out so that's our first set of "as-run" data on a virtual spacecraft. In the next week or so when the spacecraft, parts of the spacecraft start showing up, we'll start populating this with real as-run spacecraft data.
P2K: Is it in the womb yet or is it out? Does it have a personality as far as you're concerned?
Art Thompson: It's not quite out of the womb. Out of the womb is when it shows up behind these walls here. It's still being developed, but yeah, the spacecraft, every spacecraft has a personality. I guess it goes back to when I was working on Mars Pathfinder, on the Sojourner rover. She was named after a female, and we tend to think of our spacecraft as having personalities and being a woman.
P2K: What is the personality of the rovers?
Art Thompson: Well, so far she's been hesitant to make an appearance. We're hoping that when she gets here she'll be very outgoing and very easy to get along with. But I'm sure she'll have her stubborn portions that we'll have to work through to.
P2K: What is it that you personally like about working on such complicated things?
Art Thompson: That's a great question, and I'm not really sure I can answer it. This is my fourth time down this path, the flight path, and I just really enjoy taking what has been put together but not necessarily functional yet, and making it work together as a system. And then seeing it actually launch and get off to another planet is amazing. Thinking back on Mars Pathfinder, I was part of the Sojourner team, and coming to work every morning and looking at the pictures of a sunrise on Mars and saying, we're on another planet, and here we are driving an RC (remote control) vehicle, I mean think of it-how many people get paid well to come to work and drive RC vehicles in really exotic places? We just get to have maybe the most fun you can have doing stuff like that.
P2K: The Mars Exploration Rovers are something different from Pathfinder.... Is it as much fun working on the Mars Exploration Rovers mission as it was on Pathfinder?
Art Thompson: It's different. Every spacecraft is different. They all have their personalities, like we talked about earlier. This one is a little more pressure up front than there was on Pathfinder. We were pretty much left alone on Pathfinder to do our mission. Like you mentioned, because we have lost a couple of spacecraft recently, and budget's getting tighter, and we are building two, there's a lot more pressure up front to make sure that we've got our act together.
We're doing things on this project that I have not had the opportunity, or the pleasure to do because we didn't have the resources up front. We're running ATLO readiness tests, which I'd never heard of before. That's never been done, to the best of my knowledge, here at JPL. We're spending a lot of time getting all of our ATLO operators down here before the hardware gets in so that we know what to expect when they're here. The ATLO operations is a mixture of people who've been here before-"been here, done that"-and people who've never been down to ATLO. And if you've never experienced the thrill and the terror of being in ATLO, it's a real eye-opening experience.
P2K: What's the "thrill" and what's the "terror"?
Art Thompson: The thrill is to actually see a spacecraft come together and know that you're part of something that's unique in history. The terror is when you turn it on and it doesn't work and you realize that you can't slip the launch. The launch is going to happen. You're either going to make the launch or you're not, and if you don't make the launch it's over; it's all for naught. It's not like you can launch it late. So there's a lot of terror, there's a lot of concern that you do it right the first time, because with flight hardware you oftentimes don't get a second chance.
P2K: Time, budget, and mass... What are the three things that make you most concerned about your part of the mission?
Art Thompson: By the time it gets here it's always the schedule, because... things always slip, and "slip" means they're later in their delivery than planned. When you sit down and you plan a mission you always plan optimistically and you plan reserve, margins, so that when things don't go as planned you still can make it on time. Well, we're finding that as usual we use up the planned time and the "planned margin," and the only place you can get more time and subsystem development is to compress ATLO, so by the time you get to ATLO, it's usually a few weeks behind schedule where you had initially planned, when you started planning for this a year or two ago. And that's no different for the Mars Exploration Rovers mission. We are behind where we had initially planned to be, so we're constantly coming up with ways to shorten tests, possibly perform different tests on different spacecraft, and use the similarity between the two spacecraft to verify the flight worthiness of both spacecraft. So it's a challenge to constantly keep a viable ATLO schedule together as they constantly compress your schedule.
P2K: If you had a magic wand...and you could make this mission more likely to go smoothly for everybody, what would you do?
Art Thompson: In a perfect world, how would we do things? I don't really have an answer because what we're doing here, we're doing something that no one has done before, we're building things that no one has even thought of building before, and we're putting together a picture that we only have a rough idea of what it really needs to be. And in fact, even as we enter ATLO, we're still making changes to the spacecraft.
I would guess in a perfect world we could have unlimited schedule to build prototypes, to figure out exactly what it is you'd want to build, and then have lots of time to build the subsystem portion of it. Unfortunately, because of the rotation of the planets, we have a tight deadline. We've got to launch. So you come up with a concept for a mission... basically you know when you've got to launch that and you don't have any opportunity to delay that launch. The planets align, you've got to launch or you miss that opportunity and it could be as much as twenty-six, twenty-eight months later that you get an opportunity to launch again.