{"id":40940,"date":"2009-11-12T21:18:05","date_gmt":"2009-11-12T13:18:05","guid":{"rendered":"https:\/\/wp-productionenv-bjg9h2g2bgg5b8aa.southeastasia-01.azurewebsites.net\/news\/sts-129-practising-launch-flight-controllers-put-to-the-test\/"},"modified":"2009-11-12T21:18:05","modified_gmt":"2009-11-12T13:18:05","slug":"sts-129-practising-launch-flight-controllers-put-to-the-test","status":"publish","type":"post","link":"https:\/\/starpath.global\/news\/sts-129-practising-launch-flight-controllers-put-to-the-test\/","title":{"rendered":"STS-129: Practising launch \u2013 Flight controllers put to the test"},"content":{"rendered":"<p>The STS-129 ascent\/entry flight control team and flight crew concluded the fourth and final integrated ascent simulation at the Johnson Space Center (JSC) on Tuesday. NASASpaceflight.com\u2019s Philip Sloss was given a unique insight by observing the sim runs conducted by Flight Director Bryan Lunney and his team.<\/p>\n<p>STS-129 Sim Runs:<\/p>\n<p>In preparation for the upcoming launch, the ascent\/entry flight control team \u2013 led by Flight Director Lunney, and Mission Commander Charles Hobaugh \u2013 went through four ascent simulation runs, each of which presented different scenarios.<\/p>\n<p>The flight control team was assembled in the Shuttle Flight Control Room, with their \u201cbackroom\u201d support personnel working from the Multi-Purpose Support Room. Along with Commander Hobaugh, Pilot Barry Wilmore, Mission Specialists Randy Bresnik and Leland Melvin also took part in the simulations from the motion-based Shuttle Mission Simulator.<\/p>\n<p>The scenarios were choreographed by a simulation team that monitors the actions taken by the flight control team and flight crew during the different sim \u201cruns.\u201d For this last simulation prior to flight, four ascent \u201cruns\u201d were conducted.<\/p>\n<p>Run #1:<\/p>\n<p>In order to fine-tune the reactions of the flight controllers and crew, several malfunctions are scripted into each run. The teams aren\u2019t aware when such issues may arise, and anomalies can even occur simultaneously.<\/p>\n<p>The high pace of ascent events became apparent while listening to the flight director loop. For example, just about the time that CapCom Chris Ferguson relayed the \u201ctwo-engine TAL\u201d call from the Flight Dynamics Officer to the crew, the Booster officer reported a small helium leak on the left main engine. The crew were subsequently asked to make a few switch throws on the flight deck, in order to close related valves in an attempt to try to isolate the leak.<\/p>\n<p>Spaceflight<path d=\"M7.59009 18.59L9.00009 20L17.0001 12L9.00009 4L7.59009 5.41L14.1701 12\" style=\"animation: initial !important; background: initial !important; border: 0px !important; box-shadow: none !important; color: inherit !important; cursor: inherit !important; direction: inherit !important; display: inline !important; fill: currentcolor !important; filter: initial !important; float: none !important; margin: 0px !important; opacity: initial !important; outline: 0px !important; overflow: initial !important; padding: 0px !important; stroke: initial !important; transform: initial !important; vertical-align: initial !important; visibility: inherit !important;\"><\/path>SpaceX<path d=\"M7.59009 18.59L9.00009 20L17.0001 12L9.00009 4L7.59009 5.41L14.1701 12\" style=\"animation: initial !important; background: initial !important; border: 0px !important; box-shadow: none !important; color: inherit !important; cursor: inherit !important; direction: inherit !important; display: inline !important; fill: currentcolor !important; filter: initial !important; float: none !important; margin: 0px !important; opacity: initial !important; outline: 0px !important; overflow: initial !important; padding: 0px !important; stroke: initial !important; transform: initial !important; vertical-align: initial !important; visibility: inherit !important;\"><\/path>Spaceflight news subscription<path d=\"M7.59009 18.59L9.00009 20L17.0001 12L9.00009 4L7.59009 5.41L14.1701 12\" style=\"animation: initial !important; background: initial !important; border: 0px !important; box-shadow: none !important; color: inherit !important; cursor: inherit !important; direction: inherit !important; display: inline !important; fill: currentcolor !important; filter: initial !important; float: none !important; margin: 0px !important; opacity: initial !important; outline: 0px !important; overflow: initial !important; padding: 0px !important; stroke: initial !important; transform: initial !important; vertical-align: initial !important; visibility: inherit !important;\"><\/path>\n<p>     (adsbygoogle = window.adsbygoogle || []).push({});<\/p>\n<h4 class=\"widget-title penci-border-arrow\">See Also<\/h4>\n<ul>\n<li>LIVE STS-129 UPDATES<\/li>\n<li>L2 STS-129 Special Section<\/li>\n<li>Click here to Join L2<\/li>\n<\/ul>\n<p>\u201cThe main engines consume helium, it goes into a seal and keeps the hydrogen and oxygen separated by just pushing helium through there and then it just exhausts out to the ambient area,\u201d explained Lunney in an interview with NASASpaceflight.com after the simulation.<\/p>\n<p>\u201cSo it\u2019s using helium from its tanks and it has a certain flow rate; the Boosters [Booster officers] are staring at their displays and in this case, the flow rate was a little bit higher than it should have been. I think it was about twice (as much, if I remember. So to him that\u2019s considered a \u201csmall\u201d helium leak.<\/p>\n<p>\u201cSince it was so \u2018small\u2019 \u2013 and I use quotes, because in my opinion, anything I can see is not small \u2013 it was small enough to not be a concern to the overall mission. The big helium leaks mean I\u2019m not going to be able to run the engine until MECO, for eight and a half minutes \u2013 it\u2019s going to shutdown sooner.<\/p>\n<p>\u201cThe procedures onboard would be difficult for the crew to do by themselves, so Booster just walked them through what few isolation steps we can do to see if the leak was in a particular piece of the plumbing. So we did that manually, \u2018OK, close this valve, no joy, close that valve, no joy, open back up\u2019. And the in-open, out-open stuff, that just refers to a particular valve that allows that engine to take helium from, or provide helium to.\u201d<\/p>\n<p>There were several other simulated malfunctions during, and shortly after, powered flight, including a simulated fuel leak on APU number three, a slow helium leak on the left OMS engine, and a partial freon loop blockage. Despite all of the malfunctions, the teams were still able to fly the orbiter to a nominal MECO and get set up for an OMS-2 burn. The run was ended about 25 minutes after simulated liftoff.<\/p>\n<p>Run #2:<\/p>\n<p>Shortly after liftoff on the second run, the gaseous hydrogen Flow Control Valves (FCVs) for the center and right main engines failed closed. The Booster Officer asked to call up to the crew to take the LH2 ullage pressure switch to the open position. Less than a minute after liftoff, the Booster officer reported \u201cno joy\u201d on the switch throw, requesting the crew take the main engine limit switch to \u201cinhibit \/ hard enable.\u201d The crew was also told they would need to begin to manually throttle the engines later in the ascent.<\/p>\n<p>\u201cEach engine provides GH2 back into the hydrogen tank on the ET to keep it pressurized as we\u2019re consuming propellant out of it,\u201d Mr Lunney explained. \u201cSo we want to regulate how we flow the gas back into the tank, so the pressure stays where it should, and the FCV is what does that for us.<\/p>\n<p><img fetchpriority=\"high\" decoding=\"async\" class=\"alignleft size-full wp-image-12494\" title=\"A3\" src=\"https:\/\/www.nasaspaceflight.com\/wp-content\/uploads\/2009\/11\/A39.jpg\" alt=\"A3\" width=\"310\" height=\"250\">\u201cWe have three Flow Control Valves \u2013 one on each engine \u2013 and they respond opened\/closed to what the ullage pressure in the tank is telling them. If it gets too low it says open, if it gets too high it says closed \u2013 and when I say \u2018open\u2019 and \u2018closed,\u2019 it\u2019s not fully open or fully closed, it\u2019s about 30 percent open when it\u2019s \u2018closed\u2019 and then we take it to open I think it\u2019s 60 or 70 percent open, I can\u2019t remember the exact number. Also, the valve is shimmed, so it\u2019s partially open\/partially closed when we say \u2018open\u2019 and \u2018closed.\u2019<\/p>\n<p>\u201cIn this case, two of them failed closed \u2013 and we have analysis that shows us that with one full open, we can throttle back and maintain the inlet conditions at the main engines at sufficient head pressure. If you think of the turbopumps that are spinning, you have pressure sitting on top of them; if you don\u2019t have pressure sitting on top of them, those pumps will overspeed, come apart, which is bad, so we don\u2019t do that,\u201d he added, dryly.<\/p>\n<p>\u201cSo you keep pressure sitting on top of them, and we have to throttle back a little bit at a time. We have analysis that shows that by playing this game of slowing it down, slowing it down, slowing it down, we can keep engines running long enough and keep enough pressure on the pumps long enough to get to a good MECO.\u201d<\/p>\n<p>During this event, at approximately three minutes, 45 seconds mission elapsed time (MET) after the simulated liftoff, the crew throttled the engines back to 95 percent. Then around 6 minutes, 45 seconds, they throttled the engines back to 80 percent. When the propellant remaining in the External Tank was down to 2 percent, the crew throttled down the minimum power level and performed a manual MECO.<\/p>\n<p>On the engine limit switch throw, Lunney noted: \u201cWe have redlines on the engines that are running to automatic shutdown of an engine that is violating a redline. If things are going bad, you want the computer to shutdown the engine.<\/p>\n<p>\u201cThere\u2019s also a timer going and the timer is saying \u2018hey at some point you get too far and we should be shutting down the engines\u2019, so we want to disable that timer because we know we\u2019re going to go long, because we\u2019re having to throttle back, less thrust, it takes longer to get there. So we do this \u2018inhibit, hard enable\u2019 which resets that timer for me, that\u2019s all that\u2019s doing.\u201d<\/p>\n<p>In the meantime, still during first stage, the Propulsion Officer reported one of the other malfunctions \u2013 a large oxidizer leak on the left-side Reaction Control System (RCS). A few minutes later, he reported a slower leak of the right side oxidizer. Because of the leaks on both sides, the crew was asked to abort TAL a little more than six minutes after the simulated liftoff.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-full wp-image-12496\" title=\"A4\" src=\"https:\/\/www.nasaspaceflight.com\/wp-content\/uploads\/2009\/11\/A47.jpg\" alt=\"A4\" width=\"315\" height=\"211\" srcset=\"https:\/\/www.nasaspaceflight.com\/wp-content\/uploads\/2009\/11\/A47.jpg 315w, https:\/\/www.nasaspaceflight.com\/wp-content\/uploads\/2009\/11\/A47-263x175.jpg 263w\" sizes=\"(max-width: 315px) 100vw, 315px\">Lunney explained later in our interview: \u201cThe RCS is essential for attitude control during re-entry and from EI (Entry Interface) down to about a Q-bar [dynamic pressure] of 20 [pounds per square foot]. After about a Q-bar of 20, my aerosurfaces can do enough to get by to control the vehicle without the RCS. However, for that window when the aerosurfaces aren\u2019t doing any good because there\u2019s not enough air I need my RCS for attitude control.<\/p>\n<p>\u201cThe left, in this case, was a big leak \u2013 so it was mostly going to be gone. The right was a little bit slower, but if I go and rendezvous with the station, well I\u2019m going to leak out before I get there, so I won\u2019t be able to fly to the station.<\/p>\n<p>\u201cIf I wait for a rev 3 (next PLS), all that propellant may be leaked out, and I may not be able to make it around, so we go ahead and abort TAL \u2013 assuming the leak rate tells us for at least one of the systems I can get down to a Q-bar (of) 20, which is what PROP did for us. (The controller) checked his rates, recommended TAL, because he knew that even though both were leaking, the right side would support down to a Q-bar of about 20.\u201d<\/p>\n<p>In addition to these simulated problems, there were a few others issues during powered flight, including problems with one of the Inertial Measurement Units and the Flash Evaporator System. Main engine cutoff occurred at around 8 minutes and forty seconds MET, and the teams managed to get the orbiter configured for a simulated landing at Zaragoza. The run was ended at about fourteen minutes MET.<\/p>\n<p>Run #3:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-full wp-image-12495\" title=\"A2\" src=\"https:\/\/www.nasaspaceflight.com\/wp-content\/uploads\/2009\/11\/A212.jpg\" alt=\"A2\" width=\"319\" height=\"233\">Another series of simulated malfunctions occurred one on top of the other around one minute after liftoff on the third ascent run. Watching on screens at the Public Affairs Officer console, we saw the data values \u201cfreeze\u201d temporarily. This appeared to be an issue with one of the Pulse Code Modulation Master Units.<\/p>\n<p>\u201cPCMMU is the little box onboard that formats the data for downlink, so when it breaks, we don\u2019t get data,\u201d Lunney said. \u201cSo we asked them to swap to the other one so we could get data.\u201d<\/p>\n<p>A few things were happening in the same timeframe; the Booster Officer asked for AC bus sensors off around the same time as the data loss and subsequent call for the PCMMU swap, which recovered the telemetry data. Shortly after that, Booster reported that the B-side controller for the center engine was lost.<\/p>\n<p>\u201cSo in that case we\u2019re critical to AC1, which powers electronics A on that engine,\u201d Lunney said. \u201c\u2026he said \u201cAC bus sensors off\u201d \u2013 we take the action \u2013 and then he explains to us why we did that, because we lost this controller box.\u201d<\/p>\n<p>After SRB separation, the Propulsion Officer reported a slow left OMS gaseous nitrogen leak and began troubleshooting. A couple of minutes later, he reported it was an accumulator leak, just a few moments before the Booster Officer called again to report a high mixture fuel flow meter shift on the right main engine.<\/p>\n<p>\u201cThe accumulator leak is on the OMS engine, and it\u2019s independent of what\u2019s going on in the main engines, and in that case, you got the GN2 tank and you got a little bitty accumulator, and those pressurize the ball valves, to open them up and flow propellant and make the engine work,\u201d Lunney added. \u201cSo PROP was troubleshooting that leak.<\/p>\n<p>\u201cIn the other case, it\u2019s a fuel flow meter shift, it\u2019s one of many things that we can simulate that can go wrong with the main engines. The main engines are closed loop systems, they can modify the propellant flow into the engine, both the ox and the fuel, so that they get the right conditions.<\/p>\n<p>\u201cFor an OMS engine, an RCS thruster, you open the valves, you flow propellant, and what happens is going to happen \u2013 those are open loop systems. The main engines are closed loop systems, the engine monitors itself, and if there\u2019s a blockage \u2013 if there\u2019s something going on where it needs a little extra flow of ox or a little extra flow of fuel, or a little less of each of those \u2013 it can tweak that mixture ratio in real-time.<\/p>\n<p>\u201cWhat happened in this case is, the fuel flow meter is what is used to monitor how much fuel is flowing in. The instrumentation became biased, and you know that because there\u2019s two of them, and it takes an average of the two. So (for the high mixture case) it\u2019s going to flow in too much ox [liquid oxygen], not enough fuel [liquid hydrogen], and then we\u2019re going to consume our ox that much quicker.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-full wp-image-12497\" title=\"A5\" src=\"https:\/\/www.nasaspaceflight.com\/wp-content\/uploads\/2009\/11\/A54.jpg\" alt=\"A5\" width=\"295\" height=\"215\">\u201cAlso, I think in that case we had a LOX low-level cutoff, also, because that fuel flow meter was lying to us, we pulled more LOX than we should have into the engine.\u201d [There was a 30 foot-per-second underspeed at MECO.] \u201cIt was within operating conditions that the engine could operate in \u2013 none of the other redlines, temperatures, pressures were violated \u2013 but it was flowing more LOX than we wanted out of the tanks.<\/p>\n<p>\u201cAlso, I think in that case, later we lost the controller associated with the bad fuel flow meter and we went back to a good engine, it rebalanced back to a good flow condition.\u201d<\/p>\n<p>The Flight Dynamics Officer was modeling the performance for the fuel flow meter shift when the loss of AC1 system one caused the remaining good controller on the center main engine to quit, which caused the engine to shut down a little after six minutes MET. Shortly after calls about the engine shutdown, the Electrical Generation and Illumination (EGIL) Officer reported a three-phase motor stop on AC1.<\/p>\n<p>\u201cWhen the AC system has a short \u2013 which is what that case was \u2013 there\u2019s three phases, A, B, and C, and they\u2019re all out of phase by 120 degrees,\u201d Mr Lunney explained. \u201cWe had a phase-to-phase short between the phases and we didn\u2019t know which one was causing the problem. But when that happens, the voltage drops on the AC system, and when the [main engine] controller sees the voltage drop, it dies, the controller just flat cuts off.<\/p>\n<p>\u201cIt\u2019s pretty sensitive to AC changes, so we lost the controller and that\u2019s what happened there with the main engines. Going back to the AC, dealing with the problem there, we have the capability to drop one phase, and then EGIL can look and see if the short is still present. In this case, we went through all three phases and the short was still present in all cases. So this was the case where we had to just drop the entire AC bus.\u201d<\/p>\n<p>A little later, before main engine cutoff, the EGIL Officer noted steps in the Ascent Pocket Checklist that the crew were required to carry out within a few minutes after MECO.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-full wp-image-12498\" title=\"A8\" src=\"https:\/\/www.nasaspaceflight.com\/wp-content\/uploads\/2009\/11\/A81.jpg\" alt=\"A8\" width=\"300\" height=\"200\" srcset=\"https:\/\/www.nasaspaceflight.com\/wp-content\/uploads\/2009\/11\/A81.jpg 300w, https:\/\/www.nasaspaceflight.com\/wp-content\/uploads\/2009\/11\/A81-263x175.jpg 263w\" sizes=\"(max-width: 300px) 100vw, 300px\">\u201cThe fuel cells have coolant flowing through them, driven by a pump,\u201d Lunney added. \u201cWith the AC system down, the pump stops, so fuel cell 1 is dependent on AC1, fuel cell 2 is dependent on AC2, fuel cell 3 on AC3. So with the pump stopped, it\u2019s basically not getting any cooling, and when it\u2019s not getting cooling, it\u2019s going to heat up and eventually overheat.<\/p>\n<p>\u201cAfter about nine minutes, our analysis says it\u2019s going to overheat, so we want to shut it down within nine minutes, and those were the numbers she (EGIL Officer) was quoting. That was probably three, four minutes prior to MECO, I can let it run through MECO, and I can get it post-MECO.\u201d<\/p>\n<p>The simulated MECO occurred at about nine minutes, twenty seconds after liftoff, at which time the low-level cutoff was reported by the Booster Officer. While the post-MECO work was going on, the Electrical, Environmental and Consumables Engineer (EECOM) reported a cryo oxygen leak at about fifteen minutes MET and asked to take the O2 Tank 2 manifold valve to close to try to isolate the leak. However, that didn\u2019t work \u2013 as Lunney explains:<\/p>\n<p>\u201cWell, remember the sim people are very mean, they don\u2019t like us \u2013 no, I\u2019m kidding \u2013 they want to make it hard on us. We had an O2 tank leak and you want to close the valve to it. They failed that valve, which we\u2019ve actually seen from time to time on these cryogenic valves. It\u2019s a momentary switch, it\u2019s a switch that you push open and when you let go it flops back to the middle state, if you push close and vice versa.<\/p>\n<p>\u201cSo, what we\u2019ve learned is that if you continuously energize \u2013 hold in the open position or hold in the close position \u2013 then you can sometimes get lucky and the valve will defrost or start working \u2013 however you want to look at that. They put a retention device, which is just a little widget we created, to shove in there and hold the position.\u201d<\/p>\n<p>Due to an impending loss of cryo, the team began evaluating whether they would need to do an Abort Once Around (AOA) or whether it was a \u201cnext PLS\u201d (Primary Landing Site) situation that would buy some additional evaluation time. At around twenty-two minutes MET, fuel cell 3 was shutdown to try to isolate the cryo oxygen leak, which was successful. By the time that the sim run was ended about five minutes later, the flight control team had worked out a way to bring back fuel cell 1 and possibly not just get out of the AOA, but get out of the next PLS case.<\/p>\n<p>\u201cThe way that one could have played out depends on where the failures are and how things work, if we\u2019re able to basically use jumper cables, literally like a car, jumping a car,\u201d Mr Lunney explained. \u201cFuel cell 1 was the one we were trying to get back, essentially the AC system is what we\u2019re trying do. AC transfer cables is what we\u2019re using [to get power from AC system 3 to AC system 1] \u2013 so I\u2019m able to keep a couple of fuel cells running and if I can look at that and see it\u2019s stable and healthy, I can get out of the PLS case.<\/p>\n<p>\u201cWe want to get to OMS-2 safely, get on-orbit, and then we can get time to talk about it. If you can re-route the power and get that second fuel cell running, then you have two viable, independent fuel cells. So at that point, you\u2019re good to go for probably a nominal end-of-mission. I think, in that case though, I\u2019d have to check flight rules.<\/p>\n<p>\u201cWe\u2019d have probably one or two meetings \u2013 or ten \u2013 to discuss it, and make sure we all understood what we had, why it happened, and that we were comfortable continuing. We might still cut the mission short in that case, it just depends on what all the data showed in that particular case.\u201d<\/p>\n<p>Run #4:<\/p>\n<p>In comparison to the first three runs, the fourth and final run was relatively free of simulated malfunctions.<\/p>\n<p>\u201cThat was a nominal run, which is typical for our last ascent or last run,\u201d Lunney noted. \u201cWe try and have it fairly nominal, so we can kind of look at the timeline, do the nominal calls as a matter of practice, so we all remember what the nominal stuff is supposed to look like \u2013 because that\u2019s how we expect to operate on the real day.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The STS-129 ascent\/entry flight control team and flight crew concluded the fourth and final integrated ascent simulation at the Johnson Space Center (JSC) on Tuesday. NASASpaceflight.com\u2019s Philip Sloss was given a unique insight by observing the sim runs conducted by Flight Director Bryan Lunney and his team. STS-129 Sim Runs: In preparation for the upcoming [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":"","_links_to":"","_links_to_target":""},"categories":[2],"tags":[9199],"class_list":["post-40940","post","type-post","status-publish","format-standard","hentry","category-news","tag-sts-129"],"acf":[],"_links":{"self":[{"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/posts\/40940"}],"collection":[{"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/comments?post=40940"}],"version-history":[{"count":0,"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/posts\/40940\/revisions"}],"wp:attachment":[{"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/media?parent=40940"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/categories?post=40940"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/tags?post=40940"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}