{"id":12620,"date":"2020-02-28T21:57:38","date_gmt":"2020-02-28T13:57:38","guid":{"rendered":"https:\/\/wp-productionenv-bjg9h2g2bgg5b8aa.southeastasia-01.azurewebsites.net\/news\/boeing-says-thorough-testing-would-have-caught-starliner-software-problems\/"},"modified":"2020-02-28T21:57:38","modified_gmt":"2020-02-28T13:57:38","slug":"boeing-says-thorough-testing-would-have-caught-starliner-software-problems","status":"publish","type":"post","link":"https:\/\/starpath.global\/news\/boeing-says-thorough-testing-would-have-caught-starliner-software-problems\/","title":{"rendered":"Boeing says thorough testing would have caught Starliner software problems"},"content":{"rendered":"
\"\"
The crew module for the next test flight of Boeing\u2019s Starliner spacecraft is pictured inside the company\u2019s factory and processing facility at NASA\u2019s Kennedy Space Center in Florida. Credit: Boeing<\/figcaption><\/figure>\n

The program manager in charge of Boeing\u2019s Starliner crew capsule program said Friday that additional checks would have uncovered problems with the spaceship\u2019s software that plagued the craft\u2019s first unpiloted orbital test flight in December, but he pushed back against suggestions that Boeing engineers took shortcuts during ground testing.<\/p>\n

Boeing missed a pair of software errors during the Starliner\u2019s Orbital Flight Test. One prevented the spacecraft from docking with the International Space Station, and the other could have resulted in catastrophic damage to the capsule during its return to Earth.<\/p>\n

Both errors could have been caught before launch if Boeing had performed more thorough software testing on the ground, according to John Mulholland, vice president and manager of Boeing\u2019s CST-100 Starliner program.<\/p>\n

Mulholland said Boeing engineers performed testing of Starliner\u2019s software in chunks, with each test focused on a specific segment of the mission. Boeing did not perform an end-to-end test of the entire software suite, and in some cases used stand-ins, or emulators, for flight computers.<\/p>\n

\u201cWe are recommitting ourselves to the discipline needed to test and qualify our products,\u201d Mulholland said Friday in a conference call with reporters. \u201cThe Boeing team is committed to the success of the Starliner program, and we are putting in the time and the resources to move forward.\u201d<\/p>\n

The Orbital Flight Test, or OFT, in December was intended to demonstrate the Starliner\u2019s performance in space for the first time ahead of the capsule\u2019s first flight with astronauts this year. The issues that plagued the OFT mission might force Boeing and NASA to plan a second unpiloted test flight before moving on to a crewed mission.<\/p>\n

Officials have not decided whether another automated test flight might be required, or said when the Starliner might fly in space again.<\/p>\n

Boeing developed the Starliner spacecraft under contract to NASA, which is seeking to end its sole reliance on Russian Soyuz crew capsules to ferry astronauts to and from the space station. NASA awarded Boeing a $4.2 billion contract and SpaceX received a $2.6 billion deal in 2014 to complete development of the Starliner and Crew Dragon spaceships.<\/p>\n

The Crew Dragon completed a successful unpiloted test flight to the space station in March 2019, and then demonstrated the capsule\u2019s in-flight launch abort capability in January. Final preparations are underway for the first Crew Dragon flight with astronauts on-board, which could take off as soon as May.<\/p>\n

After the OFT mission exposed inadequate testing, Boeing\u2019s engineers are examining every line of Starliner software to ensure teams did not miss any other errors that went undetected during the spacecraft\u2019s December test flight.<\/p>\n

\u201cHindsight uncovered a couple of the issues, but I really don\u2019t want you or anyone to have the impression that this team tried to take shortcuts,\u201d Mulholland said. \u201cThey didn\u2019t. They did an abundance of testing, and in certain areas, obviously, we have gaps to go fill. But this is an incredibly talented and strong team.\u201d<\/p>\n

One of the software problems was immediately apparent after the Starliner\u2019s otherwise successful ascent into space Dec. 20 from Cape Canaveral aboard a United Launch Alliance Atlas 5 rocket. A mission elapsed timer on the capsule had a wrong setting, causing the spacecraft to miss a planned engine firing soon after separating from the Atlas 5\u2019s Centaur upper stage.<\/p>\n

The orbit insertion burn was required to inject the Starliner capsule into a stable orbit and begin its pursuit of the space station. After the automated sequence failed due to the on-board timer setting, ground controllers at NASA\u2019s Johnson Space Center in Houston had to uplink manual commands for the Starliner spacecraft to perform the orbit insertion burn, but the ship burned too much fuel during the process, leaving insufficient propellant to rendezvous and dock with the space station.<\/p>\n

Ground teams in Houston also encountered trouble establishing a stable communications link with the Starliner when they attempted to send commands for the orbit insertion burn, further delaying the start of the maneuver. Boeing says ground teams had issues connecting with the spacecraft on more than 30 additional occasions during the Starliner\u2019s two-day test flight.<\/p>\n

With a docking to the space station no longer possible, mission managers cut short the Starliner test flight and targeted a landing of the capsule at White Sands Space Harbor, New Mexico, on Dec. 22.<\/p>\n

After the mission timer problem, Boeing engineers reviewed other segments of the Starliner\u2019s software code to search for other problem areas. They uncovered another software error that was missed in pre-flight testing, which could have caused the Starliner\u2019s service module to slam into the craft\u2019s crew module after the ship\u2019s two elements separated just before re-entry into the atmosphere.<\/p>\n

\"\"
Boeing\u2019s Starliner spacecraft is seen after landing Dec. 22 at White Sands Space Harbor in New Mexico following the ship\u2019s first unpiloted Orbital Flight Test. Credit: NASA\/Bill Ingalls<\/figcaption><\/figure>\n

Controllers sent a software patch to the Starliner spacecraft to resolve the potential problem before it performed a deorbit burn to aim for landing in New Mexico.<\/p>\n

Mulholland said Friday that more extensive testing before the Starliner test flight would have revealed the software errors.<\/p>\n

Engineers traced the mission elapsed time problem to a coding error that caused the Starliner spacecraft retrieve the wrong time from the Atlas 5 rocket\u2019s flight control system. The Starliner set its internal clocks based on a time captured from the Atlas 5\u2019s computer hours before launch, when it should have retrieved the time from the launch vehicle in the terminal countdown.<\/p>\n

Joint software simulations between Boeing and ULA focused only on the launch sequence, when the Starliner spacecraft is attached to the Atlas 5 rocket. The simulations ended at the time of the capsule\u2019s deployment from the launcher, but testing would have revealed the timing error if the simulations continued through the time of the orbit insertion burn, which was scheduled to occur around a half-hour after liftoff.<\/p>\n

\u201cIf we had run that integrated test for a number of minutes longer, it would have uncovered the issue,\u201d Mulholland said.<\/p>\n

\u201cI think the sensitivity of this mission elapsed time was not recognized by the team and wasn\u2019t believed to be an important aspect of the mission, so ideally we would have run that (software test) through at least \u2026 the first orbital insertion burn,\u201d Mulholland said. \u201cSo from a hindsight standpoint, I think it\u2019s very easy to see what we should have done because we uncovered an error.<\/p>\n

\u201cIf we would have run the integrated test with ULA through the first orbital insertion burn timeframe, we would have seen that we would have missed the orbital insertion burn because the timing was corrupt,\u201d he said. \u201cWhen we got to that point in time, the software believed that the burn had happened many hours before, so it didn\u2019t do the burn.\u201d<\/p>\n

Mulholland said Boeing teams thought it was more logical to break the Starliner mission phases into pieces, and run software testing on each segment of the flight.<\/p>\n

\u201cWhen you do a single run from launch to docking, that\u2019s a 25-plus-hour single run in the computer,\u201d he said.<\/p>\n

\u201cThe team, at the time, decided that they would have multiple tests of different chunks of the mission,\u201d Mulholland said. \u201cIt was not a matter at all of the team consciously shortcutting, or not doing what they believed was appropriate.\u201d<\/p>\n

Before every future Starliner mission, Boeing will run longer tests in software integration labs encompassing all events from launch through docking with the space station, then from undocking through landing, according to Mulholland.<\/p>\n

Mulholland said more thorough testing could have also revealed the mis-configured software needed to safely jettison the Starliner\u2019s service module before re-entry. Without a software patch, the service module, or propulsion element, could have rammed back into the crew module after separation, damaging the ship\u2019s heat shield, or worse.<\/p>\n

A propulsion controller is responsible for coordinating thruster burns on the service module to ensure it does not recontact the crew module after separation, which occurs after the Starliner\u2019s deorbit burn and before re-entry.<\/p>\n

The service module is designed to burn up in the atmosphere, while the reusable crew module descends back to Earth protected by a heat shield.<\/p>\n

The propulsion controller on the Starliner service module is based on a design used by another program, and its software was improperly configured for the service module\u2019s disposal burn after separating from the crew module, Mulholland said. The propulsion controller had a wrong \u201cjet map,\u201d which contains information about the service module\u2019s thrusters and valves.<\/p>\n

The Starliner uses two different jet maps: One when the entire spacecraft is connected \u2014 when the crew module computers command thruster firings \u2014 and another for the disposal burn after the service module is jettisoned.<\/p>\n

\u201cThe only thing that was picked up was the one jet map for the integrated spacecraft, and we missed the jet map that was required for the service after separation,\u201d Mulholland said.<\/p>\n

He said software testing for the propulsion controller used an emulator, or a simulated component, rather than the actual controller intended to fly on the Starliner spacecraft. When Boeing ran the software simulation, the real propulsion controller was being used for test-firings of the service module thrusters in New Mexico.<\/p>\n

\u201cWhile that propulsion controller was outside supporting that other test was when they ran the qualification test of that section of the software, and because we had an incorrect emulator (and) it didn\u2019t have the correct jet mapping, that issue was not uncovered during the qualification test,\u201d Mulholland said. \u201cBecause that hardware was returned to the lab, we were able to, during the mission, re-run that sequence, identify the jet mapping issue and upload the software fix before we did the re-entry burn.\u201d<\/p>\n

One of many improvements Boeing says it is implementing is a requirement to ensure the proper hardware, avionics boxes and other components are included in future software tests.<\/p>\n

\u201cSo if it is important to have a specific piece of avionics in the lab, we\u2019ll be required to have that in there before we actually run the qualification test,\u201d Mulholland said.<\/p>\n

Another problem encountered during the Starliner test flight involved the ship\u2019s communications link with NASA\u2019s network of Tracking and Data Relay Satellites.<\/p>\n

The spacecraft had trouble locking onto the TDRS network 37 times during the two-day test flight in December, according to Mulholland. Boeing engineers have identified the cause of one of the communications interruptions, which was caused by an explainable \u201cfalse lock\u201d condition, Mulholland said.<\/p>\n

The other 36 instances of an unexpected communications outage all occurred over northern Europe and Russia, including on the Starliner\u2019s first pass over the region minutes after launching from Florida. That\u2019s when ground teams had trouble sending a command for the spacecraft to perform an orbit insertion burn after the mission elapsed timing error.<\/p>\n

An independent review team chartered to investigate the problems that cropped up on the Starliner test flight is nearing the end of its inquiry. The results of the investigation will be announced next Friday, March 6.<\/p>\n

But Mulholland said engineers are still looking into the communications issues, and a final verdict on the cause of the radio link interruptions is not expected next week.<\/p>\n

Despite the problems in flight, the Starliner spacecraft safely returned to Earth and post-landing inspections show it can be flown again, Boeing says.<\/p>\n

The ship\u2019s heat shield and parachutes performed well, as did the Starliner\u2019s life support systems, Mulholland said. Boeing was also able to test the functionality of the capsule\u2019s docking system, but teams were unable to fully check the performance of the Starliner\u2019s rendezvous and navigation sensors because the spacecraft did not dock with the space station.<\/p>\n

Boeing technicians at NASA\u2019s Kennedy Space Center in Florida are readying a second Starliner vehicle for the next test flight, whether it is a redo of the unpiloted OFT mission, or the first test flight with astronauts on-board.<\/p>\n

Email the author.<\/i><\/b><\/p>\n

Follow Stephen Clark on Twitter: @StephenClark1.<\/strong><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

The crew module for the next test flight of Boeing\u2019s Starliner spacecraft is pictured inside the company\u2019s factory and processing facility at NASA\u2019s Kennedy Space Center in Florida. Credit: Boeing The program manager in charge of Boeing\u2019s Starliner crew capsule program said Friday that additional checks would have uncovered problems with the spaceship\u2019s software that […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":"","_links_to":"","_links_to_target":""},"categories":[2],"tags":[670,524,2111,1545,190,1306],"class_list":["post-12620","post","type-post","status-publish","format-standard","hentry","category-news","tag-boeing","tag-commercial-crew","tag-cst-100-starliner-orbital-flight-test","tag-human-spaceflight","tag-nasa","tag-starliner"],"acf":[],"_links":{"self":[{"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/posts\/12620"}],"collection":[{"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/comments?post=12620"}],"version-history":[{"count":0,"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/posts\/12620\/revisions"}],"wp:attachment":[{"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/media?parent=12620"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/categories?post=12620"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/starpath.global\/blog\/wp-json\/wp\/v2\/tags?post=12620"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}