Large satellites are expensive to deploy. For high-profile, high-capability satellites like those in the latest generation of GPS satellites – Block IIIA – development and launch costs run into the hundreds of millions of dollars each. The projected cost to deploy the first eight Block IIIA satellites was estimated at $4.4 billion. Should something go wrong with just one of these satellites during launch, the contractor – Lockheed Martin – would face a substantial monetary loss. So it’s no surprise that aerospace engineers strive to approach each step of the development process with the appropriate level of caution, methodically verifying their designs through a series of computer simulations and physical tests to reduce the risk of failure.

The Genesis probe was designed to be snagged out of the air by a helicopter after being slowed by a drogue parachute and parafoil. Instead, parachute deployment failed and the probe smashed into the ground. Source: NASA

Failing to implement and follow a rigorous regime of verification and validation can have devastating consequences. In 2004, the $264 million Genesis mission ended in a crash landing in the Utah desert. The probe’s impact smashed open the capsule, contaminating sample plates containing particles from the solar wind. Investigators determined that the cause of the failure was the backwards installation of an accelerometer that would have triggered the probe’s parachute to deploy. The faulty assembly would have been noticed and corrected if a pre-flight test had not been skipped. The test involved placing the electronics containing the accelerometer on a spinning table to physically verify that the sensor triggered under the correct forces. Instead, engineers simply compared Genesis’ design to that of Stardust, a similar probe that had undergone and passed the pre-flight test.

[Discover Accelerometers and Learn More about them on Engineering360.]

Risk Management

Every real-world project is subject to a certain amount of uncertainty and risk. Risk arises from many sources, including rushed technical designs, competitive and budgetary pressures, and environmental unpredictability. Engineers place emphasis on verification and validation steps throughout a project to manage the risk of failure with potentially catastrophic consequences. In the absence of an absolute and perfect guarantee of system functionality, verification provides the best path to ensure design robustness, assembly precision, fabricated component quality and, ultimately, mission success.

[Discover Product and Component Testing Services and Learn More about them on Engineering360.]

NASA projects undergo a series of reviews throughout their life cycle. (Click image to enlarge.)

Due to the technical complexity of the missions it undertakes, NASA has some of the most comprehensive verification and validation procedures. Through each phase of a project, from the earliest concept studies to preliminary design and technology completion to operations and sustainment to project closeout, the agency performs a number of reviews. For human space flight projects, no fewer than 12 reviews are completed through a project’s life cycle, beginning with a Mission Concept Review and continuing through a System Requirements Review, System Definition Review, Preliminary Design Review, Critical Design Review and more until project closeout and the final Decommissioning Review. Many of these reviews, in particular the System Acceptance Review that occurs during the System Assembly, Integration, Test, and Launch phase, include direct evaluation of the results of verification tests to ensure that all components and the system as a whole are performing as expected.

Catastrophe

But even the extensive review processes at NASA can fail to avert a tragedy if managers faced with mission deadlines and budgetary constraints cut corners and gloss over engineering concerns. In 2003, the Space Shuttle Columbia broke apart during re-entry, killing all seven crew members. The investigation into the disaster concluded that a piece of foam had broken off the external tank during launch and impacted the leading edge of the shuttle’s left wing, fracturing the Reinforced Carbon-Carbon (RCC) thermal tiles there and creating a space where superheated gases could enter during re-entry. This compromised the structural integrity of the wing and led to uncontrollable aerodynamic forces.

Impact tests that were part of the Space Shuttle Columbia disaster investigation indicated that RCC panels were vulnerable to strikes by foam similar to the chunks that commonly broke off the shuttle’s external tank. Source: NASA

In the investigation into the cause of the disaster, a number of contributing factors were uncovered. The Space Shuttle program’s organizational culture was found lacking, including poor risk assessment and management practices. Furthermore, one of the modeling tools – Crater, used to predict impact damage to the thermal tiles – was deemed inadequate. The accident investigation board promulgated a long list of recommendations to avoid a repeat of the tragedy on future missions. Amongst its guidance was the creation and validation of physics-based computer simulations to model the effect of debris impacts on the Thermal Protection System. In addition, the board urged NASA to perform destructive testing on RCC to better understand its characteristics.

Verify and Validate

To prevent disasters like Genesis and Columbia, the importance of verification and validation must be ingrained in an organization’s culture with assessment processes baked into every phase of a project. From the beginning of the development cycle, designs should be evaluated in physics-based computer simulations. Moreover, physical testing of scale models should begin early on to verify the thermal, electrical and structural performance of designs. These tests confirm expected behavior and inform and qualify computer models. Verification must be performed at a component level to establish the acceptability of individual parts, but also at a system level after parts have been assembled to account for all systemic effects of an integrated assembly functioning as a whole.

[Discover Modeling and Simulation Software on Engineering360.]

The SJX61-2 air-breathing scramjet engine that powered the X-51A hypersonic test aircraft is tested on the ground at NASA’s Langley Research Center in conditions simulating flight speeds of Mach 5. Source: NASA

Verification of an end product in the aerospace industry, such as a satellite or other spacecraft, involves a variety of tests that subject the article to conditions mimicking those it is expected to encounter during launch, operation in space and potential re-entry. These include aerodynamic tests in a wind tunnel or free flight; thermal tests at high and low temperatures; and vacuum tests to determine if the product functions properly in the vacuum of space, including evaluation of effects like outgassing. Acoustic and vibration tests simulate the extreme noise levels, intense vibratory forces and G-loading of a rocket launch to ensure structures remain intact and system operation remains nominal. Electrical tests determine if electrical systems and circuits can handle exposure to high and low voltages including transients and surges. Electromagnetic compatibility and interference tests confirm the ability of equipment to operate in the presence of radio frequency interference (RFI) without malfunctioning. Some evaluations, in particular thermal and vacuum tests, are run repetitively to assess the effect of cyclical variation in environmental properties on product integrity.

[Discover Shock and Vibration (Dynamics) Testing Services, Electrical and EMC Testing Services, and Nondestructive Testing (NDT) Services on Engineering360.]

Completing each of these tests is critical to ensure not only that performance is within acceptable limits, but also that unforeseen behavior is characterized. Achieving quality is an iterative process. If testing reveals any deficiencies, engineers must revisit and modify the design before carrying out another testing cycle. One reason to incorporate verification and validation into the development process from the very beginning is that it is easiest and cheapest to correct deficiencies in the early phases of a project. But comprehensive testing through every stage including the last is the best way to guarantee functionality. Crucially, management must be prepared to put the mission on hold for a thorough review if even minor defects are uncovered in the flight model before launch. The alternative could be catastrophe.

[Read about the European Space Agency’s spacecraft testing facilities, including Hydra, a giant hydraulic shaker.]