Statistical engineering is a discipline that integrates engineering disciplines and statistical sciences to solve technical challenges with a quantified level of confidence. The objective is to engineer statistical methods to generate better approaches that benefit organizations through a value-added understanding of uncertainty and ambiguity to achieve research objectives. Literally, it engineers statistical sciences to generate better solutions to large, unstructured problems. (See Hoerl and Snee.) Statistical Engineering supports the effective application of statistical thinking and methods across the research, development, and procurement life cycle resulting in:
Improved specification of requirements that achieve high-level objectives; Faster understanding of system capabilities through accelerated characterization; Efficient and effective test programs that minimize test resources; and
Improved quantification of risk, thereby supporting better decision-making.
For more general information on statistical engineering, see the bibliographical Web site maintained by the American Society for Quality (ASQ), Statistics Division. Within NASA, statistical engineering has been demonstrated on numerous projects that span across NASA’s missions of exploration, science, and aeronautics. For more information on NASA applications
of statistical engineering, see the proceedings of the NASA Statistical Engineering Symposium (NSES).
While the application of statistical methods is ubiquitous in engineering, statistical engineering provides a systems perspective that transcends the application of methods in specific disciplines. It may be helpful to consider an analogy to the discipline of mechanical engineering that
develops theory and practice for practical application of fundamental sciences such as calculus, physics, and chemistry. In a similar manner, statistical engineering is an approach to build solution approaches from fundamental statistical sciences to generate impactful solutions, particularly to complex, unstructured problems.
Statistical engineering is a systems view of the knowledge sought from a research and
development effort. It is based on identifying what we need to know or learn from the physical system that is being developed. NASA’s vision and mission are to explore the unknown for the benefit of mankind and to drive advances in science, technology, and exploration to enhance knowledge. This is accomplished through research, analysis, and experimentation to observe and probe systems to find causal relationships between factors and responses. Experimentation is primarily designed to obtain knowledge, understanding, and provide new insights. In essence, NASA projects are initiated to confirm something we believe to be true or to make new
discoveries. It can be a common misconception that experiments are conducted to acquire data; rather, a statistical engineering perspective is focused on knowledge, decisions, and impact. Statistical engineering provides a framework for identifying and accommodating uncertainty and ambiguity in the formulation, planning, execution, analysis, operations, and interpretation of research and development programs. It supports risk-informed decision-making, reliability assessments, probability analyses, probabilistic risk assessment, forecasting and predictions, etc., and ensures technical excellence by efficiently achieving program objectives and quantitatively answering research questions. A statistical engineering perspective focuses on ensuring the integrity of the programmatic and scientific conclusions through the processes and methods employed. Consistently applying statistical engineering principles improves programmatic and technical decision-making, instills greater technical excellence, provides more reliable and predictable outcomes, and ensures more efficient utilization of available resources.
Statistical engineering is closely related to the crosscutting technical management processes of decision analysis (see Section 6.8) in that it provides rigor to the inputs of a formal decision analysis. It ensures that decisions are well-founded and lead to a technically defendable approach to program execution. In addition, statistical engineering is vital to successful technical risk management (see Section 6.4) that combines the probability of an undesired event with the consequence of its occurrence, thereby validating that the safety or performance requirements can be met by using the system specifications requirements defined by the decision analysis. It seeks to ensure a structured process to identify technical and programmatic risks, quantify their magnitude, and link them to programmatic and product consequences. While decision analysis and technical risk management are well-established processes, statistical engineering brings additional rigor to support and defend decision parameters by making their practice more consistent and less idiosyncratic.
Integrating the concepts of statistical engineering early in program formulation and consistently throughout the life cycle is recommended as a best practice. It can support and enhance the system design processes. At formal milestone reviews, infusing a statistical engineering perspective can assist the project in answering fundamental, plain-language questions adapted from the Heilmeier questions (see Shapiro 1994), which are outlined below.
Program and Project Definition What is the precise objective(s)? Is the objective(s) quantifiable?
What are we seeking to learn, or new knowledge sought? How will we know when we have learned it?
Is success detectable and measurable? Technical Risk Management
How well do we need to know the answer(s)?
What risk are we willing to accept if we are wrong about our conclusions? What are the consequences if we are wrong?
Planning and Execution
Do the methods support rigorous answers to the stated objectives and risk? Does the allocation of resources reflect support the objectives and risk? Are the resources justifiable and defendable?
While these questions appear straightforward, answering them quantitatively often poses a challenge for a program or project and facilitates substantive discussions that help to refine objectives. Although it is challenging, developing answers to these questions is a role for
statistical engineering and enables clear, succinct communication of the project’s success criteria throughout the organization and quantitatively supports resource justification to obtain the research objectives. Furthermore, these questions apply recursively through systems and
subsystems and throughout the project phases. For effective project leadership, it is accepted that these questions need to be addressed satisfactorily in every phase of a NASA program and project.
In summary, statistical engineering is a complimentary discipline to the systems engineering process. Statistical engineering provides a framework for integrating, linking, and sequencing statistical thinking and tools to improve project performance and for more reliably realizing research and development objectives. Strategically institutionalizing its practice will improve the Agency’s ability to achieve its mission.