Process 4 is depicted in Figure 3-7 and provides a more detailed explanation of the Verification and Validation process:
4.1 Define the M&S Objectives: Primary concern is with establishing the specific objectives for M&S within the context and scope of the system architecture. This step requires a detailed review of the ConOps, system requirements, functional architecture, behavioral analysis, supportability functions and/or other applicable documents and artifacts to assist in defining the objectives.
4.2 Characterize the Model: Define the critical measures of the system, establish the MOEs and identify the inputs for the models. The MOEs are used to evaluate the primary objectives for M&S with regard to critical measures. Inputs include assumptions, constraints, variables, parameters and analysis data. 4.3 Identify Tools: Research and define the tools needed to perform M&S. It is
important to ensure the appropriate tool is utilized. For example, if the model is simple, computer spreadsheets will suffice for minor to moderate calculations. If the model is more intricate, a higher-level modeling tool, such as Extend or Arena, is recommended.
4.4 Build a Parametric Diagram: After characterizing the model and identifying the necessary tools, the next step is to review the critical measures, input variables and required calculations for the system to facilitate the development of a parametric diagram. The parametric diagram is used to structure the characteristics of the model into a high-level mathematical diagram that will aid in depicting the flow of calculations for the simulation.
4.5 Build a High Level Model: When designing the high level model, it is important to start from the known requirements and the functional architecture. 4.6 Construct Main Functions: After generating the high level model, the
process begins for constructing the main functions of the model. It is helpful to take one main function of the model and separate it from its interfacing
4.7 Incorporate Supportability: Modeling is the creation of abstractions or representations of the system to predict and analyze performance, cost, schedules and risks, and to provide guidelines for systems research, development, design, manufacture and management. (Maier and Rechtin 2002) It is imperative that the model represents the supportability factors and incorporates the elements into the model from the beginning. Supportability factors are critical in predicting availability, reliability, readiness and life cycle costs associated with the system being modeled, and should be integrated throughout the requirements hierarchies and functional architecture design. It is important for the engineering and supportability team to examine the main functions of the model and employ changes at a high level before starting the detailed system modeling.
4.8 Build, Execute and Analyze the Model: The last step in the methodology may either be the least or most difficult to implement, depending on how well the upfront planning and/or design for the model has proceeded. The key is to utilize all the previously identified techniques, processes and tools to help construct the model.
3.5.1 SysML in Support of M&S
A key advantage in the use of SysML is the ability to represent data required to develop models and simulations. It is used to express constraints (equations) between value properties, provide support for engineering analysis (e.g., performance, reliability), and facilitate identification of critical performance properties. The following describes the use of SysML artifacts in relation to M&S:
Requirements: Requirements depiction, covered in section 5.2, provides the basis of M&S focus. The results of the M&S should use the requirements to develop the Measures of Performance (MOPs) and MOEs developed for the M&S performance measures. The requirements will be used to determine M&S requirements and will also be used to understand the decomposition of the model. For example, when constructing an M&S for PRA, the M&S will follow the
Probability of Kill (PK), Command and Control (C2), and Engage. This creates a
top down approach for modeling that allows for further iterations and models developed at a lower level to be traced to the top level requirements. Additionally, it allows improved understanding of the ability to tradeoff between sub functions.
Behavior: Behavior diagrams include sequence, state machine and use cases which serve as the basis for the model structure. The activity diagrams developed by the architect and team are used to model the system behavior modules and tasks. Additionally, it allows the sequence of events to be modeled and traced back to expected operation and performance.
Structure: Structure diagrams provide the basis for the performance variation used internal to the system performance such as processing time. Based on the allocation of functionality within the architecture, the processing time and functionality will be impacted. The architecture provides a basis for expectations, and when used in conjunction with an architecture tool such as CORE, allows a baseline variable to be input. The baseline variable is traceable directly to the proposed architecture.
Parametric: The parametric diagram represents constraints on system property values such as performance, reliability, and mass properties. This serves as a means to integrate the specification and design models with engineering analysis models. SysML also defines a model of value types that can have units and dimensions and probability distributions. The value types are used to type properties of blocks. The parametric diagram is a specialized variant of an internal block diagram that restricts diagram elements to represent constraint blocks, their parameters and the block properties that they bind A constraint block supports M&S by providing and defining a set of parameters and one or more constraints on the parameters. By default, these parameters are non-directional and have no notion of causality. These constraint blocks are used in a parametric diagram to constrain system properties. Constraint blocks may be used to express mathematical equations, statistical values and utility functions that could be used in trade studies to facilitate identification of critical performance properties.
Parametric diagram represents the usage of the constraints in an analysis context by the binding of constraint parameters to value properties of blocks.
3.5.2 M&S in Support of T&E
DoDI 5000.02 requires that T&E programs be structured to provide accurate, timely and essential information to decision makers for programs in all acquisition categories throughout the system life cycle. The program manager should develop a robust, integrated T&E strategy for Developmental Test and Evaluation (DT&E), Operational Test and Evaluation (OT&E) and Live Fire Test and Evaluation (LFT&E). This is done in order to validate system performance, and to ensure that the product provides measurable improvement to existing operational capabilities. However, this integrated approach should not compromise DT&E, OT&E or LFT&E objectives. The program manager, in concert with the user and test communities, is required to integrate M&S activities with government and contractor DT&E, OT&E, LFT&E, SoS interoperability and performance testing into an efficient continuum. (DAG 2004)
It should also be noted that by law (10 US Code 2399), the initial OT&E of a major defense program’s system effectiveness and suitability must not be based exclusively on computer modeling, simulation or analysis of system requirements and design specifications. While modeling can never entirely replace T&E in a live environment, it does play an important role in T&E. DoDI 5000.02 states the following:
Successful developmental test and evaluation (DT&E) to assess technical progress against critical technical parameters, early operational assessments, and, where proven capabilities exist, the use of modeling and simulation to demonstrate system/system-of-systems integration are critical during this effort. (DoD 2008)
The use of M&S has been touted in recent years primarily because of the ease of manipulation of the parameters of the system under test and the lower cost associated with M&S when compared to live testing.
Illustrates the process in which the system architecture is verified and validated through modeling and simulation.
Figure 3-7: Verification and Validation (Process 4)
4.1 Define the M&S Objectives 42. Characterize the Model 4.3 Identify Tools 4.5 Build a High Level Model 4.4 Build a Parametric Diagram 4.6 Construct Main Functions 4.7 Incorporate Supportability 4.8 Build, Execute and Analyze the
Model