While many of the other measures documented in this guide are associated with cost and schedule aspects of the program, Technical Performance Measures (TPMs) are usually considered in the domain of Systems Engineering. The NDIA Systems Engineering Division published a study entitled “System Development Performance Measurement” in October 2011 that contained recommendations for key information needs, indicators, and measures that could be used in the acquisition and management of defense programs from the Systems Engineering perspective. TPMs, as well as, other system engineering oriented predictive measures are cited in that report.(15) 7.1 Technical Performance Measure Compliance
Metric Definition
Technical Performance Measurement (TPM) involves predicting the future values of a key technical performance parameter of the higher-level end product under
development based on current assessments of products lower in the system structure.
Continuous verification of actual versus anticipated achievement for selected technical parameters confirms progress and identifies variances that might jeopardize meeting a higher-level end product requirement. Assessed values falling outside established tolerances indicate the need for management attention and corrective action.
A well-thought-out TPM program provides early warning of technical problems, supports assessments of the extent to which operational requirements will be met, and assesses the impacts of proposed changes made to lower-level elements in the system hierarchy on system performance.
A good TPM has the attributes of:
• Traceability – The traceability of the Technical Requirements to WBS to Technical Performance Measures to EVM Control Accounts. In the Control Account, a description of the TPM and its allowed range of values for the Period of Performance of that Control Account should be defined.
• Impact – How much of the WBS work, and therefore how much budget (BCWS), is covered by the TPM(s)? What is the impact of a non-compliant TPM at any specific stage of the program?
• TPM Banding/Sensitivity – What banding (R/Y/G) and sensitivity (EV impact) should be used for each TPM?
• Technical Readiness Level – What’s the state of the technology supporting the requirement(s) for which TPM is a metric?
Calculations
The TPMs are calculated using the attributes listed above for the system as a whole and for critical components of that system. This calculation can be for any Key
Performance Parameter that will jeopardize the success of the program if it is outside
shows a notional example of how to plot the progress of the TPM against the planned value of that Key Performance Parameter.
Figure 35. Plotting the Progress of TPMs against KPPs
Output/Threshold
The Technical Performance Measure of a key deliverable is typically defined during the requirements definition phase of the program and continually assessed for compliance at every stage of the program.
The TPM is used to:
• Assess the design process
• Define compliance to the performance requirements
• Identify technical risk.
The TPMs are limited to critical thresholds for program elements that are critical to the customer’s success and critical to technical compliance.
Candidate for Technical Performance Measures include:
• Physical size and stability – Useful life, weight, volumetric capacity
• Functional correctness – Accuracy, power performance
• All the “…ilities” – Supportability, maintainability, dependability, reliability, operability
Predictive Information
For any Key Performance Parameter that is not with the allowed limits at a specific time in the program, more work and more budgets will be needed to take corrective action.
As a result, the EVM metrics must be assessed to confirm that they reflect this out-of-compliance condition for the TPM.
With this assessment of the TPM compliance, a recovery plan can be developed and the impact on the CPI/SPI of the program can be assessed.
An example of using the TPM to make EVM adjustments is:
In the example chart shown in Figure 36, the Cost Variance and Schedule Variance are adjusted with the compliance values of the Technical Performance Measures shown in the first column, in this case WBS element 1.1. The example shows the aircraft weight as the system TPM and the composite elements of that weight as individual TPMs:
airframe, aircraft, weapons, cooling system, displays/wiring, navigation system, and radar. (This information comes from a paper by Cmdr. Nicholas Pisano.)
Each element TPM is assigned a percentage contribution, totaling 100%. The budget impacted by the TPM is assigned to each TPM as well. The TPM’s technical compliance is then used to calculate a “TPM Informed” BCWP for that WBS element.
This BCWP is not the one reported in the Contractors Performance Report (CPR), but it is used to inform the program decision makers of the confidence in the CPR values.
In the example shown in Figure 36, the result is a favorable measure of the weight against the planned weight and its impact on BCWP.
Figure 36. Using the TPM to make EVM adjustments
Caveats/Limitations/Notes
• Developing the TPM starts after requirements definition based on the Measures of Effectiveness, and the Measures of Performance for the resulting system or product. The System Engineering Management Plan (SEMP) and the resulting system engineering architectural documents are used to further define the TPMs and to set threshold values.
objectives evaluated in the operational environment under a specific set of condition.
o The Measures of Performance characterize the physical or functional attributes relating to the system operation, measured or estimated under specific conditions.
o Key Performance Parameters represent the capabilities and
characteristics so significant that failure to meet them can be cause for reevaluation, reassessing, or termination of the program.
o The Technical Performance Parameters are attributes that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal.
Each of these must be determined before TPM can inform the EVM values.
Weighting and assigning impacts for each TPM also is a Systems Engineering process.