Reliability Theory and Supportability

Reliability theory and supportability were researched extensively, with majority of the findings focused on the production and sustainment life cycle phases. Few recommendations for improving supportability during system concept or design with MBSE, SysML, SPL or SOA were found. Most of the research on supportability provided lessons learned, emphasized its effects on life cycle costs, and showed how supportability functions affect the systems engineering process. A key exception was research conducted on simulation-based acquisition, which establishes an information environment in conjunction with the systems engineering artifacts, leverages data from both, and uses the environment to provide supportability inputs into concept and design. References on reliability theory tended to focus on hardware. Policies mandating sustainment as a mandatory KPP and requiring assessment of RAM throughout the program will continue to increase focus and visibility on sustainment. (King 2007; Iyer 2007)

Achieving the Chief of Naval Operations’ (CNO) vision of supportability in design requires the systems engineering methodology to incorporate supportability concepts and design throughout the acquisition process. Jones described Integrated Logistics Support (ILS) as a “disciplined and unified management of all activities necessary to produce a supportable system design and reasonable support capability to achieve a predetermined set of measurable objectives within an acceptable cost of ownership.” (Jones 2006)

CNO strategic planning has emphasized supportability as a key objective, and academia has put in place processes to address it. Yet, as identified in the May 2008 Report of the

Defense Science Board Task Force on Developmental Test & Evaluation, “early in the

study, it became obvious that the high suitability failure rates were the result of systematic changes that had been made to the acquisition process.” (DSB 2008) This report went on to identify the following findings:

ƒ The high suitability failure rates were caused by the lack of a disciplined systems engineering process, including a robust reliability growth program during system development.

ƒ RAM shortfalls are frequently identified during Developmental Testing (DT), but program constraints (schedule and funding) often preclude incorporating fixes and delaying Initial Operational Test and Evaluation (IOT&E).

ƒ Sequential workforce cuts in the last ten years had a significant adverse impact on the Navy acquisition capability.

ƒ Acquisition personnel reductions combined with loss of guidance documents and retirement of experienced senior industry and government personnel have exacerbated the adverse impact.

The same DSB Report also indicated a continuing issue with acquisition of systems that demonstrated unacceptable supportability, reporting that only 75 programs of 228 met reliability objectives. The majority of researched supportability literature, such as OA in

Naval Combat System Computing of the 21st Century by Capt. Strei and Naval Open Architecture – Overview on OA by Capt. Shannon, addressed post-development actions,

such as sustainment and planning for common hardware. (Strei 2003; Shannon 2006) A considerable amount of literature is available on managing COTS hardware with a focus on creating common hardware solutions to optimize support.

In a 30 January 2009 memo to U.S. Secretary of Defense (SECDEF) Robert Gates, Pentagon Acquisition Chief John Young offered a candid assessment of the causes of cost growth in over 40 major weapons programs, and offered the following statement:

I think this data sample highlights that some defense acquisition program performance, and costs to the taxpayer, are driven by churn in requirements and budgets which are beyond a program manager’s control in the current DoD process. (Young 2009)

When a program experiences budget or schedule challenges and constraints, a common result is the deferment of non-critical capabilities that are not required for meeting operational performance objectives, which often negatively impacts the program’s suitability performance. This provides the basis for a scenario in which supportability and sustainment efforts may be deferred to later in the acquisition process. Recent reviews of programs, such as Aegis, DDG 1000, LPD17 and LCS, all indicate deferral of supportability requirements, such as software to perform maintenance or the acquisition of technical data. Furthermore, a lack of understanding of engineering to supportability artifact relationships brought about by the loss of experienced personnel in recent years and exacerbated by a lack of requirements traceability, results in a lack of ability to articulate impacts to supportability when engineering tradeoffs are being performed.

In summary, supportability research has indicated there is a need to integrate supportability and suitability within the systems engineering process throughout the developmental phases. The basic processes exist but are resulting in systems that are not suitable due to factors external to logistics development. Only by integrating logistics within the systems engineering process during requirements development, functional allocation and other key engineering activities, can the experienced logistician expect to characterize and articulate the logistics impact and execute risk management or aversion efforts to reduce the impact later on during sustainment.

ƒ Benefits: A significant amount of information is available from resources, including Defense Acquisition University (DAU), training courses and literature, on developing logistics products and providing detailed information on logistics

processes during the different acquisition phases. Systems engineering literature and references describe the need to plan for suitability factors such as reliability, maintainability and human factors engineering. Furthermore, numerous papers are available on improving supportability through reduction of hardware variations and a movement to common hardware architecture for enterprise planning. Mandatory KPPs that include material readiness availability will require new acquisition programs to include and assess non-functional requirements, such as reliability, in their programs during early acquisition. Models for military development and analysis methods to determine the optimal product support plans are in place to support Military Standard (MIL-STD) development.

ƒ Limitations: Reliability theory and the development of tools and analysis methods lag behind the development of approaches to meet future operational capabilities. The focus of sources that were researched tended to be from an operational context, which limits the ability of logisticians and reliability engineers to have a blueprint by which to invoke key sustainment requirements. No research material was found documenting practices that integrate supportability into systems engineering artifacts or support traceability from requirements to logistics products. Nor were any processes documented that describe how logisticians assess and characterize suitability issues when schedule, technical and cost constraints impact the engineering processes.

Very little literature was found on how to integrate logistics into the systems engineering process or the role of the logistician during early design and development. The majority of researched references focused on life cycle considerations, such as technical refresh planning or managing obsolescence. Additionally, research found that with the introduction of COTS, some programs, such as the submarine Acoustic-Rapid COTS Insertion (A-RCI), have significantly changed how business is done. (Stevens 2008) Strategies included information on how to apply Performance-Based Logistics (PBLs), minimizing churn, acquiring data rights and other sustainment issues. Very little information was found on how to integrate logistics during design, with the majority of

support strategies being based on the systems engineering “VEE” model. Some of the problems cited include the following: (1) logistics efforts that do not directly correlate to the proposed engineering processes, (2) engineering data and processes that are not reviewed by the logistician, and (3) significant changes made during the acquisition process that are not assessed for their logistics impact. Efforts are ongoing in other countries to establish and link logistics requirements planning, based on a common information tool, to engineering artifacts and products, with numerous successes documented; however, the cost and infrastructure to establish these types of processes remain a concern and is an issue for most programs. Similar to a common information source for SPLs, establishing a common method or process based on an automated information system continues to challenge programs.

The majority of supportability information was found when researching engineering topics and subjects such as SysML, which showed examples of how supportability requirements can be depicted from a design perspective. Bosch explained how supportability can be a design consideration or driver. The traditional logistician often lacks the knowledge, skill and ability to function well in an engineering environment, and may not understand how requirements can be tied to logistics products and how the relationship between SPL and logistics products is defined. Conversely, most engineers are unaccustomed to designing products with supportability considerations in mind. Significant issues remain on how supportability can be integrated, characterized and correlated during concept exploration and design. Often, suitability issues found during operational testing are not the result of logistics planning shortfalls or other logistics issues, but rather on funding shortfalls and faulty engineering. When estimating life cycle costs, programs rarely include worst-case scenarios, which usually results in the estimates being far lower than actual costs.