IMPROVING COMPLEXITY MEASURE
USING NUD CRITERIA FOR SYSTEM
SOLVABILITY
MYEONG-SEOB CHO
Naval R&D Center, Samsung Thales Co., Gumi, Korea
[email protected] YOUNG-WON PARK
Department of Systems Engineering, Ajou University, Suwon, Korea
[email protected] PEOM PARK
Department of Systems Engineering, Ajou University, Suwon, Korea
Abstract :
Due to the unprecedented integrations and applications of information and communication technologies, the majority of systems reveal themselves as complex systems. System complexity should be properly determined and managed in order to overcome the problems and uncertainty arising from it. Determining system complexity during the early phase of its life cycle can be useful in predicting project characteristics including the development cost, schedule, risk and development process. General coupling complexity quantifies the number of interfaces among system elements. Because general coupling complexity doesn’t consider the design paradigm such as reuse of components and the product line architecture, it is not useful for predicting project characteristics. This paper proposes a system complexity measure in which solvability is determined using the characteristics of Newness, Uniqueness and Difficulty (NUD) of the subject system. The level of NUD for configuration items and their associated interfaces are determined and applied to the proposed measure. To validate its effectiveness, this paper demonstrates the procedure for estimating the development cost of a new project using its system complexity measures and compares the cost estimated from NUD coupling complexity with the one from general coupling complexity.
Keywords: Complexity; NUD; Solvability; Interface; Cost
1. Introduction
As the fields of information and communication technologies are integrated and applied to all industrial products, the complexity of man-made systems has been increasing exponentially. Similarly, the system requirements are growing quantitatively as well as qualitatively which result in complex system design effort. As systems requirements and the resulting systems designs have become more complex, systems engineers acknowledge system complexity as an attribute of modern systems and attempt to measure it quantitatively and qualitatively. The most common system complexity measures are size complexity and coupling complexity due to the increase in the number of components and the associated couplings among them.
In general, if a system becomes more complex, the number of tasks increases in order to resolve problems arising from the complexity, and thereby the development cost and schedule delays increase as well. However, the recent trend is to reduce the development cost by applying the product line architecture or the family of systems engineering and by reusing the standardized system components. In these cases, the development cost is not always proportional to the existing system complexity measures which consider only the size and the coupling.
solvability attributes. Solvability is defined by adopting Newness, Uniqueness and Difficulty (NUD) characteristics which has been previously applied to the methodology of Design For Six Sigma (DFSS) [1]. The proposed complexity measure can be utilized during the exploration research and concept phase of a system life cycle by means of systems architecting.
This paper is organized as follows. Section two describes the definition of system complexity and the previous studies of system complexity measures. Section three outlines the concept of systems architecting and a systems hierarchy and introduces system configuration items. Section four discusses the proposed system complexity measure using the artifacts of systems architecting and the level of NUD. Section five illustrates the procedure for estimating the development cost of a new project using the proposed complexity measure through the example cases of naval combat management systems (CMS) and compares the proposed system complexity measure with the general coupling complexity. Section six summarizes the contents of this paper and describes areas for further work.
2. System Complexity
Trends indicate that systems of all types are continuing to become more complex in their composition, capabilities, and interfaces [2] due to the growing desire for more beneficial emergent properties. From a generic perspective, complexity can be defined as a quality of an object with many interwoven elements and attributes which make the whole object difficult to understand in a collective sense [3].
In physics, a complex system is defined as a complicated system, composed of many parts, whose properties cannot be understood [4]. In the area of complexity theory, complex systems are those that have many interrelated, interconnected, or interwoven elements and interfaces [5]. Complex systems display emergent behavior that arises from actions and interactions of components [6]. In contrast to complex systems, simple systems consist of a small number of components whose interactions can be neglected [7].
Most systems engineers acknowledge complexity as an attribute of modern systems. However, there are different interpretations of complexity. Some differentiate between internal and external complexity [8]. Others view complexity through the lens of system size [9] or system integration [10]. Others differentiate between detail and dynamic complexity [10], [11]. While detail complexity refers to how ‘complicated’ a system is, or how ‘rich in structure’, dynamic complexity refers to how ‘complex’ it is, or how ‘rich in diversity’ [12]. Complexity features can be also classified into several groups such as structural complexity related to static geometrical structures, dynamic complexity related to temporal behavior, functional complexity related to new functions in the system, organizational complexity related to the increased organizational depth of the system, and design complexity related to the process of creating the system [12].
System complexity can be measured in various ways. Existing approaches to measuring system complexity can be categorized into two different types: information approach and traditional design approach. Information approach measures uncertainty by relating the current information contents with the amount of information required to satisfy the system requirement. Information approach entails the practical difficulties in researching and calculating the probability of a certain task during the development phase of the system [13] and relies heavily upon the experience of the designer to evaluate uncertainty [14]. Traditional design approaches measures complexity by identifying system elements and relationships among them during the development phase. The advantage of traditional design approaches is that they are easily applied to real systems and the attribute data collection is relatively easy [13]. Therefore, traditional design approaches are more applicable to measuring system complexity using artifacts of systems architecting in the early phase of a system life cycle. Table 1 provides a comparison of different complexity measures among the information approaches and the traditional design approaches. Three aspects of complexity required for complexity measures are size, coupling and solvability [14]. Size, coupling, and solvability are applicable for measuring the complexity of the problem, process, and product [15].
Table 1. Complexity Measure Comparisons (Adapted from [14])
Basis Measure Evaluated Measured
I Suh [16] PB, PD V
I El-Haik and Yang [3] PB, PD S, V I Braha and Maimon [17] PB, PD S
I Komogorov [18] PR, PB S
D Simon [19] PD S, C
D Dixon et al. [20] PR, PB C
D Pahl and Beitz [21] PD S
Basis Measure Evaluated Measured D Bashir and Thomson [23] PD S, C
D Fitzhorn [24] PR S, V
I, D Ameri et al. [14] PD S, C
Basis: information based (I), traditional design (D)
Evaluated: design process (PR), design problem (PB), design product (PD) Measured: size (S), coupling (C), solvability (V)
This paper proposes a method to determine system complexity based on the configuration items, their associated interfaces and the solvability measured in Newness, Uniqueness and Difficulty (NUD) criteria. The proposed complexity measure evaluates the design solution rather than design problem and attempts to measure complicatedness of a system. It is suitable for application in the early phase of a system life cycle. In the next chapter, systems architecting and a system hierarchy are discussed to provide the basis of the proposed method.
3. Systems Architecting
3.1. System Life cycle and Systems Architecting
Every man-made system has a life cycle, even if it is not formally defined. The role of the systems engineers encompasses the entire life cycle for the system-of-interest [25]. Although the life cycles vary according to the nature, purpose, use and prevailing circumstances of a system, there are seven generic life cycle phases: Exploration research, Concept, Development, Production, Utilization, Support and Retirement [25], [26]. Systems engineers perform systems architecting during the exploration research phase, the concept phase and the development phase, in an evolutionary manner. As the system life cycle progresses, the architecture of the system-of-interest is refined and established in greater detail.
Especially, the concept phase includes the definition of a system, its elements, the key subsystem-level concept and the architecture as well as the integration, verification, and validation planning and their process definitions. The output of the concept phase becomes the foundation of data required to proceed to the next phase, the development phase. In case there is no previous phase earlier than the development phase or there are no source data except the Request For Proposal(RFP) in writing a proposal, systems engineers who are principal members of a proposal team usually have no more than one to two weeks to perform the systems architecting task. They produce the equivalent levels of outputs as the concept phase through the systems architecting. The integrated VEE model for the concept phase and the development phase is shown in Fig. 1.
Fig. 1 Integrated VEE model for Concept phase and Development phase
The major output of the systems architecting during the concept phase is the system architecture descriptions which may vary in depth according to the characteristics, scale, stakeholders and environment of the system-of-interest and the characteristics and the period of the project.
3.2.System Hierarchy
As shown in Fig. 1, a system hierarchy is one of the artifacts of the concept phase effort. A system hierarchy is a system or an organizational representation of a partitioning relationship where the entity is separated into smaller and more manageable entities [25]. In the systems engineering domain, the leaf nodes of a system hierarchy are configuration items.
A configuration item (CI) refers to the fundamental structural unit of a configuration management system and may be hardware, software or firmware [25]. CIs have a defined functionality and a unique specification. A hardware configuration item (HWCI) is the equipment of which components are fabricated in one enclosure and which has a power supply unit whether it has redundancy or not. A desktop computer, an operator console and a LAN switch are good examples of a HWCI. A computer software configuration item (CSCI) is a software module that performs or supports more than one of the system functions. Application software and middleware are good examples of a CSCI. HWCIs can have CSCIs subordinate to them [25].
An example of a system hierarchy is shown in Fig. 2. The leaf node of the system hierarchy is a HWCI, a CSCI or other upper level element such as Prime Item (PI) which can be acquired by ‘purchasing’, ‘making’ or ‘outsourcing.’
Fig. 2 An example for the System Hierarchy
As a solution to system requirements, a simple design is preferred to a complex design in which there are many configuration items and their associated interfaces. In other words, as the number of configuration items and their associated interfaces increases, so does system complexity [28]. Based on this concept and solvability, this paper proposes a formula for measuring system complexity, which is described in the following section.
4. System Complexity Measures
In this paper, two system complexity measures of the traditional design approach are addressed. General coupling complexity is introduced based on basic axioms of system complexity. NUD coupling complexity is proposed based on basic axioms and solvability.
4.1. General Coupling Complexity
General coupling complexity of a system is established based on configuration items and their associated interfaces to peers and external systems. Before introducing the formula, there are basic axioms on system complexity as follows:
Complexity is proportional to system size.
Complexity is proportional to the number of CIs
Complexity is proportional to the number of interfaces.
Complexity is proportional to the number of external system interfaces including those with the natural environment.
Although the depth of a system hierarchy may also be an important element in accounting for complexity of a system, it can be different according to an organization, a systems engineer and project environment. Thus, this paper disregards the depth of a system hierarchy by applying the rules on the definition of CI as stated in Section three.
Based on the basic axioms stated above, general coupling complexity of a system can be formulated as:
)
(
)
(
1 1 1 1 ij e m i e m j ij e n i e n jgen
H
S
CoS
( 1 )where,
m = the number of CSCIs,
e = the number of external systems including environment,
Hij = Existence of interface from i-th HWCI (or external system) to j-th HWCI (or external system), 0 or 1, Sij = Existence of interface from i-th CSCI (or external system) to j-th CSCI (or external system), 0 or 1.
This formula measures only the number of interfaces among configuration items. As a prerequisite for applying this formula to the system, the one and only criteria should be applied in defining the HWCI and the CSCI.
4.2. NUD Coupling Complexity
Coupling complexity of a system can be calculated using only the number of interfaces among CIs. However, the exclusion of the solvability can yield misleading results in measuring system complexity precisely, in comparing system complexity with that of other systems and in estimating risk, cost and schedule concerning the system development. Thus this study adopts a “Newness, Uniqueness and Difficulty (NUD)” criteria to measure the solvability taken into account. This approach has been previously applied to Quality Function Deployment (QFD) or Design For Six Sigma (DFSS) methodology. NUD can be defined as follows [1]:
Newness: features or performance your system has not historically delivered
Uniqueness: features or performance that is distinctive or highly desired beyond others
Difficulty: features or performance that is highly desired but is quite difficult to develop and will require special efforts/investments of resources
In addition to NUD, internal complexity can be considered as an attribute of a CI and the associated interfaces. Some CIs may have a simple structure, whereas other CIs have a complex structure. The latter has low solvability.
Newness, uniqueness, difficulty and internal complexity feature for each CI and each interface is evaluated as the level of NUD. In consideration of the level of NUD, the formula measuring system complexity can be updated as follows:
)
*
*
(
)
*
*
(
1 1 1 1 ij i ij e m i e m j ij i ij e n i e n jNUD
H
LH
LH
S
LS
LS
CoS
( 2 )where,
n = the number of HWCIs,
m = the number of CSCIs,
e = the number of external systems,
Hij = Existence of interface from i-th HWCI (or external system) to j-th HWCI (or external system), 0 or 1, Sij = Existence of interface from i-th CSCI (or external system) to j-th CSCI (or external system), 0 or 1, LHi = the level of NUD for Hi,
LHij = the level of NUD for interface from Hi to Hj, LSi = the level of NUD for Si,
LSij = the level of NUD for interface from Si to Sj.
As a prerequisite for applying this formula to a system, there must be consistency in determining the level of NUD. In order to provide the consistency, NUD criteria are recommended as shown in Table 2.
Points for all attributes can be determined by a chief systems engineer or a systems architect according to the criteria shown in Table 2. Each point has a value ranging from zero to one which can be determined to one decimal place. The level of NUD for a CI or an interface equals to summation of all points, thus has a value ranging from one to five. If the level of NUD for a CI is five, it means that it is new, unique and difficult to develop and its internal complexity is so high that it should be flowed down to lower units. There are other criteria or recommendations on determining the level of NUD as follows:
If a CI or interface doesn’t exist, the level of NUD is zero.
In case of reuse, points for Newness, Uniqueness and Difficulty are all zero. Thus the level of NUD will be one or two depending on internal complexity.
If having a multiple instances for a CI, the first instance is evaluated normally and the others are treated in the same manner as in the case of reuse.
Table 2 Criteria of the level of NUD for CI and interface
Attribute Point Criteria
Basic 1 CI/Interface exists. 0 CI/Interface dosen’t exist. Newness
1 New CI/interface
⁞ -
0 Exising CI/interface. Duplicated CI/interface.
Uniqueness 1
Distinctive or special requirments. - High reliability, Failover function, QoS, High capability and performance etc.
⁞ -
0 Neither distinctive nor special. Duplicated CI/interface.
Difficulty 1
Difficult to develop/implement. High-end technology required. Not standard interface.
⁞ - 0
Not difficult to develop/implement. Commercial-Off-The-Self.
Duplicated CI/interface.
Internal Complexity
1
Internal complex structure. CI consists of many components. Interface consists of many cables or many messages.
Complex cable pin map.
⁞ -
0 Internal simple structure.
In order to calculate system complexity efficiently, a worksheet will be very helpful in reducing calculation time and effort. The example of a worksheet for these formulas is shown in Fig. 3. This is modified from Design Structure Matrix (DSM) form or System View (SV)-3 of DoDAF: Systems-Systems Matrix [27].
Fig. 3 An example of a DSM worksheet for system complexity measures
After a systems architect performs architecting of a system-of-interest, CIs and interfaces among CIs are identified. Then, the systems architect can calculate the system complexity by inputting the level of NUD for CIs and interfaces to the right cells in the DSM worksheet.
The proposed NUD system complexity measure has the following advantages:
System complexity can be calculated in the very early phase of the development life cycle.
It is easy to gather information required.
Solvability is taken into account.
Interfaces to external systems are taken into account.
5. Effectiveness of the Proposed Measure
In this section, to demonstrate and verify the applicability of the proposed system complexity measure, the system complexity of a naval combat management system (CMS) is measured. The development cost of a new system is estimated by using a regression analysis which is based on the system complexity and the development cost data of former systems. Through these example cases, the proposed NUD coupling complexity is verified to be more effective than general coupling complexity in estimating the development cost.
5.1. Complexity of an Existing Naval CMS
The naval CMS is a command and fire control system that performs planning, tactical data fusion, situation awareness and engagement control on a combatant ship. The naval CMS is an example of large, complex and software-intensive cyber physical systems.
Three existing naval CMSs are investigated in terms of the proposed system complexity measure.
The CMS-A is categorized as a minimum size CMS, thus its complexity will be a small number in a relative sense. For the CMS-A, both hardware and software configuration items and interfaces among them are shown in Fig. 4 and Fig. 5, respectively.
Fig. 4 Hardware interface for CMS-A
Fig. 5 Software interface for CMS-A
Nine HWCIs and seven external systems are shown in Fig. 4. H1 and H2 are Multi-Function Consoles on which combat system operators perform military tasks. H3 is a Remote Monitor and H4 is a network switch. H5 is a main information processor and the other HWCIs are equipments interfacing with external systems.
Complexity of the CMS-A hardware is shown in Fig. 6. Only H1 is considered to be unique, H1 and H9 are considered to be difficult to implement. Because H1 and H2 are the same kinds of CI, newness, uniqueness and difficulty of H2 are set to 0. Because E7 stands for environment, interfaces to E7 can be considered as environment conditions and will have a large number in a defense product.
Seven CSCIs and six external systems are shown in Fig. 5. S1 is a Command and Control function and S2 is a Warfare function. S3 is an Infrastructure function and the other CSCIs are interfacing functions.
Complexity of the CMS-A software is shown in Fig. 7. Only S1 and S2 are considered to be unique. S1, S2 and S3 are considered to be difficult. Because E6 is a 40-mm gun, S7 interfacing with E6 is considered to be difficult.
Fig. 7 Complexity of CMS-A software
The second existing naval CMS is a CMS-B which is categorized as a small size CMS, thus its complexity will be greater than that of the CMS-A.
Hardware configuration items and the associated interfaces among them are shown in Fig. 8. There are thirty-four HWCIs and eighteen external systems. Software configuration items and the associated interfaces among them are shown in Fig. 9. There are twenty-three CSCIs and seventeen external systems. Complexity of the CMS-B hardware and the CMS-B software can be calculated in the same manner as calculated before.
Fig. 8 Hardware interface for CMS-B
The last existing naval CMS is a CMS-C which is categorized as a medium size CMS, thus its complexity will be greater than that of the CMS-B.
Hardware configuration items and the associated interfaces among them are shown in Fig. 10. There are fifty-three HWCIs and twenty-four external systems. Software configuration items and the associated interfaces among them are shown in Fig. 11. There are thirty-two CSCIs and twenty-three external systems. Complexity of the CMS-C hardware and the CMS-C software can also be calculated in the same manner.
Fig.10 Hardware interface for CMS-C
Fig. 11 Software interface for CMS-C
The respective system complexities of the CMS-A, the CMS-B and the CMS-C are summarized in Table 3. Table 3 System Complexity of Existing CMSs
System Complexity Hardware Software Total
CMS-A CoSgen 35 33 68
CoSNUD 82.4 108.7 191.1
CMS-B CoSgen 195 122 317
CoSNUD 526.22 665.19 1191.41
CMS-C CoSgen 333 165 498
CoSNUD 859.78 849.19 1708.97
Judging from the fact that the general coupling complexity (CoSgen) only counts up the number of interfaces
among CIs, the number of hardware interfaces is more than software interfaces for all three CMSs. But, because the CMS is a kind of software-intensive system, the NUD coupling complexity (CoSNUD) of the software is
greater than that of the hardware for all three CMSs.
5.2. Cost Estimating Equation for Naval CMS
Development costs for the CMS-A, the CMS-B and the CMS-C are shown in Table 4. Development costs for each CMS were obtained by the definitive estimate. The definitive estimate is prepared from well-defined engineering data and has an accuracy of five percent [29]. Four percent compound interest was applied to make up for the time gaps between projects. Each development cost includes labor, materials and support costs.
Table 4 Cost of Existing CMSs
System Kick-off year Cost (M$) Cost in 2012(M$)*
CMS-A 2009 2.61 2.94
CMS-B 2004 27.83 38.09
CMS-C 2007 53.33 64.88
* It's compounded at a yearly rate of 4%
The cost estimating equation (CEE) for the naval CMS was obtained by using Minitab software. A polynomial equation was assumed in regression analysis. The point (0, 0) was used additionally as a reference point. The CEE and a resulting graph derived from general coupling complexity are shown in Fig. 12. The CEE and a resulting graph derived from NUD coupling complexity are shown in Fig. 13.
Fig. 12 Cost estimating equation and graph derived from CoSgen
Fig. 13 Cost estimating equation and graph derived from CoSNUD
5.3. System Complexity and Cost of New Naval CMS
Fig. 14 Hardware interface for CMS-D
Fig. 15 Software interface for CMS-D
Complexity of the CMS-D hardware and the CMS-D software can also be calculated in the same manner and summarized in Table 5. While general coupling complexity of the CMS-D is very similar to that of the CMS-C, NUD coupling complexity of the CMS-D is even smaller than that of the CMS-C due to high reusability.
Table 5 System Complexity of CMS-D
CMS-D Hardware Software Total
CoSgen 336 149 485
CoSNUD 458.66 252.26 710.92
The development cost of the CMS-D can be estimated by substituting its system complexity for a variable in the equation shown in Fig. 12 and Fig. 13, respectively. The results are summarized in Table 6. The cost estimate calculated using CEE derived from general coupling complexity was 63.09 million dollar. The cost estimate calculated using CEE derived from NUD coupling complexity was 18.39 million dollar. Afterwards, the definitive estimate was 15.65 million dollar.
Table 6 Development Cost of CMS-D
Item CEEgen CEENUD
Estimate by CEE 63.09 M$ 18.39 M$ Definitive Estimate 15.65 M$
Error (M$) 47.44 2.74
Error (%) 303% 17.51%
If more samples are gathered, the equation will be more exact in a regression analysis. By adding the definitive estimate of the CMS-D, the CEE derived from NUD coupling complexity can be updated as shown in Fig 16.
Fig. 16 Updated cost estimating equation and graph
From Fig. 13 and Fig. 16, the impact of the system complexity on the development cost is well noticed. As the system complexity increases, the more expensive the development cost is likely to be. With these relationships, systems engineers can roughly estimate the development cost in the early life cycle phase.
6. Conclusions
In this paper, we have proposed the system complexity measure reflecting the solvability of a system development. The solvability is evaluated by adopting Newness, Uniqueness and Difficulty (NUD) criteria which has been previously applied to Design For Six Sigma (DFSS) methodology. The NUD system complexity measure considers newness, uniqueness, difficulty and internal complexity of CIs and the associated interfaces to model the solvability.
Compared to the existing system complexity measures, the NUD system complexity measure can be calculated in the very early phase of the system development life cycle and take interfaces to external systems into account. It is also easy to gather information required and calculate with a DSM worksheet.
To demonstrate and verify the applicability of the proposed system complexity measure, general coupling complexity and NUD coupling complexity were compared through example cases of naval combat management systems. As a result, NUD coupling complexity was verified to be more effective than general coupling complexity in estimating the development cost.
As a future extension to this work, the relationship between the system complexity and the development schedule will be investigated using the same example cases.
References
[1] C.M. Creveling, J.L. Slutsky and D. Antis, Jr., Design for six sigma in technology and product development, 1st Ed, Prentice Hall PTR, New Jersey, 2003, 123--125.
[2] INCOSE, Systems Engineering Vision 2020, INCOSE-TP-2004-004-02, September, 2007, 11--12.
[3] B. El-Haik and K. Yang, The components of complexity in engineering design, IIE Transactions, 1999, 31(10): 925--934. [4] L. Peliti and A. Vulpiani, Measures of complexity, Proceedings of the Conference, Springer-Verlag, Berlin, 1987.
[5] J.C. Domercant and D.N. Mavris, Measuring the architectural complexity of military Systems-of-Systems, IEEE Aerospace Conference, 2011, 1--16.
[6] S.A. Sheard and A. Mostashari, Principles of complex systems for systems engineering, the Journal of Systems Engineering, 2009, 12(4): 295--311.
[7] W. Kinsner, Complexity and its measures in cognitive and other complex systems, the 7th IEEE International Conference on Cognitive Informatics, 2008, 13--29.
[8] B.S. Blanchard and W.J. Fabrycky, Systems Engineering and Analysis, 4th Ed, Prentice Hall, 2006, 6--13. [9] R. Valerdi, Academic COSYSMO User Manual, MIT Lean Aerospace Initiative Consortium, 2006, 4--8.
[10] C.N. Calvano and P. John, Systems engineering in an age of complexity, the Journal of Systems Engineering, 2004, 7(1): 25--34. [11] P.M. Senge, The fifth discipline, Century Business, London, 1990.
[12] R.K. Agrawalla, Systems engineering to conquer complexity, the 3rd International Conference on Electronics Computer Technology, India, 2011, 154--159.
[13] Y.S. Kim, A system complexity approach for the integration of product development and production system design, Master’s thesis, Massachusetts Institute of Technology, June, 1999, 35--48.
[14] F. Ameri, J.D. Summers, G.M. Mocko and M. Porter, Engineering design complexity: an investigation of methods and measures, Research in Engineering Design, 2008, 19(2-3): 161--179.
[15] J.D. Summers and J.J. Shah, Developing measures of complexity for engineering design, ASME International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, ASME, Chicago, DETC2003/DTM-48633, 2003, 381--392.
[16] N.P. Suh, The principles of design, Oxford University Press, New York, 1990:147--171.
[17] D. Braha and O. Maimon, The measurement of a design structural and functional complexity, IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 1998, 28(4): 527--535.
[19] H. Simon, The sciences of the artificial, MIT Press, Cambridge, 1998, 183--215.
[20] J. Dixon, M. Duffey, R. Irani, K. Meunier and M. Orelup, A proposed taxonomy of mechanical design problems, Computers in engineering conference, ASME, San Francisco, 1988, 41--46.
[21] G. Pahl and W. Beitz, Engineering design: a systematic approach, Springer, New York, 1996, 234-247.
[22] M. Balazs and D. Brown, Design simplification by analogical reasoning, In from knowledge intensive CAD to knowledge intensive engineering(by U. Cugini and M. Wozny), Kluwer Academic Publishers, Norwell, 2002, 29--44.
[23] H. Bashir and V. Thomson, Models for estimating design effort and time, Design Studies, March, 2001, 22(2): 141--155.
[24] P. Fitzhorn, Engineering design is a computable function, Artificial Intelligence for Engineering, Design, Analysis and Manufacturing, 1994, 8: 35--44.
[25] INCOSE, Systems Engineering Handbook, v3.2, INCOSE-TP-2003-002-03.2, January, 2010, 9--88. [26] ISO/IEC 15288:2008 – Systems and software engineering – System life cycle process, February, 2008, 10--11. [27] DoD Architecture Framework, version 2.0, volume 2: architectural data and models, May, 2009, 138--220.
[28] J.L. Mathieson and J.D. Summers, Complexity metrics for directional node-link system representations: theory and applications, ASME International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, ASME, Montreal, DETC2010-28561, 2010, 13--24.
![Table 1. Complexity Measure Comparisons (Adapted from [14])](https://thumb-us.123doks.com/thumbv2/123dok_us/9667964.1494686/2.595.152.443.628.763/table-complexity-measure-comparisons-adapted.webp)
