Alignment with EA Maturity Framework

At this point, we are further interested in the alignment to the periodic EA maturity assessments. Periodic assessments for EA at our corporate partner were conducted using the IT Capability Maturity Framework (IT-CMF) (Curley 2009). This framework consists of 36 critical capabilities organized into four macro capabilities. Enterprise Architecture Management (EAM) is thereby a critical capability (in the macro capability Managing the IT Capability) which provides the necessary models and practices for defining, planning, and managing the business and IT capabilities. EAM organizes its Capability Building Blocks (CBBs) into the categories Practices, Planning, and People. Of main concern for our approach is the CBB Architecture Value which revolves around defining, measuring and communicating the value/impact of architectures and architecture practices. Each CBB has a set of practices, outcomes, and metrics that determine what to do to achieve a certain level of maturity and how to measure it. During our project, we closely work together with the Innovation Value Institute (IVI) that issues the IT-CMF. In case of our corporate partner, several EA maturity assessments were conducted throughout the years and it is therefore possible to make year-on-year comparisons. Our research project was partly motivated due to shortcomings in EA maturity, especially in the Architecture Value CBB.

7.4. Preliminary Results

During Construction of the EABV AM, evaluation constitutes a crucial part to determine whether our solution fulfilled its objectives and what measures can be taken to ameliorate eventual shortcomings. Preliminary results represent the output of a first evaluation, a prototype evaluation in our case. Thus, we present our MAID evaluation results based on the chosen MAID criteria that are aligned with DSR criteria.

7.4.1. MAID Results

We now present the results for our MAID evaluation based on 23 criteria. Our evaluation is observational and descriptive in a sense that a case study was conducted to investigate the artefacts in the organizational environment while backing up arguments with informed usage of the knowledge base (Hevner et al. 2004). All quality-audited criteria ratings can be found in the Appendix D which marks an important step to achieve trustworthiness of our evaluation As already pointed out in 6.5.5, we focused on criteria that address completeness due to the fact

that no prior assessment measure was employed and therefore our approach had to fill a huge gap in that respect based on the requirements and stakeholder needs. Regarding ratings, we achieved on overall score of Adequate. The summary of scores is illustrated in Table 7-4.

DSR Criteria Avg. Score Feasibility Adequate

Utility Very adequate

Fit with organization Slightly inadequate

Understandability Adequate

Usability Slightly inadequate

Completeness Adequate

Consistency Adequate

Accuracy Adequate

Total Score Adequate

Table 7-4: DSR criteria MAID rating score summary

These ratings and several interviews resulted in our key findings which we summarize in Table 7-5. We include strengths and weaknesses, but also include recommendations. Based on key findings and recommendations, we can evolve our approach. MAID results have been diffused to relevant stakeholders to decide the next steps to EABV AM evolution.

Key Finding Strengths Weaknesses Recommendation

Improve integration

The strength of our approach is that from a conceptual standpoint there is no inhibitor regarding integration. It is well suited to be

employed with virtually any technology.

Since our implementation is limited to tools that are not well integrated into the application landscape, yet widely used, we experience shortcomings on a technical level of integration. Switch to web-based applications supported by current technologies employed. A detailed recommendation will be made available for our corporate partner in the form of an

implementation guideline suited to the system and application landscape.

Improve reporting

Our reporting so far is concise and straight to the point. It leaves a lot of flexibility on how to communicate EABV based on basic set of attributes.

A weakness is that not the entire concept of EABV is normalized from a data modelling perspective. At this stage of development, this was not considered as requirement although it would put more detail and clarification into

reporting.

Reporting must be done frequently to gain more insights in how to best satisfy the information needs. Information products need to be more accurately adapted to satisfy different stakeholder needs.

Increase adoption

The strength of adoption so far is that our approach was well received from all stakeholder groups and that support is extended.

The major weakness regarding adoption is still that it is not implemented on a large scale for all projects.

Steadily increase the application of the approach to different projects and therefore increase adoption. In the future, it should be employed for tactical and strategic level

assessments.

Improve tool support

The strength of tool support currently is that employed tools are available to every stakeholder and all relevant concepts have been implemented.

Although tools are widely available, the actual support for implementing our approach is limited due to the fact that is not networked in addition to the shortcomings from an integration perspective.

Evaluate current BI solutions for suitable EABV AM

implementation. Regarding the database, switching to a different technology is

recommended.

Table 7-5: MAID key findings and recommendations

7.5. Contribution

In this Section, we highlight and summarise our contribution to both the academia and the industry in accordance to our research context outline in Section2.6. Thereby, we examine the theoretical contributions of our approach. This forms a part of the evaluation which we think is necessary in order to give our artefacts the necessary academic foundation, context, and justification. Thereafter, we elaborate on the practical contributions describing the impact on decision making and the impact on EA maturity. We outline all relevant theoretical and practical contributions in the following Sections.

7.5.1. Theoretical Contribution

With arguing that EA is a dynamic capability, we provide a strong underlying theoretical base for further investigation and the design of artefacts. Putting assets as a perspective in the EA BSC as fundamental points of interest for performance contribution and assessments thereof, we provide a transparent model (EABV M) to integrate, configure, gain, and release those assets or resources respectively. Moreover, we outline our findings regarding challenges during the alignment of periodic and continuous EA assessments according to our research context (cf. Sec.2.6) which therefore supports future the practice of conducting such assessments. Being aware of a problem is the first step to avoid or solve it. Thereafter, we are able to derive principles that form a good practice to design a method to assess EABV (cf. Sec. 5.1.3 and Appendix E). Furthermore, we contribute to DSR by presenting a research process centred on artefacts which is therefore coined as artefact build cycle (ABC) (cf. Sec.2.2). We further

elucidate involved roles and responsibilities for this research endeavour (cf. Sec.2.4). Another interesting contribution amounts to shaping research and the corresponding output depending on various research criteria. For this purpose, we introduce a DSR profile to indicate in which general direction research is carried out (cf. Sec.2.5).

7.5.2. Practical Contribution

Having described what our approach contributed to the theoretical knowledge base, we now take a closer look what we administer to practice.

7.5.2.1. Impact on Decision Making

Data-driven decision making is improving productivity by 5-6% according to a study of 179 companies employing this approach (Brynjolfsson et al. 2011). With our approach, we facilitate decision making improvements which impact not only productivity, but in addition time-to- market, and business agility. Productivity improvements thereby manifest themselves in faster availability of infrastructure for office and enterprise applications due to better resource management. According to our EA stakeholders, decision making is alleviated and decision makers display more confidence in making decisions. The main benefit for decision makers is represented by the timeliness of decisions resulting in greatly reduced service delivery time which in turn increases productivity. Productivity increase is estimated on average 6-7% as we learned from SMEs at our corporate partner. We do not want to underestimate the impact of investing time to understand goals and their indented purpose, an inherent benefit of goal-driven approaches as common understanding reduces risks and greatly reduces communication time by avoiding redundant information flows. A more thorough analysis on decision making impact involving more projects is a projected objective in the future.

7.5.2.2. Impact on EA Maturity

At the time of finishing our research project, no new EA maturity assessment was conducted. Nevertheless, the unanimous opinion of various SMEs was that this approach, once fully adopted, increases the Architecture Value CBB to level 4 from 2.1. At the current prototype stage, a rating of at least 3 is expected. Thereby, the SMEs consist of stakeholders from our corporate partner as well as experts from IVI, publishers of the IT-CMF. We aim to find out the actual maturity level of this CBB during the next maturity assessment as well as the impact on the overall EAM critical capability.

7.6. Chapter Summary

This Chapter shed light on how we performed our instantiation. Firstly, we described instantiation details for each of the IT artefacts. Then, we went through the phases of the MACE scheme which encompasses the steps of the EABV AP. Furthermore, we elaborated on the alignment with the maturity framework IT-CMF. We argued that insights from both approaches are valuable to each other and result in an overall increase of framework quality. Another important part of this Chapter are the preliminary results of our MAID evaluation. We found out, that we need to concentrate mostly on integration and adoption in order to improve the EABV AM. We concluded the Chapter with elaborating on the achieved theoretical and practical contributions. Theoretical contributions comprise identified challenges for EABV assessments and a set of principles for constructing, operating, and improving an EABV assessment approach, and setting them into an adequate theoretical context by exploiting dynamic capabilities. Moreover, we contributed to DSR by introducing the ABC and DSR profiles. Regarding practical contributions, we elaborated on the impact on decision making and EA maturity. Both impacts yield positive results.