In order to evaluate how the maturity model fits to practice, a number of assessment questions have been derived from the level descriptions of each risk area. These assessment questions allow for the model to be easily applied to use cases through self-assessment or assisted through a third party.
8.2.1 Assessment Questions
The assessment questions are developed based on the descriptions of the different maturity levels. Each element described in the maturity level description is extracted and placed in individual assess- ment criteria. During an assessment, the assessor will note if an application fulfilled the statement or not. While in general each level builds upon the previous level, some elements of the descriptions should not be reproduced in higher levels. When, for example, an application is evaluated on their Ecosystem partners, level 1 dictates that the network is run internally while in level 2 and higher the network should run with partners as well. In order to support the nuances in the maturity level descriptions, the assessment questions have been assigned a criteria if they are required for the higher levels or not. Table 8.2 presents and example of the statements for one of the risk areas, a complete list of statements can be found in appendix D. The ’0’ in the required column means it is not necessary to fulfill this statement if a higher level is fulfilled.
Table 8.2:Example of statements for DLT Platform Choice risk area
Level Statement Required
1 The platform has been chosen based on the experience of the developers. 0 1 The platform has been chosen based on what is available. 0 2 A requirement analysis for the use case has been performed to analyze what
type of platform is needed.
1 3 The criteria for fit of a DLT platform to a use case have been standardized. 1 4 The platform needs for the use case are periodically examined. 1 4 Processes in place to optimize existing platform or switch platform if needed 0 5 The platform is continuously evaluated through pre-defined criteria 1 5 The platform is periodically compared to other platforms to evaluate the best
fit
8.2. EXTENDING THEMODEL 69
8.2.2 Visualizing the Model
In order increase the comprehensibility of the outcome of an assessment and to easily identify areas for improvement, a dashboard has been developed. Using the answers to the assessment statements within an assessment excel sheet as input, the dashboard automatically visualizes the levels reached for each risk area. The dashboard is presented to a DLT expert involved in multiple projects relating to DLT applications for validation.
Back-end Design
The dashboard has been designed using the business intelligence and analytics software of Tableau (Tableau Software, 2018). This software allows for the creation of dashboards from multiple data sources. Within this software the reached levels for each sub-risk area are calculated and visualized. Through an interactive dashboard the individual levels and level descriptions can be identified along with which statements relate to the levels.
There are two data sources for this visualization. One of the data sources is constant and houses the information related to the created maturity model. For this data, a MySQL 8.0 database is used with the schema design presented in figure 8.1. The SQL used to create and populate the database can be found in appendix E.
Figure 8.1:Database Schema
The second data source is an Excel sheet with the raw results of a maturity assessment. This sheet includes all the statements found in appendix D including an extra field which states if the statement is fulfilled for the application. This data is combined with the data from the database within Tableau using the statement ID as a union field.
Front-end Design
The design of the dashboard is focused on creating a quick overview of the outcome of a maturity assessment while still providing enough information to identify points for improvement. It is a two layered design with a main page shown in figure 8.2 which shows the overall maturity level of the application on the left and the individual maturity levels per risk area on the right. When hovering
70 CHAPTER8. MODELVERIFICATION ANDVALIDATION
over the bars representing the levels, the text description of the level is presented. When a bar is faded, the level has not been reached.
Figure 8.2:Main screen of Maturity Dashboard
The second layer of the dashboard zooms in on the individual risk areas. These screens can be reached by clicking on the arrows or names of the risk areas within the main screen. An example screen of the strategic risk area is shown in figure 8.3. The screen includes the maturity levels for the different sub-risk areas which fall under the strategic risk area. Additional information about the level descriptions of each level can be obtained by hovering over the individual levels.
On the bottom of the screen an additional bar shows how each statement related to the sub risk area is answered. By hovering over the bar information can be obtained about what statement needs to be fulfilled before the next level is reached.