Limitations

The case study describes the advantages of applying CadaML through evolving a data architecture of an industrial web application. However, we could not fully demonstrate applicability of the modeling language. The reason for this is the data layer of the use case is designed for relational databases, although CadaML supports di↵erent cloud data storage types o↵ered at the PaaS service model. This limitation requires another study where we need to model a multi-tenant data architecture as a combination of multiple cloud storage solutions.

Moreover, we did not compareCadaMLagainst other DSLs or modeling techniques due to the lack of other works that tackle multi-tenancy at the data layer. This is discussed and highlighted in more detail in Chapter 2. We also did not compare CadaML against manual code re-factoring, as a baseline due to its common use for implementation of multi-tenancy. Therefore, we need a more in-depth study to quantify the benefits of the modeling language. More concretely, we need to identify how CadaML a↵ects productivity of developers compared to the manual method, assess quality of the generated code, and evaluate usability of the language. Such evaluation requires conducting an experimental user study with real developers who are familiar with Java programming language, cloud application and data layer implementation. In order to address these limitations, we performed a controlled experiment with task analysis technique that is described in Chapter 5.

4.6

Summary

This chapter described the application of CadaML and our qualitative evaluation of the modeling language. The application demonstrates how the modeling environ- ment of the language can facilitate designing di↵erent data partitioning patterns, and generating source code that corresponds to each data partitioning option. Mean- while, the evaluation shows the feasibility of CadaML in practical settings and the correctness of multi-tenancy implementation by the generated code through a case study. The feasibility is investigated by qualitatively assessing CadaML, its model- ing environment and code generation tool. While, the adequacy of multi-tenancy implementation is assessed by applying double approach of code reviewing and unit testing. Although the evaluation outcome validates the applicability ofCadaMLin an industrial setting, and emphasizes the sufficiency of multi-tenancy implementation, we conducted a more in-depth study with real developers to further evaluate the modeling language. The study is presented in the next chapter.

Experimental Evaluation

In order to quantify benefits of CadaML, the experimental use case described in Sec- tion 4.2 is exploited. The purpose of the evaluation is to assess the productivity of developers, reliability of the generated code, and usability of CadaML. The produc- tivity is measured in terms of time required to design the data layer and completion rate of the experiment tasks. The reliability is calculated by debugging the generated code against several test cases. Meanwhile, the usability is evaluated through an interview regarding the concepts and graphical editor of CadaML. Moreover,CadaML is compared against manual code re-factoring where developers implement the data layer of the given use case for AWS.

This chapter starts with the description of the application’s data layer evolution in Section 5.1. Then, Section 5.2 presents the experimental design and evaluation methods that are adopted to assess the modeling language. The proficiency levels of participants in programming languages, cloud application/data layer implemen- tation, and modeling tools are interpreted in Section 5.3, while their allocation is presented in Section 5.4. Section 5.5 explains the modeling process of the data architecture using CadaML. The evaluation results of productivity, reliability, and exit interviews are illustrated in Sections 5.6, 5.7, and 5.8, respectively. Section 5.9 discusses the findings and limitations of the experiment, and Section 5.10 analyzes threats to validity. Finally, Section 5.11 summarizes this chapter.

5.1

Evolving the Application

During the evolution process, the data architecture of the use case is re-designed to use a combination of di↵erent cloud storage solutions. This, in turn, provides scalability, customizability and extensibility of the data layer, and reduces the costs for data storage. Figure 5.1 shows, at a high level, which data is stored in the di↵erent types of storage.

Figure 5.1: Data storage in the business process application.

during on-boarding process, and stores them as a single object in a public bucket named‘tenants’. A tenant and its configuration are modeled as separate entities to enable customization and management of each entity independently. These entities will be deployed to Amazon S3.

In the meantime, the application stores process definitions, processes, task defi- nitions and tasks entities in separate non-relational tables in Amazon DynamoDB. Storing these entities in non-relational tables simplifies the implementation of cus- tomizability and extensibility of the application. To extend these tables in a rela- tional database, additional tables are used to hold custom attributes (see Figure 4.3). Fortunately, non-relational tables allow to use multiple schemes in the same table, thus, each tenant can have its own custom attributes.

Nevertheless, non-relational databases support limited operations which restricts execution of complex queries. Therefore, for tenants who need complex analysis and management of their own custom reporting requirement, the application will provision a new relational database instance of Amazon RDS during the on-boarding process. For tenants with such requirements, the provisioning process will create necessary tables in the database. Ideally, the actual database scheme should be remained unchanged. For the experiment, the same set of entities that are used for non-relational databases but with di↵erent organizational structure are constructed.