In order to ensure that the framework is structured well and meets all the intended requirements, as well as to convince the stakeholders/ end users to implement and use the proposed framework, the framework needs to be demonstrated and validated. Therefore, it is important to establish a methodology for validating and evaluating the framework. Sargent (2005) mentioned several approaches to decide whether the framework or model is valid or not. In this study, two approaches were used in combination: (1) involving end users in determining the framework validity; and (2) the scoring model approach. According to Sargent (2005), the validation process is carried out in a conceptual framework to ensure that the framework is structured correctly without mistakes, the assumption and specifications are clear and the model or framework representation meets the intended purpose of the model. Sargent mentioned that there is no formalised guide or procedure for selecting the best approach to use in the validating framework as each modeling guide presents a set of challenges. Sengupta (2004) also noted that it is the researcher who can decide what is the best means to validate the model based on the nature of the research.
Al-Yafei, 2018
156
Sargent (2005) outlined four basic approaches to decide whether the framework or model is valid or not:
A subjective decision can be made based on the results of the various tests and evaluations conducted as part of the model development process by the model development team.
Involving end users with the development team (researchers) in determining the validity of the model/framework. Hence, the determination of validity will move from the development team to the end users. This approach is used to help in model credibility.
A third party (independent) can be used to decide whether the model is valid or not. There are two different ways to conduct the validity in this approach: either the validity is conducted concurrently with development of the model, or it is conducted after the model has been developed.
A scoring model can be used in this approach when conducting the validation process; scores are used to evaluate the model. The model/framework is considered valid if the overall scores are greater than some passing scores.
There are various techniques that can be applied to validate the model/framework; however, Sargent (2005) mentioned that the primary validation technique or method used in validating the conceptual model is the face validity method. Face validity involves asking experts to evaluate the conceptual model/framework in order to determine if it is correct and reasonable for its purpose. This sometimes involves examining the flowchart, graphical model, or model equations.
4.10.1 Selected approaches and methods
In order to get more credible results and valid framework, it has been decided to use a combination approaches. Therefore from the previous section, approaches 2 and 4 have been chosen.
Al-Yafei, 2018
157
1.The first approach: involving end users in determining the framework validity. Method: face validity; through performing case application, to check the
applicability and workability of the framework with real data and to check the framework’s logic and structure.
2.The second approach: scoring model.
Method: semi structured interviews comprise of closed ended and open ended questions to evaluate the framework and get feedback from the end users. It was decided to select the interviewees from the same sample who took part in the first interviews as part of data collection (refer to section 4.8), as they will be the end users of the developed framework. Figure 28 shows the process of the framework validation.
Al-Yafei, 2018 158 Framework Validation and Evaluation Approach selection Involving end
users/ Face validity Scoring Model
Case application with real data
Semi- structured Interviews
Open ended
questions Closed ended questions
Evaluation Validation
End
Figure 28 Framework validation process
4.10.2 Involving end users through case application
The proposed framework was tested and examined through a case application in order to check the applicability and workability of the framework with real data, as well as to check the framework’s logic and structure. The framework was used to rank and select among four alternatives. Offshore experts participated in this validation stage (for more detail refer to chapter 8).
4.10.3 Scoring model approach
The evaluation was carried out by using semi-structured interviews (face to face). Participants from the first round semi-structured interviews (data collection, section 4.8.4) were selected to study the proposed framework and provide feedback on and evaluation
Al-Yafei, 2018
159
of some assessment criteria. The selection of the sample size considered two engineers from the main disciplines in the engineering department. The interview included two parts: the first part was closed-ended questions where the participants were asked to rate the developed framework based on assessment criteria. The second part included open - ended questions and the interviewees were asked to provide their opinion on the potential weaknesses, limitations and the strengths of the framework (refer to chapter 8 for more detail).
4.10.3.1 Semi-Structure Interviews
The selection of the sample size considered two engineers from the main disciplines in the engineering department, as shown in table 14. In total, eight engineers were interviewed based on closed ended and open ended questions.
Table 14 Profile of the interviewees
Discipline Positions Years of experience
Structural Department Lead Engineer 14
Structural Department Senior Engineer 25
Piping Department Senior Piping Engineer 10
Piping Department Piping Engineer 14
Mechanical and Materials
Departments Lead Engineer 23
Mechanical and Materials
Departments Materials and Corrosion Engineer 18
Process Department Senior Engineer 10
Process Department Process Engineer 7
In the closed ended part, the respondents were asked to rate the proposed framework based on the assessment criteria, as shown in table 15, by using a scale of (1 to 5), where (1) is poor, (2) below average, (3) moderate, (4) above average, and (5) excellent.
Al-Yafei, 2018
160 Table 15 Framework assessment criteria
Framework Assessment Criteria Clear and easy to understand
Systematic and well structured
Comprehensiveness (includes all aspects of sustainability and engineering) Framework applicability
Framework efficiency Framework practicability
Appropriate to the construction projects for offshore topside facilities Helps to understand the concept of sustainability
Helps to understand the concept of life cycle costing and its application Easy to use without complicated software
In the second part of the interviews, which comprised the open-ended questions, the interviewees were asked to provide their opinions about the potential weaknesses, limitations and the strengths of the framework, as well as providing any comments or suggestions that might help in implementing the developed framework. The analysis was presented in chapter 8.