The third stage of the study involves the collection of data with the use of a questionnaire. Two aspects of work were carried out at this stage, which is the pilot study 2 and the administration of the refined questionnaire data as explained in the two sub-sections discussed herein.
Pilot Study 2
There is the refinement of the questionnaire required for stage 4 of the research through a pilot study. The pilot study seeks to address the research objectives 3 to 5 and serve as a trial survey before the initial research at the advanced stage 4 of the research. The development of the draft questions required for the quantitative data collection address pertinent issues about objectives 3, 4, and 5. The draft questionnaire, which serves, as the instruments for data collection at stage 4 of the research were refined based on the comments and suggestions of the respondents.
The pilot investigation further affords the opportunity to test the methods and instruments of the research for suitability in the project environment. It further assists in the evaluation of the adequacy of the sampling frame of the study and the appropriateness of the structured questions. Also, the second pilot study proved beneficial as it enables the preliminary discovery and identification of the limitations, deficiency, mistakes, and errors, which requires necessary corrective actions before the commencement of the full-scale data collection at stage 4 of the research. The pilot questionnaires, therefore, help to guide the instrument through the second phase of the empirical data collection.
143
Hence, five questionnaire surveys from the field experts and senior researchers assisted in refining and modifying the questionnaire instrument for correctness, consistency, and accuracy towards the execution of the advanced research.
The comments and responses obtained were carefully modified and tailored to the advanced research. These also assisted in the modification of the questions incorporated into the full case studies instruments. The data collected at the second pilot study was premised on the findings of the literature and the interview survey, which was adopted in the development of the final questionnaire. It also established the course of the second phase of the quantitative research, assist in examining the accuracy and the practicality of the questionnaire designed to foster efficient conduction of the questionnaire survey. The enumerated activities uncover the significance of the research and further intensify and ascertain the need for the study, research problems, and sub-problems. There was the identification of the readiness of the proposed participants, the practicability of the research within the intended environment and the appropriateness of the research approaches and questions.
The questionnaire was developed based on the findings of the literature and the case study interviews. The results were subjected to further pilot study with industry experts working in procuring authorities and private concessionaire’s establishments. The contributions of these industry experts included in the questionnaire, which reflects the renegotiation characteristics and peculiarities of the UK road concessions. The pilot study serves as an exercise to refine the findings of the literature and the case studies to ensure its applicability to the UK environment and foster respondents understanding. The reason is based on the researcher’s knowledge that the questionnaire is to be completed individually by the respondent without any third-party interference or the researcher’s assistance.
Five questionnaires were distributed to both the public and private sector representatives at the second pilot stage to elicit information on the case study projects. Three professionals working in the private and public sector and two academic members of staff at UK Universities experienced in PFI procurement assisted in assessing the questionnaire to ensure that the data collected are comprehensive and suitable for analysis. Also, two academic staff and industry experts working at universities in the UK provided answers to the questionnaire; personnel were contacted to ascertain the adequacy and conciseness of the questions, the respective identity of the respondents and to give their opinion on how to gain the audience of relevant stakeholders working on individual DBFO road projects in the UK. One of the purposes of the pilot study before the administration of the questionnaires is to ensure the relevance of the questions asked and the practicality of adopting the survey within the study population.
144
The results of the investigations, especially the questionnaire survey is discussed and form the basis for the execution of the sixth research objective, which is the development of a framework for VfM renegotiation in PFI (DBFO) road projects.
Main Quantitative Survey
The comments and suggestions of the respondents at the pilot stage are in the final draft of the questionnaire. On receiving the approval of the supervisory team, the closed-ended questionnaires requiring the respondents to choose from an array of answers received and analysed to provide solutions regarding objectives 3, 4, and 5 of the research. These led to the development, of a VfM renegotiation framework. Thus, the second stage of research at the fourth stage of the research framework involved the second pilot study and the administration of questionnaires. The process of the study for the advanced stage is in Figure 5.6.
Figure 5.6: Questionnaire Survey Research Process
Out of the Forty-one (41) questionnaires administered, twenty-five (25) were completed, i.e. 60.98% response rate of the sampled population (England, Scotland and Wales road DBFO). However, a thorough analysis of the questionnaires reveals that only 22 surveys are fully completed by the respondents, which represents 32.35% of the available total of the public and
145
private sector respondents in the UK (i.e., 68 respondents) as shown in Table 2.4. Hence, the responses received can be considered adequate and representative of the entire population of the study, which includes Northern Ireland, even though it is not part of the study. Blaike (2009) agree with this submission by stating that a researcher may choose an entire population of an investigation, or sample or select a whole element(s) of a sampling unit, as the sampling frame. This study chose PFI (DBFO) road projects in England, Scotland and Wales due to the decline of the Northern Ireland department of transport to participate in the research.
The response rate, as revealed by the analysis of the questionnaire is 32.35% based on the 22 respondents that fully complete the surveys from the sample population, which suggest that the data collected seems adequate. The evidence of the literature supports the argument based on the response rate of 20-30%, which is the norm in questionnaire surveys in the construction industry (Akintoye and Fitzgerald, 2000). Similarly, Sacks (2010) claims that student questionnaire surveys or employee surveys do not receive more than between 30-40% response rate or more, on average. The results of the quantitative study are discussed and form the basis for the sixth objective of the research, which is the development of a framework for VfM renegotiation in PFI (DBFO) road projects. The response rate of the central questionnaire survey is in Table 5.3.
Table 5.3: Response Rate of the Main Questionnaire Survey
S/N Questionnaire Administered Total Number Returned Total Completed (Fully) Number Rate (%) Number Rate (%)
1 41 25 60.98 22 53.66
The results from Table 5.3 reveal that 53.66% respondents returned the questionnaire fully completed. The fully completed surveys, therefore represent 32.35% of the total number of primary renegotiation stakeholders (i.e. public and private partners) of the sample population as shown in Table 2.4 as earlier described.
Reliability and Validity of the Data Collected
Reliability of research is measured and understood in terms variance of the numerical results, which can be ascertained by an indicator, which shows the characteristics of the process of measurement process and instrument adopted (Neuman, 2006). The reliability test, therefore, evaluates the degree of dependability or consistency of the measure of a variable in given research. The internal consistency of the data collected is usually assessed when conducting a reliability test to identify whether the degree that makes up the scale relate together (Pallant, 2010).
146
The data collected are typically measured to evidence the reliability of the results based on a set level and standard while providing information regarding the relationships between the individual items in the scale. One of the most commonly used indicators used to measure the internal consistency is Cronbach’s alpha coefficient (Pallant, 2010). DeVellis (2003) as cited in Pallant, 2010), the Cronbach alpha coefficient of a scale should be above 0.7. Table 4.9 gives the Cronbach’s alpha values of the scales used in this research. The reliability of both categories of respondents measured as shown in Table 5.4
Table 5.4: Overall Cronbach’s Alpha Values of the Categories of Respondents
As shown in Table 5.4, the only significant variable of respondents without practical renegotiation experience is 0.781, whose information is considered very reliable. Though, 0.000 could not conduct reliability tests due to few cases available for analysis. The results of the analysis suggest that the information provided by these respondents cannot be discarded entirely for two reasons. First, there is an indication that the responses given by the respondents without renegotiation experiences are based on privileged information by their respective positions in the public and private sector organisations. Secondly, the combined reliability and validity test results regarding the findings of the respondents involved with renegotiation and respondents without renegotiation indicate that most of the Cronbach alpha values are within the required range of significance. The reliability and validity test statistics, therefore, ascertain the quality and consistencies of the responses provided by the respondents (McCrae et al., 2011, Forza, 2002). For instance, the reliability and validity test, based on the research objectives as defined in the data collected, highlights the factors leading to renegotiation, renegotiation impacts on VfM, outcomes of the renegotiation and measures to ensure VfM at renegotiation. Firstly, the internal consistencies and measurement of the data collected are calculated by testing the reliability of the sample through SPSS. Furthermore, the calculation of the Cronbach's alpha coefficient of all the items was completed in comparison with the overall total coefficient of alpha, to arrive at an estimation of the internal consistencies of the data collected, which assisted in establishing the results of the survey.
Renegotiation Involvement (Yes) Renegotiation Involvement (No) Question No Variable Cronbach’s Alpha Question No Variable Cronbach’s Alpha
Q9_1-37 Factors leading to renegotiation 0.919 Q9_1-37 Factors leading to renegotiation
0.000
Q10_1-14 Impacts of the renegotiation VfM criteria
0.952 Q10_1-14 Impacts of the renegotiation VfM criteria
0.411
Q27_1-42 Measures to ensure the achievement of VfM at renegotiation
0.956 Q27_1-42 Measures to ensure the achievement of VfM at renegotiation
147
The values of the Cronbach alpha coefficient of all the items and overall total coefficient of alpha vary from 0 to 1 respectively. According to Field (2009), the number of issues on the scale determines the value obtained, i.e. the higher the number of items on the scale, the higher the alpha coefficient and vice versa. The significant value of the Cronbach alpha coefficient can be estimated as significant if the results or its value is between 0.7 and 0.9 (Field, 2009; DeVellis, 2003). Pallant (2010), citing DeVellis (2003), states that a Cronbach alpha value, which is above 0.7 proves to be significant and indicates the reliability of the system. In the context of the analysis of all the responses to the questionnaire used to collect the data for this study, the Cronbach’s alpha values of the survey data are as shown in Table 5.5.
Table 5.5:Standardised Cronbach Alpha Values of the Questionnaire Survey Data S/N Question
Number
Variables Cronbach’s Alpha (Standardised)
ANOVA
1 9 Factors leading to renegotiation 0.959 0.00 2 10 Impacts of the renegotiation on the VfM criteria 0.964 0.01 3 27 Measures to ensure VfM at the renegotiation of road DBFO’s 0.941 0.00
As shown in Table 5.5, the results of all the 22 questionnaire numbers that are analysed with SPSS, falls within the remit of the Cronbach alpha values theoretically established. As a result, the findings of the questionnaire infer that the results obtained during the survey, regarding the respective variables of the research, are reliable and valid. Hence the responses of all the 22 respondents are adopted for analysis.
5.6.4.1 Mean Statistics Comparison and Analysis of Variance
All elements of the descriptive statistics were evaluated and through the estimation of frequency, such as the mean and Analysis of Variance (ANOVA) amongst others. The results of the questionnaire data provided information concerning the factors influencing the renegotiation of PFI (DBFO) road projects in their varying degrees, based upon the frequency and mean values amongst other things. Analysis of Variance (ANOVA) is used to explore data and verify the possibility of having more samples sharing the common trait of likelihood (Field, 2009). The central tendency and sample mean of the different items and data are compared and tested indirectly with ANOVA.
The results produced are usually the F-ratio, which compares the number of systematic variances in the data with the number of unsystematic variations. Therefore, the "F-ratio" test value can be considered as significant when the value obtained by evaluation is less than or approximately equal to 0.05 (5%).
148
In all the sections of the questionnaire analysed, the ANOVA values are less than 0.05, which justifies that more samples within the respective variables have typical traits of likelihood, and as a result there is evidence that the values of the data collected are significant, as shown in Table 5.5.
5.6.4.2 Relative Critical Index of the Research Variables
The critical index estimated and adopted for the interpretation of the level of criticalness of the respective factors, impacts and measures that have a mean value of 4-5 imply very critical factors. The mean value of a factor, which falls between 3-4 imply critical factors, while the mean value of a variable or criteria, which falls between 2-3, is considered moderately critical. Mean value of 0-2 is assumed non-critical and is regarded as a variable with less significance or level of importance. The criticality estimation of factors has been estimated through the adoption of the mean values. Though, standard deviation has also been found useful in other analysis of data. Somers and Nelson (2001) indicate the criticality levels of individual variables through the means and standard deviation, based on twenty-two CSF’s in descending order of importance, which are 5 = critical, 4 = very high, 3 = high, 2 = moderate and 1 = low: to establish the critical success factors across enterprise resource planning implementation stages. However, in the context of this research, the mean values are estimated in the case of each variable of factors, impacts, outcomes and measures, as identified in the questionnaires adopted in the data collection. The interpretation of the analysis on the criticality indexes was based upon the criticality level defined in the literature (Somers and Nelson, 2001).