Quality of the Research Methods

With the purpose of ensuring that the instruments developed for this study made precise and accurate measurements, it was necessary to assess the "goodness" of measures. There are three criteria for testing the goodness of measures: reliability, replicability and validity of the research (Bryman 2012). Furthermore, due to the descriptive and explanatory nature of this research, the quality of research methods also needs to be measured in terms of its usability. And finally, the consideration of possible sources of error is also useful for the appraisal of the research quality.

Reliability means that the instrument chosen gives consistent results of the scale over time, taking into consideration a margin error (Golafshani 2003; Creswell 2009; Bryman 2012). The assessment of the reliability guarantees that the data collection instrument is suitable in obtaining information regarding the topic under research. This kind of reliability has been ensured through a piloting process and the inclusion of comment boxes within the questionnaire.

According to Collis and Hussey (2009), research measurements or findings can be reliable if they produce the same results by re-testing them with the same test at a different time. There are three common ways to validate the reliability of the participants’ responses to the questions: test and re-test method; alternate-form; and internal consistency method (Litwin 1995). Field (2013) argues that the internal consistency method, which is based on ‘Cronbach’s Alpha Test’ the best way to assess the reliability of questionnaire responses statistically. According to Hair et al. (2007, p.243) “this type of reliability is used to assess a summated scale where several statements (items) are summed to form a total score for a construct”.

The value of Cronbach's alpha coefficient can range from zero (i.e. no internal consistency) to one (i.e. complete internal consistency) (Bryman and Bell 2011). As a rule of thumb, the closer Cronbach’s Alpha coefficient to 1 means higher intimal consistency and more reliable (Hair et al. 2007; Gray and Kinnear 2012; Davis 2013; Field 2013). However, Cronbach’s Alpha value is inflated by a larger number of variables, so there is no set interpretation as to what is an acceptable alpha value. In general, researchers agree that an alpha value of at least 0.7 is considered acceptable for reliability (Sekaran 2013). However, Hair et al. (2007) suggest that values below 0.70 also can be realistically expected.

Replicability refers to the probability of repeating the research process, allowing other researchers to test the results and it typically occurs in quantitative research. The replicability of this study has been optimised by demonstrating the whole procedure for the selection of methods, measures, participants, and designing data collection instruments. Moreover, the implementing an online self-completion questionnaire without the intervention of a researcher reduces the possibility of bias and enhances the replicability of the study.

And finally, validity refers to the strength of the inferences in the research project. According to Litwin (1995), validity means that the data collection instrument is valid if it measures what it designed to measure. In other words, the instrument is valid for the research if it measures the research objectives (Bryman 2012). Ruane (2005) explained that validity is concerned with the extent to which a research tool gives the correct answer. Another issue related to validity is the extent to which the research results can be generalised to the total population. Therefore, it is important to ensure that a research tool (i.e. questionnaire survey) carefully reflects the correct meanings and connects to the research questions. There are different types of validity; the most common types included for the evaluation of the research project are the construct validity, the internal validity, the external validity, content validity and the ecological validity.

Construct validity, also termed measurement validity confirms that the measurement technique represents the concept under research. This type of validity has been addressed by means of a pre-test approach (i.e. pilot test) depending on research academics and professionals (i.e. the senior managers in Jordanian hotels) as outlined earlier in this chapter. Therefore, the development of the questionnaire for this study was based upon the results of the pilot study, as well as the findings from the relevant literature review. Ruane (2005) posits that a research tool should be pilot tested before conducting the empirical research to assess its validity. Furthermore, participants were given the opportunity to include further information within the comment boxes and a glossary was developed to assist participants in completing the questionnaire. In this way, it ensured that the survey questionnaire for this study would provide data that related to accepted meanings of the concepts involved.

Internal validity denotes the degree of confidence that the observations made on can really clarify the phenomenon and are not affected by further variables. Internal validity tends to be high in qualitative research as a consequence the contextualisation of the phenomena and the wide inclusion of variables in this kind of research. It is easier to challenge in quantitative research, where the degree to which further variables influence the phenomena under research is more difficult to track. Experimental design minimises the likelihood of biased

through the influence of additional variables when examining a cause and effect relationship, through the integration of a temporal dimension. However, the internal validity in cross- sectional research studies is limited, and can only support the identification of associations between variables, without inferences on the existence of a cause and effect relationship. In order to address the internal validity, many further variables that may influence ICT adoption and marketing performance was included in the data collection instrument depending on the academic literature.

External validity brings up the extent by which the results can be generalised to the entire population. In this research, surveying the entire population maximises the generalisation of the results. However, there are issues related to the electronic distribution of the data collection instrument, which might have possibly skewed the sample towards participants with knowledge about ICT, it is expected that this type of bias will equally affect the population used for benchmark from other country. Additional limitations could be connected to the low representation of hotels supporting some specific analysis.

Content validity relates to the extent by which a research mechanism collects all the elements which are specific to the domain of content. Content validity is one of the methods that evaluate the validity of an instrument by the judgement of a group of experts to ensure that the questionnaire includes an adequate and representative set of questions that reflect the real meaning of the concept (Litwin 1995; Zikmund et al. 2013). Content validity was also address through the placement of comment boxes throughout the questionnaire, enabling participants to introduce further ICT systems supporting their business operations and contributing to marketing performance.

Ecological validity refers to the ability of the research instrument to capture the daily life conditions, which is improved through methods like participant observation. Collecting data through an online self-completion questionnaire may provide results do not capture what happens in people's everyday life. In order to address the ecological validity, the data collection instrument was built upon the academic literature. Moreover, interviews with people from the industry were developed during pilot study.