Pilot Study and analysis

Conducting a pilot study is essential in a research study, which is usually a small scale study, conducted before the main data collection- using survey questionnaire (Wilson, 2010).

Selecting reasonable sample size from the target population is consistent and recommended for this scale of study (Sekaran 2003) . In preparation for the main survey execution for this study, pilot study was conducted between 14th August 2015 and 28th August 2015. Main objective of this pilot study was to evaluate measuring scale’s reliability and validity of measurement data. This along with respondent’s feedback on questionnaire

140 wording, content flow and structure and finally to ensure that the time taken to complete the questionnaire was suitable to most potential participants to accurately provide answers.

Considering the location - Bengaluru based spiritual organisations from where the respondents were chosen (see section 4.13for more details on selecting location), it was important to ascertain that the willingness of the spiritual organisations’ premises for distribution and collection of questionnaires and also to establish if the respondents understood the questions and were able to accurately respond based on their experience.

This was important as this digital media could be subjective and understood differently based on how the respondent used digital media and how they understood its capabilities (Dadzie, 2002).

The pilot study was done in four stages, in the first stage questionnaires were sent to supervisors for critical review and comments. After receiving the suggestions and comments via both email and personal discussions - changes were made to add more details to the terminologies used in the questionnaires. In the second phase, the questionnaires were sent to eight peer colleagues for their feedback based on their knowledge and experience (Dillman 2011) and further forty questionnaires were administered for pilot study after prior screening from a gathering of spiritually motivated individuals in Bengaluru (the time lines for the data collection process are shown in the Table 4:8) . Sixteen responses were incomplete and only twenty four responses were completed and deemed fit for inclusion in the data analysis. Of the twenty four responses 83 per cent were males and 17 per cent were females, aged between seventeen and thirty. Data analysis was performed involving data cleaning, EFA, CFA, reliability tests, validity tests and path analysis was performed using

141 SPSS and AMOS. Investigations on the results revealed essential changes to the wording of the contents and the presentation style. The term interactive systems, modern systems and spiritual strivings were clearly redefined with examples. The presentation style changes included changing the page layout to portrait from landscape and increased font size to accommodate the respondent’s feedback. It turned out that most participants were keen to be contacted again if the study needed additional information and showed the interest to leave their email and name. Thus name, email and additional details options were added as optional data to enter if preferred. This phase included understanding the real experience and appearance of questionnaire before launching it, using the actual setting over the internet.

Data analysis stage was performed to refine the scale instruments as part of scale purification depending on the results of the data analysis (Churchill, 1979; Appiah-Adu, 1998). The measurement scales that have been used in this study are the adaptations of the original scales from previous studies and modified to fit to the context. The adapted questionnaire has been reviewed by subject experts from academics for its appropriateness and wordings prior to pilot testing.

Data analysis began with data cleaning and verified for unengaged responses (none), outliers (none) and kurtosis and made no changes. Kaiser-Meyer-Olkin Measure of Sampling Adequacy for individual items was 0.84 (> 0.5 min and close to 1 was expected).

Communalities were greater than 0.3 which indicated that the items were well correlated.

For assessing the internal reliability of the scale, Cronbach's alpha coefficient was used to

142 provide support and was greater than 0.82 where the coefficient up to 0.7 is suggested as providing a strong reliability (Cronbach, 1947). All items showed that they were loaded to at least one factor and there showed no negative loadings and also the average of group loads indicated greater than 0.7. Thus after satisfactory results of reliability tests, validity tests were performed. Pattern matrix was used to verify for a very clean factor structure in which convergent and discriminant validity was evident by the high loadings within factors, and no cross-loadings between factors. Variables were highly correlated where the item loadings were >0.5 and Ave>0.7. The measures included verifying if all the items measured the same factor (Item loading>0.3) and also verified for how much each variable were correlated with other variables. Variables showed that they were relating more strongly to their own factor than to other factors (discriminant) and indicated <0.7 showing that they had less shared variance. After performing the exploratory tests, confirmatory tests were performed where measurement model was validated using (AMOS 22.0 version). Model fit measures were verified as recommended by Hair et al (2010) and Hu and Bentler (1999) (Hair et al. 2010 on page 654) and Hu and Bentler (1999). The results indicated a good fit. Standard deviation tests with mean scores were conducted. Using SEM and multiple regression techniques, the developed hypotheses were tested. Overall the changes to the instrument were only made to the problems that were raised by respondents and issues that were found during data analysis. These steps were necessary to establish the comprehensiveness of the measurement scales that were used in the study. As part of pilot testing phase- the same questionnaire was pilot tested on websites of spiritual organisations by posting a link to the survey and also used email requests with access to survey in the email. The main intention of this phase was to evaluate technicality aspects that were involved by respondents to

143 answer and also to verify the design and content aesthetic view with all the necessary information that respondents may want to know or contact. Changes were made with this phase of pilot testing are described in detail in section 4.11.

Table 4:8 shows all the data collection process that was involved in this study commencing with pilot study to the final main data collection.

144 Table 4:8 The data collection process timeline

Phase Description Respondents data Time

span measurement data along with respondent’s feedback on questionnaire wording, content flow and structure and finally to ensure that the time taken to complete the questionnaire was suitable to most potential test technicality, design and content of the web survey.