The pilot study involves the preliminary investigation of a research with relatively small sample before the main study. This is usually done to pre-test the research instrument and correct all observed anomalies before the main study (Alasuutari, Bickman & Brannen, 2008; Welman, Kruger & Mitchell, 2005). Alasuutari, Bickman and Brannen, (2008: 266) explain that:
It seems obvious that the answer to the question needs to reflect what we wanted to know. However, respondents may not understand the question in the way the person who wrote it expects….writing good questions requires pre-testing.
Welman, Kruger and Mitchell (2005: 148) affirmed that it is necessary and useful to pre-test the instrument for data collection “before administering it to the actual sample”. So, in order to write good questions for the survey research (with questionnaire or structured interview), the two instruments for data collection: the questionnaire and the interview schedule were pre-tested in the study through a pilot study. Deng (2010) gave the need to ensure content validity of the questionnaire as a basic reason for a pilot study. So, a pilot study helps a researcher to modify questions in the survey with both questionnaire and interview as well as the mode of presentation of these questions by the researcher (6 & Bellamy, 2012; Maykut & Morehouse, 1994; Silverman, 2001).
A pilot study with the survey method was conducted at the University of Calabar. A random sampling technique was used to select 35 academic staff as respondents in the pilot study. The designed questionnaires were administered face-to-face to the respondents and the feedback from
the retrieved and completed questionnaires was helpful in modification and revision of the questions on the designed questionnaire for the main study. Similarly, a random sampling technique was used to select 5 academic staff for pilot study using interview schedule as instrument for data collection. The pilot survey was vital for the revision of questions on the interview schedule before the main study. This was absolutely useful in re-defining or streamlining questions for the final interview of the academic staff in the study. A pilot study was also conducted with bibliometric method with The Web of Science as instrument for data collection. A bibliometric analysis with publication count of research productivity of academic staff was conducted with the University of Calabar as organization. The pilot study was helpful in enhancing the skills of bibliometric analysis in the main study by the researcher. Thus, the pilot study was vital in getting the researcher to gain familiarity with bibliometric method before the final study was conducted at the two surveyed universities.
4.8.1 Validity and reliability of instruments
Validity is the ability of a research instrument or instrument for data collection to measure a research variable effectively or the degree in which a variable is measured well by the research instrument (Antonius, 2003; Frankfort-Nachmias & Nachmias, 1996; 6 & Bellamy, 2012). Simply put validity “refers to the extent to which measures actually measure what they claim to measure” (6 & Bellamy, 2012: 92). Validity helps a researcher to draw a sound inference or conclusion from his/her data. Expert’s view on the research instrument is a common measure of its validity in social science research (Frankfort-Nachmias & Nachmias, 1996). Thus, the designed questionnaire and interview schedule were validated by experts in Information Science, precisely, the two supervisors of the researcher as veritable tools for data collection in the survey. Similarly, in bibliometric method, the use of The Web of Science as instrument for data collection was validated by the two supervisors as capable of measuring effectively the research productivity of the academic staff in the surveyed universities.
Reliability is a process of ensuring consistency in measuring a research variable by the research instrument (6 & Bellamy, 2006; Field, 2005). According to Field (2005: 666), “reliability just means that a scale should consistently reflect the construct it is measuring”. 6 & Bellamy (2012: 21) explain that “a reliable system of measurement or coding is consistent in that, each time it
used on the same data, it yields the same measure or code”. SPSS software was used to determine the reliability of the questionnaire as instrument for data collection in the survey. The reliability estimates obtained are shown in Table 4.5. The values of the reliability estimates range from 0.784 to 0.922 and these indicate that the questionnaire instrument was reliable to use for data collection in the study.
Table 4.5 Reliability estimates of questionnaire instrument
Variable Cronbach’s alpha
Electronic information environment 0.847
Accessibility and utilization of e-resources 0.784
ICT policy/strategy 0.922
Perception of effect of e-resources on productivity 0.887
Reliability measure is difficult to assess with interview schedule as instrument for data collection (6 & Bellamy, 2012; Silverman, 2001). This is aptly captured by Silverman (2001: 13) that “A central methodological issue for quantitative researchers is the reliability of the interview schedule”. However, and in order to maximize reliability of the interview schedule, these scholars proffered solutions which were adopted in this study (6 & Bellamy, 2012; Silverman, 2001). Silverman (2001: 229) suggested that the researcher should ensure “that each respondent understands the questions in the same way and that the answers can be coded without the possibility of uncertainty”. In addition, 6 and Bellamy (2012: 94) argued that reliability is dependent “upon methodical attention to detail” by the researcher. Thus, the following measures were used to increase or maximize reliability of the interview schedule in the study. One, attentions were given to the concepts used to frame the questions in the interview schedule in order to aid each respondent to have the same understanding of these concepts. Ambiguous questions were avoided. Secondly, pilot study was conducted to pre-test interview schedule, and thereafter necessary modifications were made. Finally, attention was given by the researcher in recording the responses of the respondents with clarity and consistency as guided by the interview schedule.
In terms of bibliometrics, the reliability of The Web of Science as the instrument for data collection was affirmed by the two supervisors and relevant literature. Nwagwu and Egbon (2011: 439) referred to The Web of Science as a:
very popular index and its Journal Citation Reports might be reflecting merely the global perception and assessment of science performance, but they also probably give some insight about the condition of science in Africa.
The reliability of The Web of Science as instrument for data collection in bibliometric studies is affirmed by Okafor and Dike (2010) who state that the database “provides objective and accurate data pertaining to journals, years and countries wherein the academics publish and even such details as number of authors for each publication”. So, The Web of Science was considered as a reliable instrument to capture data on publication counts for the two surveyed universities (in view of non-availability of relevant or similar database in Nigeria and Africa in general) (Ani & Onyancha, 2011; Nwagwu & Egbon, 2011; Okafor & Dike, 2010). In terms of measuring and getting consistent results, the researcher pre-tested the instrument with his name as an authour and that of five notable authours to assess their research productivity within a given period of time (2005-2012 used in the study). A repeated measure by the researcher indicated that the results obtained were consistent for each of the authours. Thus, the reliability of The Web of Science as instrument for data collection in the study was not in doubt, but observed by the researcher to be trustworthy in line with literature and experts’ views.