Premalatha Karupiah
School of Social Sciences, Universiti Sains Malaysia, Malaysia E-mail: [email protected]
Abstract
This paper is motivated by some common misconceptions among young researchers on random sampling in survey research. In order to identify if these misconceptions are common among researchers, the author conducted a content analysis of 38 journal articles that have been published in peer reviewed journals from July 2013 to June 2014. Two misconceptions were identified among young researchers and in published articles i.e. meaning of random sampling and the use of inferential statistics with data from a non-random sample. The number of published article with these misconceptions is small but it presents a challenge to the teaching survey research methods and graduate supervision.
Keywords: Probability sampling, Inferential statistics, Random sampling 1. Introduction
This paper is motivated by some common misconceptions among young researcher on random sampling when conducting a survey as part of their research. It is based on the author‘s experience as a research methodology lecturer, trainer and graduate supervisor. Young researchers here refer to undergraduate and graduate students who are conducting a social science research. However, in order to identify if these misconceptions are common among other researchers, the author conducted a content analysis of journal articles that have been published in peer reviewed journals. This paper therefore discusses not only the common misconceptions experienced as a teacher but also similar misconceptions identified in published journal articles.
2. Methods
The content analysis used a sample of 38 journal articles published between July 2013 to June 2014 identified through a search in Proquest Sociology using the terms ―random sample‖ or ―random sampling‖. The sample only included peer-reviewed articles in English published in scholarly journals. There were 39 articles in this search but one was excluded because it was irrelevant to the study. This analysis used a purposive sampling technique, therefore, the findings are not meant for generalization.
The analysis of the articles focused on two aspects. The first round of analysis focused on how the sample was selected in each article. It looked not only if the article used words such as ‗random‘ and ‗probability‘ when describing the sample. In addition to this, it also looked at how the sample was actually selected (i.e. if the author has described the process of selecting the sample). Some of the things identified were how the questionnaires were distributed and to whom it was distributed.
The second round of analysis looked at the types of analysis used in analyzing the data which have been collected from the sample. In addition to this, the author also looked at how the
37 findings were interpreted and the type of generalization used in each article and if there are any cautions regarding the type of generalization that is being made according to the type of sample being used in the study.
3. Findings
From the author‘s experience of being a teacher and supervisor for more than ten years, one of the most common misconceptions among young researcher is on the meaning of the word ‗random‘ in random sampling. A random sample refers to a sample which is selected using probability techniques. Examples of random sampling techniques are: simple random sampling, stratified sampling, systematic sampling and cluster sampling (see Blaikie, 2003 and Neuman, 2014 for a detailed discussion on sampling techniques).
In daily life, it [random] can mean unpredictable, unusual, unexpected, or haphazard. In mathematics, random has a specific meaning: a selection process without any pattern. In mathematics, random processes mean that each element will have an equal probability of being selected (Neuman, 2014: 254).
In other words, young researchers have difficulties understanding the technical meaning of the word ‗random‘. Their understanding is based on the way the word ‗random‘ is used in everyday life. Therefore, many researchers use the term ‗random sample‘ to describe various types of non-probability sampling.
Related to this, they also have problems in selecting a suitable procedure to select a random sample. Random sampling, for example, is often mistakenly used to describe distribution of questionnaires to ‗random‘ respondents, which is a type of non-probability sampling technique (convenience sampling).
Similar misconceptions were identified in the articles. From the content analysis, two articles clearly used the word ‗random‘ to describe convenience sampling. One article, for example, explained that the researchers used a random sampling by distributing the questionnaires to random strangers.
Another common problem identified both in the papers and among students is the use of inferential statistics with data from a non-random sample. Inferential statistics is used to generalize the results from a random sample to the population from which the sample was drawn. This analysis is suitable for data collected from a random sample with a high response rate. This analysis is not suitable for data from a population or if the data was collected from a sample selected using a non-random technique (Blaikie, 2003).
The content analysis showed that seven articles used inferential statistics with data from a non-random sample. The first two are related to the misconception on the meaning of a random sample. Another three used inferential statistics on data from a non-random sample (either convenience or snowball sampling) even though both articles clearly acknowledged that the results cannot be generalized to the population. Another two articles used a non- random sample but used the bootstrapping procedures to do significance tests.
4. Implications for teaching and supervision
38 is small, it presents a challenge to the process of teaching research methods. It is difficult for students to understand the need to follow basic statistical assumptions in data analysis when some published articles also do not follow these assumptions closely. Similarly, the technical meaning of the word ‗random‘ is also lost in some of these articles and similar meaning may be used by the young researchers. The discrepancy between what is presented in statistics and research methods text and how it is used in actual research can be very confusing for young researchers and this becomes a challenge in teaching and supervising young researchers. Similar problems may arise when the work of these young researchers are being examined in the form of theses and dissertations. Therefore, it is important that the young researchers understand the basic assumptions related to random sampling and inferential statistics. This would enable them to select a suitable sampling technique and analysis for their study and defend their selection in the examination process.
5. Limitations
Only one database was used to select a sample of published journal articles. Future research should include more databases in the selection of journal articles. This study focused only on articles in sociology. In addition to this, terms used for the search of articles should include other related terms such as probability sampling or sample.
6. References
Blaikie, N., 2003. Analyzing quantitative data: from description to explanation. London: Sage Publications.
Neuman, W. L., 2014. Social research methods: qualitative and quantitative approaches. Essex: Pearson.
39