The 2013 dataset

The sixth wave of the WVS – once again conducted under the auspices of Harris and Kotzé - took place between August and October 2013. A nationally representative sample of 3 531 adults were interviewed in the aforementioned South African languages. The Face To Face Computer Assisted Personal Interview (CAPI) method was used to execute the survey. Interviews were personally conducted and answers entered into the CAPI computer. Paper copies of the questionnaire were available for use in instances of electricity failure or absence. According to Ipsos (2013), the CAPI methodology provides results that are highly

representative of the population. Ipsos also estimates that the margin of sampling error42 is 1.65%. Researchers can be 95% confident that the true opinion of South Africans fall within a margin of 1.65% of either side of the survey results (Ipsos, 2013:14-26). The data was also once again weighted to represent the population.

As was the case with the 2006 survey, the field workers who conducted the sixth wave were experienced and well-trained. They completed the Ipsos training course (including training in probability sampling and questionnaire administration) and passed competency tests. Field workers were thoroughly familiarised with all aspects of the study (including sampling and respondent selection) and once again participated in mock interviews before taking to the field. Once fieldwork was underway, the 192 fieldworkers reported to 20 supervisors who in turn informed the field coordinators of the progress made and challenges encountered. 17.75% Of the interviews were back-checked for quality control purposes (Ipsos, 2013:14-29).43

42_{Sampling error reflects how much the sample differs from the real population. The use of random sampling} enables the researcher to calculate how big the sampling error is (Neuman, 2011:248-249).

43

55 3.6 Conceptualisation and operationalisation of variables

The study of values is an attempt to measure the intangible. In order to do so one needs very clear, conceptual definitions of the relevant values (conceptualisation) as well as trusted ways in which to measure them (operationalisation). Researchers who want to study intangible matters that cannot be directly observed and easily measured, have to:

“Obtain an estimate indirectly by using a surrogate measure that is observable and assumed to be related to the more interesting phenomenon. The critical assumption of such indirect measurement techniques is that there is a relationship between the observable measure and the unobservable event of interest. Even when this assumption is correct, however, the relationship may be decidedly less than perfect” (Stewart & Kamins, 2012:20).

Researchers therefore need to have confidence in the validity and reliability of the measures they use, thereby ensuring minimal measurement error (Field, 2013:12). Validity refers to correctness or truthfulness: whether scientific measurement is an accurate reflection of reality. Validity is lacking when the ideas used to analyse a society differ significantly from the lived experience in that society. Reliability is broadly defined as the dependability or consistency of the results obtained (Neuman, 2011:207-208). If measurement techniques have an undue influence on results, the results may be unreliable and unlikely to be consistent with that of other studies.

Neuman (2011:208-214) distinguishes between four main types of measurement validity: face validity, content validity, criterion validity and construct validity. He also mentions three types of reliability: stability reliability, representative reliability and equivalence reliability. Face validity occurs when there is consensus in the scientific community regarding which indicators measure which constructs. It is quite common for consensus of the definitions and measurement of constructs to be lacking (Stewart & Kamins, 2012:20-21). Inconsistent results are therefore often due to simple differences in how a concept is defined rather than disagreement of the actual nature of the subject under study.

Content validity refers to whether all aspects of a definition are represented in a measure. If a construct is defined as consisting of three specific elements, the measure must accurately measure all three elements – the entire content of the definition. If only part of the construct is measured, the researcher will have low content validity. In such cases, Neuman (2011:212-

56

213) suggests that the measure must either be expanded or the definition narrowed in order to obtain content-validity.

Criterion validity is when a researcher uses objective criteria to establish that an instrument does indeed measure what it claims to measure (Field, 2013:12). Researchers can establish that an instrument or indicator is valid by comparing its results with that of a trusted indicator (for example, one that has face validity) (Neuman, 2011:213). When researchers record data while using both the trusted indicator and the one being tested, they are assessing concurrent validity. When data from the new indicator is used to predict future observations, researchers are assessing predictive validity (Field, 2013:12). Concurrent and predictive validity are both subtypes of criterion validity.

Finally, construct validity refers to measures with multiple indicators. There are two types of construct validity: convergent and discriminant. Convergent validity occurs when many indicators are associated with each other and tend to converge. If researchers measure a construct and find that all the construct‟s indicators converge, there is strong convergent validity. Discriminant validity occurs when a construct‟s indicators not only converge, but are also negatively associated with an opposing construct. For example, one would expect the indicators of political conservatism to converge and to be negatively associated with the indicators of liberal political attitudes (Neuman, 2011:213-214).

The three kinds of reliability refer to whether one‟s measures are reliable (deliver the same answers) across time (stability reliability), across different subpopulations (representative reliability) and when using multiple indicators to measure the same construct (equivalence reliability) (Neuman, 2011:208-209). The variables used to measure values in this study are reliable across time and across the subpopulations of South Africa. The variables are also valid in terms of face validity (their conceptualisation is widely accepted and makes sense) and content validity (they measure what is intended).

3.6.1 Evil

Evil as a belief is conceptualised in two ways in this study, namely the literal and the secular definitions as discussed in Chapter Two. The belief in a literal evil is defined as the belief in real (if unseen), malignant spiritual forces that can cause real and direct harm in the lives of

57

individuals. Secular evil is conceptualised as an entirely non-spiritual phenomenon

characterised by wrongdoing on the part of flawed human beings. In the WVS (2006a; 2013) respondents are asked to strongly agree, agree, neither agree nor disagree, disagree, or

strongly disagree with three statements regarding evil:

 “Evil is everywhere”

 “Evil can take possession of some people”

 “The community must get rid of people possessed by evil even if it means that they have to be killed”

As explained in Chapter One, all believers in evil can agree with the first statement – it therefore measures the belief in evil in general. The other two statements specifically refer to evil as a force that can take possession of people. These statements are therefore meant to measure only belief in literal evil. Evil is thus included as a categorical variable in this study. With regards to the latter two statements, it should be noted that although the statements were formulated with the intention that respondents interpret evil literally, one cannot be sure of such interpretation. A figurative understanding of “possession” may still have been possible on the part of some respondents. However, regardless of literal or figurative interpretation, there is no mistaking the serious nature of the third statement. This statement measures support for the harsh and violent punishment of evil-doers. It is therefore of relevance as an illiberal value regardless of the individual understanding of evil.