Inaccuracy refers to either intentional or unintentional errors in a respondent’s answers, in the future
(predictive) or in the present (concurrent). Predictive inaccuracy is a special case of response error
caused by inaccurate intentions.
Example: A respondent indicates that he intends to buy a new sports car within 6 months, but
he does not. In this case, the respondent’s intention was clear, but was not followed. This situation is a predictive inaccuracy.
A similar type of predictive inaccuracy can occur when marketing researchers try to predict actual market response to a price by asking consumers, “How much are you willing to pay for Product X?” Differences between predicted and actual price acceptability may occur because the true range of an acceptable price may change by the time of purchase. Budget constraints, windfalls, similar products at the point of purchase, search costs, and purchase urgency must be considered.
Problems: The interviewer must formulate and adapt questions naturally. As a result, he or
she may influence the quality of information gathered. For example, the completeness, the objectivity, and the bias in the interview may be affected. Thus, a major problem with unstructured-direct interviews is finding competent interviewers. These responses cost more per-interview because of the additional attention required to assure quality in unstructured interviews. As you would assume, these interviews generally take longer than those that use a questionnaire. In addition, editing and tabulating problems becomes more complicated because of the varied order of asking questions and difficulty in evaluating, coding and recording answers.
Concurrent inaccuracy occurs when the respondent intentionally does not provide accurate information
because of an inability or an unwillingness to respond.
Concurrent inaccuracies are a major concern for information obtained in interviews (for information on past behavior, socioeconomic characteristics, level of knowledge, and opinions and attitudes). Concurrent inaccuracies may also apply to instances where respondents are observed but not formally interviewed) and the interviewer is unable or unwilling to provide the desired information. It is clear from these brief examples that inability and unwillingness to respond are major contributors to response bias and warrant more detailed attention to understand how they can be controlled.
Inability to Respond
Even a straightforward question such as “What is the model year of your family car?” may result with inaccuracies, particularly if the car is several years old. If respondents were asked, “What brand or brands of tires do you have on your car?” most would have difficulty providing an accurate answer without looking. Finally, if respondents were asked, “What reasons did you have for buying Brand ‘A’ tires instead of some other brand?” most respondents struggle in providing an accurate answer. Semon (2000a, 2000b) suggests that inaccuracies due to inability to respond stem from three major conditions:
• Memory error: A respondent gives the wrong factual information because he or she does not remember the details of a specific event. Often, the amount of time since a purchase was made is underestimated or overestimated. A better questionnaire and survey design can help reduce this error. Proven techniques such as follow-up interviews will verify answers. How ever, they often are not used because they add to the length and cost of the survey.
• Ignorance error: This refers to the respondent’s lack of understanding, awareness, or perception of the question. If too many customers struggle with interview questions, your research design or even entire questionnaire may be unrealistic, deficient, or directed to the wrong population.
• Misunderstanding: Poorly defined terms or words with different meanings can lead to inaccurate responses. Proper question design avoids words with multiple meanings and definitions. Proper questions will also clearly define the context of word usage in the questionnaire.
Unwillingness to Respond
When we start looking at respondents’ motivations, the problem of reducing bias gets more complicated. We have to figure out why respondents are not willing to accurately provide the information desired. There are always negative perceptions attached to sharing personal information, and there are no complete theories to explain this behavior.
46 | CONDuCTINg INTERvIEwS
Investigator Expectations
One complex source of inaccuracy in response stems from the respondents’ appraisal of the investigator.
Example: A cosmetics study reported an unexpectedly high usage of luxury cosmetics
among women from low-income families. In this case, a well-dressed and carefully groomed interviewer conducted all of the interviews. The study was repeated with a matronly woman. Her clothing was similar to the women previously interviewed. This time, the same respondents reported that they used fewer expensive brands. In this case, the specific interviewer had a direct effect on the answers of the respondents.
Investigator Unwillingness
Sometimes the respondent is willing to provide accurate data, but the interviewer is cheating. This happens when an interviewer finds a particular question too embarrassing to ask; when the interviewer finds it easier to complete the survey forms himself/herself rather than conduct the interviews; or when interviewers have friends complete the survey. An interviewer may also complete some of the questions legitimately and then estimate or infer other information such as age, income, and certain attitudes or behaviors of respondents.
Carefully selecting, training, and supervising can reduce the level of interview cheating. Control procedures can also help. The simplest control procedure is to call back a subsample. If the information on an initial interview disagrees significantly with that on the call-back interview, establish who the interviewer was and call back his/her respondents.
Other control procedures include the use of “cheater” questions. Cheater questions have specified answers or are common knowledge, such as, “For this question, select option three”, and “The sun revolves around the earth”. Likewise, analyzing patterns of responses for interviewer differences will disclose interviewer cheating when there are significant variations from expected norms.
Costs of Time and Effort
Perhaps the most common reason for respondent unwillingness to provide accurate information is the time and effort required to provide answers. This is especially problematic in online surveys. Here respondents often give hasty, ill-considered, or incomplete answers. They resist completing the open ended text questions that probe for more accurate information. A respondent will tend to act in a manner that will reduce time costs. Such behavior often results in inaccurate or missing information.
When using telephone questionnaires, a good solution is to have interviewers ask, “Is this a good time to answer some questions, or would you rather set a time when I could contact you again?” Experience has shown this latter technique only slightly lowers response rates.
Perceived Losses of Prestige
When respondents attribute prestige to the information, there is a tendency to give higher-prestige responses. Questions that elicit this type of bias often include sensitive information related to socioeconomic status, including age, income, educational level, occupation, place of birth, and residence.
Example: People subtly associate prestige with brands of beer. In a survey
on what national brand respondents preferred, one of the questions was, “Do you prefer light or regular beer?” The response was overwhelmingly in favor of light beer. Nevertheless, sales data indicated a strong preference for regular beer. The reported information was clearly inaccurate. Subsequent investigation revealed that the respondents viewed people who drank light beer as being more discriminating. Therefore, respondents had given answers that, in their view, were associated with a higher level of prestige.
Measuring the amount of inaccuracy is a difficult task. One solution is to ask for the information two different ways. For example, when obtaining information on respondents’ ages, it is a common practice to ask early in the interview, “What is your present age?” and later “In what year did you graduate from high school?” Likewise, a third-person technique can be used. Respondents could be asked, “Are you afraid to fly?” followed later by “Do you think your neighbor is afraid to fly?” In this case, most neighbors turned out to have severe anxieties about flying.
Invasion of Privacy
Clearly, some types of information are viewed as private. Both non-response and inaccuracy can be expected. Finances, family, personal life, personal hygiene, political beliefs, religious beliefs, and occupation information can be delicate issues. Respondents sometimes respond badly when they feel an interviewer is prying. Privacy is a sensitive matter, and what one person considers private may not be viewed that way by others.
The investigator should attempt to determine sensitivity if it is suspected to be a problem. One way of handling this is to add questions in the pretest stage which ask about the extent of sensitivity to topics and specific questions. A comprehensive treatment of sensitive information and how to ask questions about it is given by Bradburn and Sudman (1979), and by Reuzetti and Lee (1993).