Methods: Data Collection & Survey Design

Methods: Data Collection & Survey Design

Dr. Patrick Toche

References :

† William G. Zikmund, Barry J. Babin, Jon C. Carr, Mitch Griffin, Business Research Methods, South-Western College Pub, 9th edition (2012), 978- 1111826925.

Other references may be given from time to time.

Learning Objectives

1. Outline the stages of the research process.

2. Understand the ethical issues raised by research.

3. Distinguish the various categories of surveys. Describe the category of questions used in a questionnaire.

4. Understand the principles of experimental design. Identify sampling errors and systemic biases.

5. Understand the purpose of case studies, experience surveys, and

focus group interviews. Distinguish primary and secondary data.

The Research Process

I Interpret empirical evidence to confirm or disprove a hypothesis.

I Stages in the research process: 1. Research objectives. 2. Re- search design. 3. Sample design. 4. Data collection. 5. Data analysis. 6. Research report.

1. Exploratory Research: Define the nature of a problem, clarify ambiguous situations, discover potential opportunities, shapes hypotheses, outline research strategy.

2. Descriptive Research: Describes the characteristics of objects, people, groups, organizations, or environments. Who? What?

When? Where? Why? How?

3. Causal Research: Identify cause and effect. Rule out spurious correlations.

Causality

I Absolute Causality: The cause is necessary and sufficient to bring about the effect.

I Conditional Causality: A cause is necessary but not sufficient to bring about an effect.

I Contributory Causality: A cause need be neither necessary nor sufficient to bring about an effect — The weakest form of causality.

I Concomitant variation: Apparent covariation without a causal relation.

I Spurious association: Apparent covariation that can be explained

by the action of another variable.

Stages in the Research Process

1. Research objectives 2. Research design 3. Sample design

4. Data collection 5. Data analysis 6. Research report

Research Objectives

1. Research objectives

Exploratory research: Identify and clarify the problem.

Literature review: A directed search of published works, including periodicals and books, that discusses theory and presents empir- ical results that are relevant to the topic.

Pilot studies: A small-scale research project that collects data from

respondents similar to those to be used in the full study.

Research Design

2. Research design

Methods and procedures for collecting and analyzing the needed information. Basic design techniques for descriptive and causal research:

Surveys Experiments Secondary data Observation

Sample Design

3. Sample design

Procedure that draws conclusions based on measurements of a portion of the population. Basic steps in sample design:

Select the target population: Who to sample?

Select the sample size: How many people to contact?

Select the sampling procedure: Simple Random Sampling, Strati-

fied Sampling, Cluster Sampling, Systematic Sampling, Multistage

Sampling.

Data Collection

4. Data collection

Data may be gathered by human observers or recorded electroni- cally. Basic steps in data collection:

Data cleaning: Involves checking the data collection forms for omis- sions, legibility, and consistency in classification. Often the most time-consuming aspect of data collection.

Data coding: Rules for interpreting, categorizing, recording, and transferring the data to the data storage media.

Data analysis: Application of statistical analysis. The ultimate pur- pose is often causal inference.

Data Analysis

5. Data analysis

The purpose of data analysis is to summarize relevant research details and identify consistent patterns. Points to note include:

The appropriate technique is determined by the characteristics of the research design and the nature of the data collected.

Statistical analysis starts with scatter plots, frequency distributions,

and summary statistics to more advanced multivariate regression

analysis, panel time series, and robust techniques.

Research Report

6. Summary & Conclusions

The final stage of the research project.

The summary and report consists in interpreting the research re- sults, describing implications, and drawing appropriate conclusions for managerial decisions.

A good summary must be adapted to the varying abilities of people to understand the research results.

Management may not be interested in detailed reporting of the research design and statistical findings.

Stages in the Research Process

Defining the Research Objectives

Exhibit 4.6 shows that the research process begins with research objectives. Research objectives are the goals to be achieved by conducting research. In consulting, the term deliverables is often used to describe the objectives to a research client. The genesis of the research objectives lies in the type of decision situation faced. The objectives may involve exploring the possibilities of enter- ing a new market. Alternatively, they may involve testing the effect of some policy change on employee job satisfaction. Different types of objectives lead to different types of research designs.

In applied business research, the objectives cannot really be determined until there is a clear understanding of the managerial decision to be made. This understanding must be shared between the actual decision maker and the lead researcher. We often describe this understanding as a prob- lem statement. In general usage, the word problem suggests that something has gone wrong. This isn’t always the case before research gets started. Actually, the research objective may be to simply clarify a situation, define an opportunity, or monitor and evaluate current business operations. The research objectives cannot be developed until managers and researchers have agreed on the actual

research objectives The goals to be achieved by con- ducting research.

deliverables

The term used often in consult- ing to describe research objec- tives to a research client.

p

D D D D

Deeeffffiiiin n niiiin n ng g g tttth h h heee R R R R Reeessseeeaaarrrccch h h h O O O O Ob b b b bjjjjjeeecccttttiiiivvveeesss

Exhibit 4.6 shows that the research process begins with research objectives. Research objectives

h l b hi d b d i h I l i h d li bl i f

research objectives EXHIBIT 4.6 Flowchart of the Business Research Process

Define research objectives

Selection of exploratory research

technique

Problem definition (statement of research objectives) Secondary

(historical) data

Previous

research Case

study

Selection of basic research

method

Experiment Laboratory Field Survey

Interview Questionnaire Problem Discovery

and Definition

Planning the Research Design

Selection of sample design

Data Gathering

Data Processing and Analysis

Drawing Conclusions and Preparing Report

Sampling

Experience survey

Observation Secondary

data study

Collection of (fieldwork)data

Editing and coding

data

processingData and analysis

Interpretation findingsof

Report  Probability

sampling Nonprobability sampling

Note: Diamond-shaped boxes indicate stages in the research process in which a choice of one or more techniques must be made. The dotted line indicates an alternative path that skips exploratory research.

93754_04_ch04_p050-074.indd 63

93754_04_ch04_p050-074.indd 63 7/11/09 10:27:53 PM7/11/09 10:27:53 PM

Experimental Design

Controlled experiment

The experimenter manipulates one or more independent vari- ables and holds constant all other independent variables while observing the effect on the dependent variable. In quasi- experiments, natural conditions that influence the variation are selected for observation.

I Experimental design:

– Manipulation of the independent variables.

– Selection and measurement of the dependent variables.

– Selection and assignment of experimental subjects.

– Control over extraneous variables.

I Experimental checks:

– ^Validity – Reliability

– Replicability

Experiments: Treatment & Control

I Manipulation of the independent variable:

– An experimental treatment refers to the way an experiment variable is manipulated.

– The treatment levels should be noticeably different and realistic.

– An experimental group refers to a group of subjects to whom an experimental treatment is administrated.

– A control group refers to a group of subjects to whom no experi- mental treatment is administrated.

– A placebo treatment is designed to appear like treatment but to have no real effect. The purpose of the placebo group is to account for the placebo effect, that is, effects from treatment that do not depend on the treatment itself.

I Selection and measurement of the dependent variable:

– The dependent variables must represent an outcome of interest.

Experiments: Random Assignment

I Selection and assignment: The participants selected for the ex- periment should be representative of the population of interest.

Imagine you want to test a new drug and ask sick patients to vol- unteer. The problem is that the self-selection process may induce a bias, e.g. the patients who volunteer may typically be in poorer health than the average patient and therefore not ‘representative’.

I Systematic error: Occurs if the sampling units in an experimental cell are different from the units in another cell, and this difference affects the dependent variable. To control for systematic error, randomization and matching can be used.

I Randomization: Random assignment of treatment to groups.

I Matching: Assigning participants in a way that their characteris- tics are the same each group.

Experiments: Control

I ^Control: Extraneous variables influence the relationship between the variables of interest in an experiment. These variables are undesirable because they add error to an experiment. Extraneous variables can produce confounded results.

I ^Confound: An alternative explanation, beyond the experimental variables, for observed differences in the dependent variable. A confound challenges the validity of the experiment.

I Internal validity: The extent that an experimental variable is truly responsible for any variance in the dependent variable.

I External validity: The accuracy with which experimental results

can be generalized beyond the experimental subjects.

Experiments: Observer Bias

I Observer bias: Errors associated with observations. Observers may accidentally introduce a bias by their presence alone — e.g.

the Placebo effect, the Pygmalion effect, the Hawthorne effect.

I Placebo effect: Patients’ belief in the efficacy of a medical inter- vention can improve their health. A randomly assigned placebo group is given an inert drug (sugar in water) to control for the placebo effect.

I Pygmalion effect: Teachers’ expectations can strongly affect stu- dent performance — a type of self-fulfilling prophecy.

Experiments: Hawthorne Effect

I Hawthorne effect: Subjects perform differently when they know they are experimental subjects.

I In a study of the effects of various working conditions on productiv- ity — such as hours of work, rest periods, lighting, and methods of pay — at the Western Electric Hawthorne plant in Cicero, Illinois, researchers found that workers’ productivity increased whether the work hours were lengthened or shortened, whether lighting was very bright or very dim, etc..

I Productivity increased because workers reacted positively to being

part of an experiment. This unintended effect is now known as the

Hawthorne effect.

Occam’s Razor

Occam’s Razor

Principle: Among competing hypotheses, the one with the fewest assumptions should be selected. Simpler theories are preferable because they are more amenable to testing.

I The principle of Occam’s razor is known in latin as lex parsimoniae

— The law of parsimony — attributed to English philosopher (and Franciscan friar) William of Ockham (c.1287–1347).

I Ptolemy (c.AD90–c.AD168) stated an equivalent principle: “We consider it a good principle to explain the phenomena by the sim- plest hypothesis possible.”

I Isaac Newton (1642-1726), “We are to admit no more causes of natural things than such as are both true and sufficient to explain their appearances. Therefore, to the same natural effects we must, as far as possible, assign the same causes.”

Research Ethics

I Informed consent: When an individual understands the purpose and consents to being a subject of research.

I Honest cooperation is the main obligation of the research partic- ipant. In return, the research participant has the right to expect privacy and confidentiality.

I Subjects should be informed of their right to be left alone or to break off the interview at any time.

I Objective reporting: The researcher should not misrepresent re- sults and should report difficulties, potential biases, limitations.

I The sponsor should respect participant confidentiality, not alter

data to meet a desired objective, not introduce bias into interpre-

tation of the study, not omit results, not make recommendations

beyond the scope of the study.

Research Ethics

I Leading questions: Designed to condition the subject about a particular issue, e.g. “Do you agree with killing innocent crea- tures? . . . Is it wrong to eat meat?”

I Loaded question: Suggests a socially desirable answer and/or is emotionally charged, e.g. “People have become more conscious about fuel emission. Would you consider buying an electric car?”

I Sponsored research: If the sponsor expects the research to sup- port certain types of results, it is clearly unethical. Mixing sales or fund-raising with research is unethical. Examples: The gun lobby and the effects of guns on gun violence; the oil industry lobby and the effects of emissions on global warming.

I Pseudo-research: Research conducted not to gather information with a true research purpose, but rather to bolster an established point of view.

Leading Questions: Yes Minister

Sir Humphrey: You know what happens: nice young lady comes up to you. Obviously you want to create a good impression, you don’t want to look a fool, do you?

Bernard Woolley: No

Sir Humphrey: So she starts asking you some questions: Mr. Woolley, are you worried about the number of young people without jobs?

Bernard Woolley: Yes

Sir Humphrey: Are you worried about the rise in crime among teenagers?

Bernard Woolley: Yes.

Sir Humphrey: Do you think there is lack of discipline in our Comprehensive Schools?

Bernard Woolley: Yes.

Sir Humphrey: Do you think young people welcome some authority and leadership in their lives?

Bernard Woolley: Yes.

Leading Questions: Yes Minister

Sir Humphrey: Do you think they respond to a challenge?

Bernard Woolley: Yes.

Sir Humphrey: Would you be in favour of reintroducing National Service?

Bernard Woolley: Oh, well I suppose I might.

Sir Humphrey: Yes or no?

Bernard Woolley: Yes.

Sir Humphrey: Of course you would, Bernard. After all you told you can’t say no to that. So they don’t mention the first five questions and they publish the last one. Bernard Woolley: Is that really what they do?

Sir Humphrey: Well, not the reputable ones, no, but there aren’t many of those. So alternatively the young lady can get the opposite result.

Bernard Woolley: How?

Sir Humphrey: Mr. Woolley, are you worried about the danger of war?

Bernard Woolley: Yes.

Sir Humphrey: Are you worried about the growth of armaments?

Leading Questions: Yes Minister

Bernard Woolley: Yes.

Sir Humphrey: Do you think there’s a danger in giving young people guns and teaching them how to kill?

Bernard Woolley: Yes.

Sir Humphrey: Do you think it’s wrong to force people to take arms against their will?

Bernard Woolley: Yes.

Sir Humphrey: Would you oppose the reintroduction of National Service?

Bernard Woolley: Yes.

Sir Humphrey: There you are, you see, Bernard. The perfect balanced

sample.

Surveys

I Survey: A survey is a method of collecting primary data based on communication with a sampling unit.

I Sampling units: A single element randomly selected from the population. Could be one person, a couple, a family.

I Respondent: Person who answers an interviewer’s questions.

I Objectives: describe events or opinion about particular activities, e.g. Identify characteristics of target markets, describe employee job satisfaction, describe the risk aversion of financial investor.

I Examples: Personal interviews, Telephone interviews, Mail sur- veys, Internet surveys, Self administrated questionnaires.

I Margin of error: A measure of the amount of random sampling error in a survey’s results. Sampling theory provides methods for calculating the probability that the survey results differ from the sampled population by more than a certain amount, simply due to chance. The margin of error only accounts for random sampling error, ignoring other sources of errors.

Errors in Surveys

I Random Sampling Error: Occurs because of chance variation in the sampling units selected for a particular sample.

I Systematic Error: Occurs because of some imperfect aspect of the research design or from persistent mistakes in execution.

I Sample Bias: Occurs when samples deviate systematically from the (unobserved) population. May be caused by flawed sample design or improper sampling procedure execution.

I Self-Selection Bias: Occurs because respondents who feel strongly about a subject are more likely to respond to survey questions than people who feel indifferent about it.

I Response Bias: Occurs when respondents either consciously or unconsciously tend to answer questions with a certain slant that misrepresents the truth.

I Nonresponse Error: Occurs when the sampling units selected

for a sample are not interviewed, because they are unavailable or

unwilling to take part.

Errors in Surveys: Respondents

I Unconscious misrepresentation: When a respondent is con- sciously trying to be truthful and cooperative, response bias can arise from the question format, the question content, or some other stimulus that affects their response to a question.

I Acquiescence Bias: A tendency to agree with most questions.

I Extremity Bias: A tendency to select extreme responses.

I Auspices Bias: Occurs when answers reflect the person’s liking or disliking of the sponsor organization.

I Social Desirability Bias: Occurs when respondents’ answers are biased by their desire, either conscious or unconscious, to be viewed favourably, e.g. over-reporting “good behavior”.

I Deliberate falsification: can sometimes be detected.

Errors in Surveys: Researchers

I Administrative error: An error caused by the improper adminis- tration or execution of the research task.

I Data-Processing Error: incorrect data entry, incorrect computer programming, or other procedural errors during data analysis.

I Interviewer Error: mistakes made by interviewers, failing to record survey responses correctly.

I Interviewer Cheating: filling in fake answers or falsifying ques-

tionnaires by an interviewer.

Questionnaires

I In a questionnaire, the questions must meet the basic criteria of relevance and accuracy.

I Questionnaire Relevancy: Collected information should address the research question.

I The researcher should be specific about data needs and have ra- tionale for each item requesting information.

I Irrelevant questions will make the survey needlessly long.

I Questionnaire Accuracy: Obtaining accurate answers from re- spondents depend on the questionnaire design, to facilitate recall and motivate respondents to cooperate.

Questionnaires

I Open-Ended Questions:

Pose some problem and ask respondents to answer in their own words. Example: “What things do you like most about studying in Hong Kong Baptist University?”

Are most beneficial in exploratory research, especially when the range of responses is not known.

High cost of administering open-ended response questions.

The analysis is subjected to the interpretation of the researcher

and may reflect a combination of the respondent’s and interviewer’s

ideas.

Questionnaires

I Fixed-Alternate Questions:

Questions in which respondents are given specific, limited-alternative responses and asked to choose the one closest to their own.

Researcher needs to know the potential responses to a question.

The alternative should be mutually exclusive and collectively ex- haustive.

Example: Would you say the labor quality in Hong Kong is higher, about the same, or not as good as it was 10 years ago? Higher / About the same / Not as good.

Require less interviewer skill, less time, is easier for the respon- dent to answer, and easier to analyze.

Questionnaires

I Avoid complexity:

Use simple, conversational language, be short and to the point.

Example: The stock market is too risky? True / False / No Opinion.

I Avoid ambiguity:

Be as specific as possible. Ambiguous question can be interpreted differently by different individual.

Example: “What media do you rely on most?” Television / Radio / Internet / Newspaper.

I Avoid double-barreled questions: A question that may induce error because it covers two issues at once.

Example: “Do you feel our hospital emergency room waiting area

is clean and comfortable?” Yes / No.

Questionnaires

I Avoid making assumptions:

Example: “What is your opinion on the new tax law to be imple- mented next year?” Respondents may not be informed!

I Avoid burdensome questions:

Example: The respondent is asked to process large amount of information and/or make complicated calculations.

I Generate variance: It is important that the response categories provided cover the breath of possibilities, but also critical that they yield variance across respondents. If most respondents give the same answer, hypothesis tests will be weak.

Case Studies

I Case studies: Up-close, in-depth, and detailed examination of an individual, group, organization, or event in its natural context.

I Typically, a case study focuses on the contextual analysis of a small number of critical events, e.g. the context leading up to a company’s initial public offering.

I A case study can help identify potential explanations of a particular success or failure.

I Origin of case studies: As early as 1870, Harvard Law School

professor Christopher Langdell began using cases pled before

courts as the basis for class discussions for a class in contract

law. By 1920, this practice had become the dominant pedagogical

approach in law schools throughout the United States

Experience Survey

I Experience Survey: Designed to gauge the overall satisfaction of a group of people who have shared a common experience.

I Businesses conduct experience surveys of their customers.

I Universities conduct experience surveys of their students.

I Experience surveys usually ask people to rate certain aspects of their experience on a scale or by selecting from a list of fixed- alternate answers, e.g. Yes/No answers. Open-ended questions may also be included.

I Experience surveys help identify areas that need improvement, reward employees that provide the best service, and collect in- formation from the respondents, e.g. “Have you flown with other airlines in the past year?”

Focus Group Interview

I Focus Group: A small group discussion led by a moderator who guides discussion among participants.

I Unstructured and free-flowing discussions: The moderator broadly steers the discussion in the intended direction, avoiding direct ques- tions and confrontations.

I Focus group interviews are used for learning about groups and their patterns of interaction.

I Focus group interviews can be used in the early stages of product or concept development, when organizations are trying to create an overall direction for marketing initiative.

I Focus group interviews are less expensive than other forms of tra- ditional marketing research and easy to execute.

I Sessions can be observed via two-way mirrors and recorded.

Focus Group Interview

I Ideal group composition: 6-10 people per session.

I Homogenous groups seem to work best because they allow re- searchers to concentrate on consumers with similar lifestyles, ex- periences and communication skills.

I Focus group interviews may not be useful for discussing sensitive topics in face-to-face situations.

I In focus group interviews, participants may not be representative of the target market.

Data Collection

I Primary data: Data collected for the current study.

I Secondary data: Data collected by someone else and for a pur- pose other than the current project. Pros & Cons: Secondary data is immediately available and more cheaply than primary data.

Secondary data may not match variables needed for the current study; it is difficult to verify its reliability; it may be dated.

I ^Examples: bookkeeping accounts, sales receipts, customer com- plaints, service records, publicly-available data from libraries, ven- dors, internet sales.

I Databases: Wilson Business Center, Hoovers, PROQUEST, IN-

FOTRAC, DIALOG (Dialog Information Services, Inc.), LEXIS-NEXIS,

and Dow Jones News Retrieval Services. Companies such as

Claritas, Urban Decision Systems, and CACI offer geographic de-

mographic databases.

Data Analysis

I Advances in computer technology have dramatically increased data collection, storage, and computational ability. Methods of identify- ing patterns in data include regression analysis, neural networks, cluster analysis, genetic algorithms.

I Data Mining: Analyze large amounts of data to discover patterns in the data. This is automated with computer algorithms.

I Not all patterns found by data mining algorithms are necessarily valid. Data mining can produce results which appear to be sig- nificant; but which cannot be used for prediction and cannot be reproduced on a new sample of data.

Key Terms & Concepts

exploratory research descriptive research causal research experiment design deductive reasoning inductive reasoning self-selection bias

randomization & matching random sampling

causal inference concomitant variation spurious association placebo effect

Hawthorne effect Occam’s razor

sample selection error respondent error

nonresponse error interviewer error acquiescence bias social desirability bias deliberate falsification conflict of interest case study

experience survey

focus group interview

Summary

1. A theory is a formal explanation of some events based on logical and causal links among different variables and predictions about the effects of changes in some variables.

2. A theory may follow deductive reasoning or inductive reasoning.

Deductive reasoning is the logical process of deriving a

conclusion about a specific instance based on a known general premise or something known to be true. Inductive reasoning is the logical process of establishing a general proposition on the basis of observation of particular facts.

3. The scientific method guides research from the abstract nature of concepts and propositions to the empirical variables and

hypotheses used to test the theory.

4. The scientific method include: (1) definition of objectives, (2) formulation of propositions and statement of hypotheses, (3) design of empirical tests, (4) data collection, (5) analysis of data, (6) conclusions and report.

Summary

5. Business research involves data gathering about customers, competitors, economic trends, employees, and other factors that affect business success. This assists in decisions ranging from long-range planning to near-term tactical decisions.

6. Computer-assisted databases make access to external data easier. Secondary data is data collected by someone else for purposes other than those of the current researcher. They are usually cheaper than primary data.

7. There are three general categories of secondary research

objectives: fact-finding, model building, and database marketing.

Model building is more complicated than fact-finding: it involves

specifying relationships between two or more variables. The

practice of database marketing involves maintaining customer

databases with customers’ names, addresses, phone numbers,

past purchases, responses to past promotional offers, and other

relevant data such as demographic and financial data.

Summary

8. Data mining is the use of powerful computers to dig through volumes of data to discover patterns about an organization’s customers and products. It is a broad term that applies to many different forms of analysis.

9. A focus group outline begins with introductory comments followed by a very general opening question that does not lead the

respondent. A skilled moderator can often lead the group without having to explicitly state the questions of interest.

10. The survey is a common tool for asking respondents questions.

Sample surveys can provide quick, inexpensive, and accurate information.

11. The typical survey is a descriptive research study with the

objective of measuring awareness, knowledge, behavior, opinions and attitudes, both inside and outside of the organization.

Common survey populations including customers, employees, suppliers and distributors.

Summary

12. Random sampling error is caused by chance variation and results in a sample that is not absolutely representative of the target population. Such errors are inevitable, but can be predicted using statistical methods.

13. Nonresponse error is caused by subjects’ failing to respond to a survey. Response bias occurs when a response to a

questionnaire is falsified or misrepresented, either intentionally or inadvertently. There are four specific categories of response bias:

acquiescence bias, extremity bias, interviewer bias, and social desirability bias. Administrative errors are another source of error.

14. Questionnaires may be structured, with limited choices of responses, or unstructured, to allow open-ended responses.

15. Longitudinal studies may involve contacting different sets of respondents or the same ones repeatedly. One form of

longitudinal study is the consumer panel. Consumer panels are

expensive to conduct.

Summary

16. Interviews can be conducted door-to-door, in shopping malls, by telephone. Personal interviews are a flexible method that allows researchers to use visual aids and various kinds of props.

However, the presence of an interviewer may influence subjects’

responses.

17. Door-to-door personal interviews can get high response rates, but they are more costly to administer. Telephone interviews are fast, but not all telephone numbers are listed in directories.

18. Mail questionnaires are less expensive than telephone or personal interviews, but are more vulnerable to nonresponse error. The Internet and other interactive media provide convenient ways for organizations to conduct surveys, but not everyone has Internet access.

19. Pretesting a questionnaire on a small sample of respondents is a useful way to discover problems while they can still be corrected.

Summary

20. In a given experimental condition, the levels of an independent variable are unique. As the independent variable changes, so does the dependent variable. Subjects may or may not be given the treatment.

21. Systematic experimental error occurs because sampling units in one experimental cell differ from those in another cell in a way that affects the dependent variable.

22. Randomization minimizes systematic experimental error. If research subjects are randomly assigned to different treatment combinations, then the differences that exist naturally within a population should also exist within each experimental cell.

23. In a between-subjects design, every subject receives only one

experimental treatment. In a within-subjects design, the subjects

receive multiple treatments. The key to randomization is to assign

subjects to experimental cells in a way that spreads extraneous

variables out evenly across every condition.

Practice Questions

What sources of error might be associated with the following?

1. In a survey of frequent fliers age 50 and older, researchers concluded that price does not play a significant role in airline travel because only 25 percent of the respondents check off price as the most important consideration in determining where and how they travel.

2. Researchers who must conduct a 45-minute personal interview decide to offer $25 to each respondent because they believe that people who will sell their opinions are more typical than someone who will talk to a stranger for 45 minutes.

3. A company’s sales representatives are asked what percentage of the time they spend making presentations to prospects, traveling, talking on the telephone, participating in meetings, working on the computer, and engaging in other on-the-job activities.

Practice Questions

What sources of error might be associated with the following?

4. A survey comes with a water hardness packet to test the hardness of the water in a respondent’s home. The packet

includes a color chart and a plastic strip to dip into hot water. The respondent is given instructions in six steps on how to compare the color of the plastic strip with the color chart that indicates water hardness.

5. A researcher investigating public health issues goes into a junior

high school classroom and asks the students if they have ever

smoked a cigarette. The students are asked to respond orally in

the presence of other students. What types of error might enter

into this process? What might be a better approach?

Practice Questions

Evaluate the following survey designs:

6. A researcher suggests mailing a small safe (a metal file box with a built-in lock) without the lock combination to respondents, with a note explaining that respondents will be called in a few days for a telephone interview. During the telephone interview, the

respondent is given the combination and the safe may be opened.

7. A shopping mall that wishes to evaluate its image places packets including a questionnaire, cover letter, and stamped return

envelope in the mall where customers can pick them up.

8. An e-mail message is sent to individuals who own computers, asking them to complete a questionnaire on a Web site.

Respondents answer the questions and then have the

opportunity to play a slot-machine game on the Web site. Each respondent is guaranteed a monetary incentive but has the option to increase it by playing the slot-machine game.

Practice Questions

Evaluate the following survey designs:

9. A mall intercept interviewing service is located in a regional shopping center. The facility contains a small room for television and movie presentations. Shoppers are used as sampling units.

However, mall intercept interviewers recruit additional subjects for television commercial experiments by offering them several

complimentary tickets for special sneak previews. Individuals contacted at the mall are allowed to bring up to five guests. In some cases the complimentary tickets are offered through ads in a local newspaper.

10. Time magazine opts to conduct a mail survey rather than a

telephone survey for a study to determine the demographic

characteristics and purchasing behavior of its subscribers.