Questionnaire, measures, reliability and validity

This study adopted a self-administrated questionnaire to gather findings. Wolf (2008) refers to a self-administered questionnaire as a questionnaire that has been designed specifically to be completed by a respondent without intervention by the researchers. The design of questions in this questionnaire closely corresponds to the research questions and hypotheses. The questionnaire includes four parts: Section A focuses on American television viewership and motivations of watching American TV series; Section B measures Chinese audiences’ perception of American reality and their cultural identity; Section C tested Chinese perceptions of their own cultural values and American cultural values; and Section D gathered demographic data from the viewers (Appendix D).

To ensure the validity and reliability of this study, the questionnaire was designed with simple and clear instructions (Nestor & Schutt, 2014), and avoided asking two questions at once or switching questions back and forth (Merrigan & Huston, 2009a). A pre-test or pilot study was run before conducting the full survey. The samples in the pre-test, which were used to test for clarity, are not part of the final sample.

The survey was designed in English, then translated into Chinese. The Chinese version of the questionnaire was used for the pilot study to improve the semantic validity of the survey before it was fully executed. The welcome statement and invitation letter clearly mentioned that the online survey would be investigating the Chinese audience’s exposure to American television and its perceived impact on them. The letter also mentioned that participation was completely voluntary and anonymous.

Viewing preference. In assessing the viewer’s preferences, respondents were asked: ‘do you watch American TV series?’ Non-American TV series viewers were only assessed according to cultural

values and their perception of Chinese reality. The data of American TV series viewers was separated from non-American series viewers. Of the respondents, 2, 509 reported that they watch American TV series and 506 reported that they do not watch American TV series.

Viewership. The viewership was measured by the total viewing time and length of viewing. The total viewing time was measured by asking respondents to recall the total time the spent viewing American TV series on an average weekday, and on typical weekends and holidays. The answer was on a scale ranging from 1 (rarely), 2 (less than one hour), 3 (one hour or more, but less than three hours), 4 (three hours or more, but less than five hours) to 5 (five hours or more). The mean value shows that the average respondents were watching American TV series for less than one hour on an average weekday (M=2.64, SD=1.08), and more than one hour but less than three hours on typical weekends and holidays (M=3.04, SD=1.10). By merging the two measurement, the Cronbach’s Alpha was .776, which means the two items have enough reliability and internal consistency, and thus could be merged into one variable (α < .50, unacceptable reliability; .60>α≥ .50, poor reliability; .70>α≥ .60, questionable reliability; .80>α≥ .70, acceptable reliability; 0.9 >α≥ 0.8, good reliability; α ≥ 0.9, excellent reliability) (DeVellis, 2012; Pallant, 2010). The length of viewing was measured by asking respondents how long they have been watching American TV series on a scale ranging from 1(0-3 months), 2 (3-6 months), 3 (6-12 months), 4 (1-2 years), 5 (2-3 years), 6 (3-4 years), 7 (4-5 years), to 8 (more than 5 years). The mean value shows that the average respondents watched American TV series for 1-2 years (M=4.71, SD=1.94).

Perception of American reality. In assessing perception of American reality, respondents were asked, ‘how accurate do you think American TV series reflects US society and culture?’ The results were measured on a 10-point scale, from 1 (not at all accurate) to 10 (very much accurate). The mean value shows that the average respondents think American TV series reflect US society and culture a little accurately (M=5.72, SD=1.93).

Gratifications. The gratifications was measured by Jiang and Leung (2012)’s scale of Chinese viewers’ gratifications sought from viewing American TV series. The Cronbach’s Alpha of the items for entertainment (α=.92), learning (α=.87), sociability (α=.92), escape (α=.83) shows the high reliability of the items measured. The measurement of gratifications was used a five-point Likert scale, ranging from 1 (strongly disagree), 2 (disagree), 3 (neutral), 4 (agree), to 5 (strongly agree). The mean value shows that the average respondents are more likely to watch American TV series for learning (M=3.78, SD=0.78) and entertainment (M=3.74, SD=0.74) than sociability (M=3.61,

SD=0.78) and escape (M=2.94, SD=0.84).

Cultural values. The measurement of seven Chinese values (family orientation, relationship by status, non-competitiveness, prudence, thrift, respect for tradition, sexual conservatism) was adapted from Chinese Culture Connection (1987)’s Chinese Value Survey (CVS) by ask the respondents about each values’ importance within statements and descriptions on a five-point Likert scale, ranging from 1 (strongly disagree/not at all like me), 2 (disagree/not like me), 3(neutral/some-what like me), 4 (agree/like me), to 5 (strongly agree/very much like me). The measurement of seven American values (individual orientation, equality, competition, adventurous, leisure, modernity, sexual liberation) was adapted from Hofstede (2001)’s research comparing cultural values by accessing respondents’ opinion about each item on a five-point Likert scale, ranging from 1 (strongly disagree/not at all like me), 2 (disagree/not like me), 3 (neutral/some-what like me), 4 (agree/like me), to 5 (strongly agree/very much like me). The reliability tests Cronbach’s Alpha conducted to measure Chinese values and American values were α=.71 and α=.70, respectively, which shows as reliably measured items. The overall Chinese value and the overall American value were the mean score of seven Chinese values and seven American values, respectively. The mean value shows that the average respondents are a little more favorable towards the overall American value (M=3.60, SD=0.51) than the overall Chinese value (M=3.27, SD=0.53).

Perception of Chinese reality. In assessing perceptions of Chinese reality, respondents were asked to report their satisfaction with the Chinese economy, society, culture and being a Chinese person on a 10-point Likert scale, ranging from 1 (not at all satisfied) to 10 (very much satisfied). The reliability test conducted to measure Chinese reality was .76, which is above the acceptable reliability standard. The mean value shows that the average respondents’ satisfaction with Chinese culture (M=3.28, SD=0.88) is the lowest among the variables, such as being a Chinese person (M=3.82, SD=0.88), the Chinese economy (M=3.41, SD=0.84), and the Chinese society (M=3.36, SD=0.81).

Demographic information. Demographic information, including gender, age, education level, wealth, and whether the viewer had visited the US or not were assessed by commonly accepted measures. The gender was coded by using 1 (male) or 2 (female). Age was assessed by a scale ranging from 1 (below age 18), 2 (18-25), 3 (26-30), 4 (31-40), 5 (41-50), 6 (51-60), to 7 (above 60). The viewer’s education level was coded using 1 (primary school), 2 (junior high school), 3 (senior high school), 4 (college or university level with an undergraduate degree), to 5 (university

level with a postgraduate degree). In coding whether the viewer had visited the US, respondents were asked ‘have you been to the US’? The answer was measured by 1 (yes) or 2 (no). Their wealth was measured by the self-assessment on a 10 point Likert Scale, ranging from 1 (poor) to 10 (rich). The mean value shows that the respondents are mainly female (M=1.7, SD=0.46), 26-30 years old (M=2.17, SD=0.49), with a college or university degree (M=4.07, SD=0.50), have not visited the US (M=1.94, SD=0.23), and are a little wealthier than average (M=5.39, SD=1.91).

4.3.4 Pilot study

Before conducting the survey on a large population, a pilot study was conducted to improve the questionnaire. The pilot study was crucial for adjusting the survey questions and gathering feedback from participants. The pilot study also helped to refine variables.

First, pilot study helped the researcher to choose the right online survey website. This study selected Sojump.com rather than others as the survey website because it is the largest survey site for research purposes in China. It covers more than 90 percent of Chinese higher education institutes and consulting companies such as Tsinghua University, Beijing University, Chinese University Hong Kong, PWC, and McKinsey. Sojump.com, founded in 2006, has 20 million users and has collected 1,346 million responses since its founding4_.

Second, the original questionnaire had many more questions than the final one. To avoid survey fatigue and make the survey more efficient, the researcher refined the questions based on feedback and focused on four variables: viewership, gratifications, perceptions of cultural values, and perceptions of realities. The perception of Chinese reality was intended to discover the audiences’ opinion about their reality, which includes the Chinese economy, culture, society, and being a Chinese person. The original set of questions also included a question on the Chinese political environment; however, due to censorship of the Internet, political questions did not pass the screening on the survey website, causing the questionnaire to be suspended. Therefore, the researcher had to delete that question to continue the survey creation process.

4.4 Quasi-experiment