Content Analysis

Krippenddorf (1981) defines content analysis as a research method for “making replicable and valid inferences from data to their context”. According to Kripendorf (1981) “in content analysis, data result from the procedures the researcher has chosen to answer specific questions concerning phenomena in the context of given texts.

Hence, data are made, not found, and researchers are obligated to say how they made their data”. Neuendorf (2002) explains content analysis as the “summarising, quantitative analysis of messages that relies on the scientific method”. She suggests that it is “not limited as to the types of variables that may be measured or the context in which the messages are created or presented” (Neuendorf, 2002). Content analysis method is applied to a wide range of material; namely “the texts to which meanings are conventionally attributed” (Krippendorff, 1989). These texts can vary from

written texts or documents to transcribed speech, verbal interactions and discourses, visual images or representations, and to “characterisations, nonverbal behaviours, sound events, or any other message type” (Neuendorf, 2002; Krippendorff, 1989).

Examples of such texts include “childrens’ talk, answers to open-ended interview questions, computer conferences”, “websites” and also, “newpapers, magazines, books, radio broadcasts, films, comics, television programming” (Neuendorf, 2002;

Krippendorff, 1989). In other words, content analysis can be used to analyse the content of any piece of communication, independent of the subject that is communicated and the media through which the communication is conducted. In this study, content analysis is used in order to analyse the content of the Facebook group pages used throughout the cycles of the implementation; the interviews and the open-ended questions in the questionnaires.

According to Neuendorf (2002) content analysis can be explained in terms of its six characteristics that: it “relies on scientific method”; the message is “the unit of analysis or the unit of data or both”; it is “quantitative”; it is “summarising”; it can be applied to “all contexts”; and “all message characteristics” can be content analysed”.

First of all, content analysis is based on scientific method, which includes

“objectivity-intersubjectivity, a priori design, reliability, validity, generalisibility, replicability, and hypothesis testing” (Neuendorf, 2002). Objectivity requires that the research process is independent of the researcher’s biases; “that it does not matter who performs the analysis or where and when” (Krippendorff, 1989). Neundorf (2002) emphasises the difference between objectivity and intersubjectivity. On the basis of the view that “all human inquiry is inherently subjective”, the sceintific method can be decribed in terms of intersubjectivity, which, instead of posing the question “Is it true?”, poses the question “Do we agree that it is true?” (Neuendorf, 2002). In this study, intersubjectivity is aimed to be addressed by laying out all the details of the context and process of analysis, with the decision taken and the steps used for the analysis. A priori design refers to the planning of the analysis, such as the “variables, their measurements, coding rules” to be made beforehand (Neuendorf, 2002). Though such planning is, and needs to be, flexible throughout the analysis, a process which is described as “a combination of induction and deduction”

(Neuendorf, 2002). The content analysis of the Facebook groups in this study is based on the questions, units and variables determined in the beginning of the

analysis process, as discussed in the next chapter. Reliability, a key concept in scientific method and in content analysis, suggests that “when other researchers, at different points in time and perhaps under different circumstances, apply the same technique to the same data, the results must be the same” (Krippendorff, 1981). In content analysis, one of the key issues related to reliability is “intercoder reliability”, which is not within the context of this study, as there is only one coder: the researcher. Apart from the coders, reliability depends on the coding scheme, which is vital in this study. Krippendorff (1981) defines three types of reliability design:

stability, reproducibility, and accuracy. Reliability is defined as the “necessary but not sufficient condition” of the next characteristic of the scientific method: validity (Neuendorf, 2002; Krippendorff, 1981). Validity of a research refers to its findings being acceptable as “indisputable facts” (Krippendorff, 1981). Krippendorff (1981) relates the term with “empirical truth, predictive accuracy, and consistency with established knowledge”. Accroding to Krippendorff (1989) “content analysis may also parallel other research techniques and check or shed light on the validity of either’s findings”. The parallel uses of different methods “enhance the analysts’

confidence in the validity of their findings and justify their substitutability”

(Krippendorff, 1989). In this study the content analysis of the Facebook groups, the answers to the interviews and the questionnaires are used all together to evaluate the results and findings of the implementation. Generalisability refers to the applicability of a research to broader ranges of study, such as the “population” from which the sample of inquiry was drawn (Neuendorf, 2002). Population is the “set of units being studied, the set of units to which the researcher wishes to generalise” and is defined by the researcher (Neuendorf, 2002). Here, the “representativeness” of the sample within the population is important (Neuendorf, 2002). Replicability suggests that when the research study is repeated in a different context similar results can be achieved (Neuendorf, 2002). Finally, the scientific method requires that a hypothesis and a research question are generated before the research process starts (Neuendorf, 2002).

The second characteristic of content analysis is that the message is “the unit of analysis, the unit of data, or both” (Neuendorf, 2002). A unit is an identifiable message or message component, and can be in the form of “words, characters, themes, time periods, interactions, or any other result of “breaking up a

‘communication’ into bits” (Carney, 1971, p.52)” (Neuendorf, 2002). There are three types of units in content analysis: units of sampling or sampling units; units of data collection or recording units; and units of analysis or recording units” (Neuendorf, 2002; Krippendorff, 1981). Units of sampling or sampling units “serve as the basis for identifying the population and drawing a sample” (Neuendorf, 2002). They are

“those parts of observed reality or of the stream of source language expressions that are regarded independent of each other” (Krippendorff, 1981). “Sampling units make possible the drawing of a statistically representative sample from a population of potentially available data” (Kripendorf, 1989). Units of data collection or recording units are those “on which variables are measured” (Neuendorf, 2002). Krippendorff (1981) suggests that they are “separately described and can therefore be regarded as the separately analysable parts of a sampling unit”. He (1981) quotes Holsti (1962) who defines recording units as “the specific segment of content that is characterised by placing it in a given category”. Recording units are defined within sampling units because sampling units are usually “too large, too rich, or too complex to serve as a unit for description” (Krippendorff, 1981). In some cases, recording units can be defined as the co-occurrence of smaller units within one bigger recording unit.

Finally, the units of analysis or the context units, “serve as the basis for reporting analyses” (Neuendorf, 2002). According to Krippendorff (1981), context units “set limits to the contextual information that may enter the description of a recording unit”. The three kinds of units, the populations and the variables specific to the content analyses made in this study are defined and described, in detail, in the next chapter.

The third characteristic of content analysis is that it is quantitative, which suggests that it is based on numerical mesaurements of the data. The data in hand is categorised according to the research theme, and the units are counted in order to find meaning within the general context. Fourthly, content analysis is summarising;

in other words, it aims to achieve “generalisable conclusions” instead of presenting

“all details concerning a message” (Neuendorf, 2002). Fifth property of content analysis, as mentioned above, is its applicability to all texts or contexts. Under this topic, Neuendorf (2002) lists a number of different contexts, such as “individual messaging, interpersonal and group messaging, organisational messaging, mass messaging, and, applied contexts”. Finally, all types of messages can be the subject

matter of content analysis, which are defined by Neuendorf (2002) as “manifest versus latent, content versus form, and text versus other types”.

Krippendorff (1989) defines six steps in the process of making content analysis:

“design; unitising, sampling, coding, drawing inferences and validation”. The first step, the design, is described as a “conceptual step”. Here, the “framework of the analysis” is formed by defining the “context” in three features: defining what the desired information is which cannot be observed directly; “exploring the source of relevant data that either are or may become available”; and “adopting an analytical construct that formalises the knowledge available about the data-context relationship, thereby justifying the inferential step involved in going from one to the other”

Krippendorff (1989). The second step is unitising, where the sampling, recording and context units, as described above, are defined. Sampling is “the process of selecting a subset of units for study from the larger population” (Neuendorf, 2002). Here, the focus is to “undo the statistical biases”, and also, “to ensure that the hierarchy of chosen sampling units becomes representative” (Krippendorff, 1989). Neuendorf (2002) defined two types of sampling: “random and non-random sampling”. In order to be able to generalise the findings of a content analysis “to some population of messages, the sample for the analysis should be randomly selected” (Neuendorf, 2002). The findings of non-random sampling are not generalisable to the population.

Coding, the step, which includes an “elementary notion of meaning”, involves the

“classification of the recording units in terms of the categories of the analytical constructs chosen” (Krippendorff, 1989). The fifth step, which is drawing inferences,

“applies the stable knowledge about how the variable accounts of coded data related to the phenomena the researcher wants to know about” (Krippendorff, 1989).

Validation in content analysis, which is the final step, “is limited by the intention of the technique to infer what cannot be observed directly and for which validating evidence is not readily available” (Krippendorff, 1989).

In this study, a combination of random and non-random sampling techniques is used in the implementation stage. First of all, in order to establish the implementation sites, a nonrandom, purposive or judgement sampling is made in favour of all the departments of industrial design in all state universities in Turkey, out of the population of all the departments of industrial design in all universities in Turkey.

The results of the analysis are not aimed to be generalised to all departments in all

universities, thus the population is redefined as the departments of industrial design in all state universities in Turkey. All departments in state universities are contacted for the Facebook group implementation in their design studios. The departments that accept to be part of the implementation are the selection of the first stage of the sampling. After establishing the implementation sites, the Facebook groups are used in each of them; while also, interviews and questionniares are held with teachers and students of the departments. In sampling the interviewees out of the whole populations of teachers, random sampling method was used. In sampling the students, the teachers of the studio were asked for their advice in the selection of students to be interviewed. In the 3^rd year studio of the second cycle, a socially active student was adviced. In the 3^rd year studio of the third cycle, a student who is very positive about the use of Facebook group, a student who was very negative about it, and one with neutral opinions were interviewed.

4.4 Interview

The interview is “a face-to-face verbal interchange in which one person, the interviewer, attempts to elicit information or expressions of opinions or belief from another person or persons (Macoby and Macoby, 1954)” (Denzin, 2009). There are three kinds of interviews, which are based on how structured, or standardised the content of the interview is, and also the characteristics of the group of interviewees.

The first type of interview is the “schedule standardised interview in which the wording and order of all the questions is exactly the same for every respondent”

(Denzin 2009). The respondents should have a common vocabulary and infer the same meaning from the standard wording; in other words they need to be

“homogenous group of samples” (Denzin, 2009). Denzin (2009) refers to Richardson, Dohrenwend, and Klein (1965) who suggest that in standardised interviews, not only the meaning and the context of each question but also their sequence must be identical. In this type of interview, the interviewer intends to receive the same set of information from all respondents; and, according to Macoby and Macoby (1954), it is “the best suited for hypothesis testing and rigourous qualification of results” (Denzin, 2009). The second type is the “non-schedule standardised interview”, in which the content of the questions is the same for each respondent, but the phrasing and the order of questions differ (Denzin, 2009). In the

non-standardised interview, which is the third type, there are no specific set of questions nor a schedule. In this study, interviews were held with staff members and students of the two implementations sites. The first set of interviews were non-scheduled and non-standardised. They were exploratory, in the form of casual talks with the staff members in ITU, with no set questions. The second set of interviews can be defined as both explanatory and exploratory, and they were more structured.

The intention of the interviews was to receive the same set of information from all interviewees. The staff members of each site were homogenous sample groups, individually, while students were homogenous within themselves. However, all interviewees also form quite a homogenous group, considering the subject of industrial design studio course. The interviews were scheduled standardised within the 3 different sample groups; namely, the staff members of ITU, the staff members of MSUFA, and the students in ITU.

Another concern in interview as a method, is the questions asked. Denzin (2009) suggests that the questions should “accurately convey meaning to the respondent”, motivate the respondent, be clear, and be precise. The questions in the interviews in this study were prepared to cover all the situations in the design studio course. The interviews were presented in sections each referring to a topic, and then detailed questions were introduced under each topic. The questions were scheduled and standardised, but the interviews were held in a flexible manner, where the order of the questions could change, and the interviewee could lead the interview, if s/he wanted. In other words, the interviews were conducted in a flexible schedule of topics and questions. Denzin (2009) emphasises the common deficiencies of the three types of interviews. The first one is the “difficulty of penetrating a group’s language and mechanisms of symbolisation” (Denzin, 2009). Secondly, the interviewees may not want to tell the straight answer. And, thirdly, the hieararchy in the group of interviewees may create answers with different values or meanings within the group itself. In the interviews conducted in this study, the interviewer (the reseacher) shared very similar backgrounds, especially with the staff members and students in ITU: she studied the same subject in a department with very similar structure. She also worked in a design department as a research assistant and a lecturer, which gave her the insight of the staff members. In MSUFA, the educational system is different than the experience of the researcher, but the interviewees had

similar background which made it easier to share a common language. In ITU, the number of interviewees, and the friendly relationships of the researcher with some of the interviewees were ways of addressing the shortcoming of interviewees not wanting to give some information, and also, having different values or meanings to their words. In MSFUA, such deficiencies were perceived, but were difficult to address, as an outsider to the department.

4.5 Survey

Survey is a technique which “requires the systematic collection of data from populations, or samples through the use of the interview or the self-administered questionnaire” (Denzin, 2009). Denzin (2009) defines six types of survey designs, which include three common characteristics: “the use of interviewing and/or questionnaires as the major mode of data collection; collection of data from large numbers of persons; and the use of multivariate analysis as the major method of data analysis”. Furthermore, these six types of designs differ from each other in that they may or may not have the following features: “randomisation; comparison groups”;

and “repeated observations” (Denzin, 2009).

The six survey designs fall under two main topics: the non-experimental survey designs and the quasi-experimental survey designs. The non-experimental surveys lack “one or more of the critical features of the experimental model”, which are

“randomisation, control groups, repeated observations, investigator control over experimental variables” (Denzin, 2009). There are three types of non-experimental surveys: the one-shot case study; the one-group pre-test-post-test design; and the static-group comparison. In one-shot case study, the “analyst randomly samples from a larger population a group of subjects who have been exposed to a series of critical events” (Denzin, 2009). There are no control groups and also only one observation is recorded. Though, because there is only one observation, the problems of “reactivity, time and subject-observer changes are reduced” (Denzin, 2009). The one-group pre-test-post-test survey includes two sets of observation on the same group of samples.

This design “lacks a control group” and “reactive effects of repeated observation”

and also the problem of “changes in the subject and observer” are present (Denzin, 2009). In the static-group comparison, two random or non-random groups are observed together, for once. The quasi-experimental surveys involve “repeated

observations, randomisation, a focus on naturally induced experimental treatments, an the optional use of comparison groups” (Denzin, 2009). Similarly, there are three types of quasi-experimental survey designs: the same-group recurrent-time-series survey without comparison groups; the different-group recurrent-time series survey without comparison groups; and the same-group recurrent-time series survey with comparison groups design. In the same-group recurrent-time-series survey without comparison groups the same group of samples are observed repeatedly with no control groups (Denzin, 2009). It is a strong design, as the continuous observations of the same group “increases the investigator’s ability to adopt the perspective of those stuied” (Denzin, 2009). The different-group recurrent-time series survey without comparison groups differs from the first one, in that here the investigator observes a different sample group, in each observation. This type of design aims to overcome the problem of “repeated observations”, which may cause “new behaviours” in the samples (Denzin, 2009). In the same-group recurrent-time series survey with comparison groups, the investigator makes multiple observations, for each of which a comparison group is created. This way, the investigator can both observe changes in the sample group, and also detect and exclude the “reactive effects” of repeated observations. Denzin (2009) defines this survey design as the most sophisticated of all.

In this study two types of survey designs are used. In the first and third cycles of the

In this study two types of survey designs are used. In the first and third cycles of the