sorts around a factor

The interviews were undertaken with councillors, officers and citizens who were directly involved in the two planning processes, ensuring the research was “focused on issues which are mostly or wholly raised by the participants, rather than the researcher” (Barry & Proops, 1999:339). The interviews were audio-taped and transcribed. Statements in the transcripts were sampled for the concourse of statements, as described below. Material from the six interviews was also used to build a picture of the two wastewater planning processes, and as supporting evidence for factor interpretation.

Filling ‘gaps’ in the concourse

Where there were gaps in the issues covered in interviews, additional statements were generated from media reports, written submissions, consultants’ reports and council documents that related to the two processes (see Appendix Six for a full list of statements). Where gaps remained in this material, further statements were sampled from published sources on environmental policy and public policy issues5. These statements generally addressed less contextually specific, philosophical and public policy issues. This strategy followed a common approach to concourse development (McKeown & Thomas, 1988; Steelman and Maguire, 1999), but it does require some theoretical expansion.

Concourse development was informed by a combination of deductive and inductive approaches. The concourse reflected actors’ perspectives on the two wastewater planning processes and on environmental issues and planning and decision-making more generally. Its design was initially based on deductive theoretical considerations – from the literature review it was clear that by far the least well-understood aspects of planning and decision-making were actors’ perspectives, comprising both epistemological and procedural dimensions. Those dimensions were further divided into the four-fold typology.

Further structured sampling was undertaken as described earlier in this chapter. Brown (1980:38) maintains that structured statement samples that are bounded by theoretical considerations ensure that balance is achieved in the distribution of statements. This also ensures that, as far as possible respondents with different points of view will have an equal opportunity to express their views through statement sorting.

5

The sources used were Dayton, 2000; Hayward, 2000; Focht & Lawler, 2000; Webler, Tuler, & Krueger, 2001, Memon, 2000; Tenbensel, 2000.

Concourse development reflected Barry and Proops’ research (1999:340) where the “overwhelming” source of statements was contextually-grounded material but where statements were also used from academic literature to fill gaps in the concourse. I used contextually grounded statements where possible, to guard against any unnecessary intervention by the researcher. As McKeown and Thomas (1988:25) maintain, naturalistic samples “reduce the risk of missing the respondents’ meanings or confusing them with alternative meanings derived from an external frame of reference”. They further note that “hybrid” concourses can be developed, comprising items from naturalistic samples and other sources, maintaining that “neither is inherently superior to the other; one should select the type best suited to the research at hand” (ibid:27).

In this study, “gaps” were identified in the concourse in two particular areas: (a) statements of values on actors’ roles in public participatory processes, and (b) statements of values on the relative importance of different substantive considerations in environmental decision-making. The structured interview schedule did not solicit enough statements on these more philosophical and overarching aspects of the issues under study, to allow their canvassing in the Q sorts. These gaps were filled with statements from published sources (listed in the footnote).

From interview transcripts, it appeared that interviewees had simply reflected less on the more rarified philosophical aspects of the issue under study, and perhaps unsurprisingly, had focused more closely on context-specific issues. However, my research question sought data on commonalities and differences between the two wastewater studies, and on wider issues for planning and decision-making. In filling those “gaps” I followed Brown’s (1980) advice that those issues not seen as important or applicable to respondents (in the particular context in which they were sorting statements), would not feature significantly in sorting or factor analysis. My conditions of instruction also required respondents to sort statements to reflect their relative importance, including into a “neutral and/or no opinion” pile. During sorting and in post- sort interviews, it was also generally clear from comments and interactions which statements respondents had fitted the neutral/no opinion category.

As Brown (1980) further notes, if some respondents for whom the statement does not seem applicable do actually respond to the statement either positively or negatively, then clearly the statement did have applicability to them. This reflects the extent to which individual statements “take on meaning in relation to the whole factor array”

(Brown, 1980:251). Thus, the data gathered from Q sorting may lead in quite different directions to that initially theorised as important.

The statement sample

A concourse of statements reflects the “volume of discussion on any topic” (Dryzek & Berejikian, 1993:50). It typically comprises at least 200-300 statements, making it too large to administer as a Q-sort. Brown (1980) and McKeown and Thomas (1988) recommend a structured sampling process for reducing the number of statements to a manageable size for Q-sorting. Structured samples are composed inductively, emerging from patterns and themes that are observed as statements are collected. Samples can also be composed deductively, in line with the researcher’s particular theoretical focus (Brown, 1980:38).

The selection of samples follows the principles of homogeneity (in the initial grouping of the statements into categories of similar statements), and heterogeneity (in the subsequent selection of those most different from one another within a category). As Brown (1980:189) notes:

Selecting the most un-alike statements from those which are alike in kind serves to minimise the constraining effects of the design and tends to produce a sample of stimuli more nearly approximating the complexity of the phenomenon under investigation.

Final samples of statements typically number 36-80 in size, as a larger sample would make sorting statements an unwieldy and lengthy process (Barry & Proops, 1999; Dryzek and Berejikian, 1993). Relatively wide-ranging statements can easily be rank- ordered by respondents. Statements are generally presented without amendment, as, although the researcher defines the domain of study, they seek to avoid further inputs that might determine the scope of the concourse. Thus, Dryzek and Berejikian (1993:50) describe Q-methodology as a “reconstructive methodology” since the principal analytical tools are stakeholders’ unaltered statements or “reality constructions”.

My statement sample structure was similar to the concourse matrices employed by Dryzek and Berejikian (1993), Barry and Proops (1999), and Dayton (2000). I followed Brown’s advice (1980:189) in examining the range of statements in the concourse and

clustering them into key themes. The initial set of over 300 statements was reduced to 181 statements (averaging 20 words each), by removing statements that unnecessarily duplicated themes. I identified four basic themes and divided the 181 statements into those themes:

1. Processes (e.g. format of decision-making, consultation, issues of equity and legitimacy, balancing of issues) (68 statements).

2. Actors (e.g. relationships, who is included or excluded, status) (25 statements). 3. Substantive issues and their role and status (e.g. technology, science,

environmental and financial matters) (39 statements).

4. Subjective personal and corporate values, and their role and status (e.g. environmental, spiritual, capitalist) (49 statements).

Within those themes there were also clear differences in the types of views being expressed. I replicated one dimension of Dryzek and Berejikian’s (1993) 4x4 matrix in employing the following categories for statements:

1. Definitive (statement about the meaning of something, or how something is) 2. Designative (issues of fact, claims to be factual)

3. Evaluative (expressing the worth of something)

4. Advocative (opinion; how something should or should not be)

This gave me a 4x4 matrix for sorting the 181 statements (see Appendix Six for the full set of statements and matrix). Once sorted, each of the 16 cells in the matrix contained at least 5 statements, indicating a reasonable spread of statements. A final sample of 43 statements was taken from this. Because respondents were required to rank-order statements in a forced distribution, the final sample was fixed to fit that distribution of 43 statements.

I decided to administer a combined Q-sort to all respondents from both wastewater planning processes in order to simplify the process and the analysis. This followed other researchers who had administered a single set of statements on a single issue to multiple, geographically discrete, sometimes widespread communities (see for example, Webler et al., 2001). In order for the Q-sorts to be applicable to all respondents, statements had to accurately reflect the relevant issues in each wastewater planning process, without losing applicability to the other process. In the

event, given the strong similarities between planning processes, most statements were equally applicable to either process.

A pilot study was undertaken using the Q-sort of 43 statements, with respondents who had some knowledge of the two wastewater planning processes and of issues related to environmental planning in general. Respondents were asked for advice on the utility of the Q-sort cards and distribution sheet, ease of following instructions, and so forth.

The respondent sample

Q-sorts are typically administered to small groups of sampled respondents who often comprise a large percentage, if not all the population of stakeholders involved in a particular activity. For example, a Q-methodological study of a public policy-making process might survey 30 respondents from a population of 50 directly involved stakeholders. A review of Q-methodological studies showed that the use of small samples in intensive studies is common practice, in contrast to conventional R studies, where the emphasis is often on gaining a statistically generalisable sample size6. A small respondent sample is adequate (Brown, 1980), as Barry and Proops (1999:344) maintain that:

…as few as 12 participants can generate statistically meaningful results, in terms of the range of implicit discourses uncovered. The reason for this is that each participant’s Q-sort provides a very large amount of information.

An adequate respondent sample is ultimately measured by having statistically well- defined factors. Definition is expressed by ‘factor loadings’ that indicate the percentage of correlation of each individual respondent’s Q-sort to a particular factor. Computer programmes such as PQMethod generate factor loadings for each respondent, identifying which loadings are ‘significant’. Brown (ibid.) maintains that a factor should have four or five significant loadings in order to be considered well-defined. Where factors lack definition, further sampling and Q-sorting by respondents can be

6

A review of 13 published Q studies in the area of public policy and environmental planning showed sample sizes being used of between 15 and 141 respondents, with an average of 60, and five samples of 40 or less. The 13 studies and number of respondents for each were: Brewer et al. (2000) 74 Q-sorts; Dayton (2000) 30 Q-sorts; Dryzek and Berejikian (1993) 37 Q-sorts; Steelman (2000) 15 Q-sorts; Focht and Lawler (2000) 108 Q-sorts; Webler et al. (2001) 27 Q-sorts; Dryzek and Braithwaite (2000) 60 Q- sorts; Fairweather (1994) 77 Q-sorts; Van Eeten (2001) 38 Q-sorts; Pelletier et al. (1999) 141 Q-sorts; Fairweather and Swaffield (2000) 88 Q-sorts; Fairweather and Swaffield (2001) 66 Q-sorts.

undertaken. The goal is not a set number of respondents, but a sufficient number for good factor definition.

My fieldwork for the two wastewater planning processes used a sample of 24 respondents, comprising five councillors, five council officers, and fourteen individual citizens or members of interest groups from both wastewater planning processes. These are described in the section on factor analysis.

The Q-sort

The Q-sort was tested before its actual administration on a small number of people who were informed about the two wastewater planning processes. The focus of testing was on the practicality and manageability of the Q-sort, including issues such as the amount of time taken, the number of statements, duplication of statements, and clarity of instruction.

I chose to conduct the 24 Q-sorts face-to-face7, and to conduct an unstructured interview with each respondent immediately following the Q-sort. Notes were taken of interviews and questions were not pre-planned. Interviews sought elaboration from respondents on particular sorting decisions of interest, and on comments they had made or aspects of demeanour that indicated a particular point of interest and so forth. The interview time was also an opportunity to respond directly to any problems or queries from respondents.

The process of administering a Q-sort is relatively simple. Respondents undertake the Q-sort with minimum interaction with the researcher under set conditions of instruction that vary little from one instance to another.

7

Q-sorts can also be administered via postal surveys, Internet websites, and with focus groups, where each individual completes a Q-sort following group discussion (Steelman and Maguire, 1999:384)

In this research the following protocol was adopted.

1. Respondents were presented with a set of statements, each printed on a separate card, rather like a deck of playing cards.

(1) Those who dispute the scientific evidence are now few and far between.

(2) The continued resistance to the proposal is from a minority of the population who are radicals and extremists.

Figure 3: Example of Q-sort cards used in this study

2. Respondents read through the statements to gain a broad impression of their overall content. At the same time, they divided the cards into three groups, according to whether they agreed, disagreed, or were neutral or had no opinion. 3. After this initial grouping of statements, respondents moved on to make more

detailed distinctions between the statements in each group.

4. Respondents were instructed to spread out the ‘agree with’ statements, then to read through them again, and to select the three of those available, which they most agreed with. These were placed on the illustrative score sheet in the ‘+4’ column (see Figure 3 on the following page).

5. Next the respondents spread out the ‘disagree with’ statements, and read through them, selecting the three with which they most disagreed, and placing them on the score sheet in the ‘-4’ column.

6. This was followed by the four next-most agreed with statements in the ‘+3’ column, followed by the four next most disagreed with statements in the ‘-3’ column, and so forth, working back and forwards between ‘agree with’ and ‘disagree with’.

7. Finally, the statements that were neutral or no opinion were included in the remaining spaces between ‘agree’ and ‘disagree’ (Those spaces would not necessarily correspond to the ‘0’ point at the middle of the range).

8. The respondents then re-examined the entire array to make certain it represented their views adequately, and to make any adjustments desired.

9. The statement scores were then recorded in the relevant cells on a copy of the score sheet.

Most disagree

Most Agree

-4 -3 -2 -1 0 +1 +2 +3 +4

Figure 2: The Q-sort Score Sheet

A forced distribution8, as shown in Figure 3, encourages respondents to think about the relationship between statements more systematically (Van Eeten, 2001). Effectively, respondents are forced to make choices between all statements, rather than simply say they ‘agree with’ a number of statements. Through that ranking process, finer distinctions are revealed between respondents’ subjective choices.

The rank ordering process reveals one of the strengths of Q-methodology when compared to a conventional ranking technique such as a Likert scale, which simply requires discrete responses on a range of items (e.g. strongly agree, agree, neutral). Those responses reveal little about the relationship between statements, and the overall meaning often remains obscure. When asked to rank order statements in relation to each other (as in a Q-sort), greater discrimination is required. Thus, the respondent begins to reveal a more comprehensive and nuanced overall perspective.

Q-sorting is one of the two specific methods that set Q-methodology apart from most conventional surveying processes. The other is the ‘inverted’ statistical factor analysis process.

8

A range of possible distributions can be used, depending on the total number of statements and preferred distribution “curve”. A relatively “normal” distribution curve fits those topics where participants are expected to have limited knowledge or interest, and to therefore be neutral/no opinion on a greater number of statements. For a highly controversial issue, or one on which participants are knowledgeable and opinionated, a more “flattened' distribution is appropriate. This provides more opportunities for participants to agree or disagree, and therefore, fewer opportunities for neutral responses (Brown, 1980).

Factor analysis

Factor analysis is a method for classifying variables that is used in both R- methodological and Q-methodological studies. A factor is a “dimension or construct which is a condensed statement of the relationship between a set of variables” (Kline, 1994:5). In Q-methodology the variables are the respondents’ perspectives, as revealed and made measurable through their Q-sorts. Individual Q-sorts are correlated in order to identify similarities and differences. A factor is a model Q-sort that presents an abstract composite of the perspective revealed from among similar Q-sorts. Factor analysis allows the researcher to “determine how persons have classified themselves” through their Q-sorts (Brown, 1980:208). Common or sympathetic viewpoints should cluster on the same factor, whereas differing viewpoints should appear on different factors. It can be useful to picture the factor as an ideal Q-sort—an array of statement cards on the score sheet that most closely models a group of like-minded respondents’ Q-sorts.

‘PQMethod’ was the statistical computer programme used for the factor analysis of my respondents’ Q-sorts. Factors were extracted using the centroid analysis option9. A review of Q-methodology researchers confirmed the strong preference for the centroid extraction method10, and this was used in this study.

Seven factors were initially extracted from the data. PQMethod offers a default number of seven factors to be extracted. This does not affect the number of factors the researcher can ultimately choose to use for analysis (Schmolk, 2002). PQMethod is designed so that the addition of each extra factor does not affect the composition of the already extracted factors in any way.

Whether or not a factor should be taken into account in analysis depends on its ‘significance’. Factor significance is determined by a combination of statistical and theoretical means (McKeown & Thomas, 1988). Statistical significance is defined according to expressed ‘factor loadings’, which indicate the extent to which each respondent’s Q-sort is similar or dissimilar to the model Q-sort for that factor. Where a

9

PQMethod offers two options for extracting factors: “QCENT” performs a “centroid analysis” while “QPCA” performs a “principal components analysis”. Centroid analysis continues to be the method of choice for most Q researchers, although principal components analysis is the default method of factor extraction in statistical packages like SPSS.

10

See for example, Dryzek and Berejikian (1993); Dayton (2000); Barry and Proops (1999); Steelman (2000); Webler et al. (2001); Fairweather et al. (1994); Van Eeten (2001); Pelletier et al. (1999).

factor has a number of high factor loadings (sometimes referred to as ‘defining sorts’)— at least four or five—it is regarded as a well-defined factor11.

For this study, PQMethod automatically identified those Q-sorts that loaded significantly on each factor12. Six respondents had mixed loadings on two factors and each of the three factors had at least one mixed loading (see Appendix Seven for a full list of Q-sorts and loadings).

Before the researcher can begin to examine the extracted factors, those factors must also be ‘rotated’13. The aim of rotation of factors is to reach the most meaningful explanation for the observed correlations between Q-sorts. Without rotation, factors cannot be used to explain and account for the correlations that exist between Q-sorts,