Chapter 1: Introduction and background
3.5 The six steps in concept mapping
3.5.3 Step 3: Structuring the statements (prioritising and clustering)
The steering group met to review the full set of statements generated by all participants. The aim of this meeting was to ensure the final set of statements was 98 or fewer The Ariadne® software package for concept mapping (NcGv/Talcott, 1995) can only accept a maximum of 98 statements). This was achieved by synthesising statements that were the same or similar and by excluding statements that were irrelevant or too vague.
Different coloured paper was used to print the statements from each group. This helped to clarify any ambiguous statements and allowed us to ensure that the statements included in the final set included a fair representation of the statements generated by all three groups. A random number generator was used to allocate a random number to each statement in the final set. The
final set of statements was printed on small cards (8cm x 3cm) and two full sets of 91 cards were presented to each participant for the prioritising and clustering tasks. The two sets of cards were printed on different coloured card in order to prevent the cards from becoming mixed. Each set of cards was shuffled to ensure that the participants would view the statements in a different order to one another as it was felt that this would reduce any chance of bias from the order in which statements were viewed.
Preparation for prioritising and clustering
In preparation for the prioritisation and clustering tasks, each of the 91 statements was allocated a random number from 1 to 91 using a random number generator. Statements were printed on individual coloured cards with the random number displayed in the top left corner as shown in Figure 4.1 below. Appendix 34 shows the final set of 91 statements in numerical order using the number allocated by the random number allocator. Each full set of 91 statement cards was shuffled so that participants would view the statements in a random order that was different to other participants. Participants were provided with two full sets of the 91 statements, one for the prioritising task and the other for clustering. The sets of cards were printed on different coloured card to reduce the chances of the two sets of cards getting mixed up during the prioritising or clustering tasks.
2. …if there was more availability of savoury gluten-free snacks and not just sugary cakes and biscuits.
Figure 4.1: Example of a statement card for use in the prioritising and
Structuring: Prioritisation and clustering
Prioritising and clustering were performed individually by each participant without the input from other participants or from the researchers.
Participants were asked to rate the importance of the 91 statements for importance on a 5-point Likert-type scale with 1 being the least important and 5 the most important. Participants were asked to place the statements into five fairly equal piles representing the priority rank assigned to them (Appendix 35). It is unlikely that any of the statements generated during brainstorming would be considered to be completely unimportant, and it was stressed that the level of importance of a statement should be judged in relation to that of the other statements and ranked accordingly.
Participants were asked to group the statements into themes in a way that made sense to them. Participants were instructed that they should not put all the statements into 1 group or to have each individual statement as its own group. Also, participants were advised that they should not have a ‘miscellaneous’ or ‘other’ group where a group of unrelated statements were
grouped together. Any statements that were unique and could not be grouped with other statements should be put in a ‘pile’ on their own as a single statement. After sorting the statements into themes, participants were asked to assign a name to each of their piles and to complete a data sheet (Appendix 36 ) listing the random number of the clusters under each of their cluster names.
3.5.4 Step 4: Analysing the data
Data were entered into the Ariadne® software package for concept mapping (NcGv/Talcott, 1995; Severens, 1987). To ensure anonymity, random 3- digit reference numbers were used to identify participants, rather than imputing participants' names. A coding system was used to identify which of the three stakeholder groups each participant belonged to. The full list of statements was entered into Ariadne along with the random reference number for each statement. The priority rating and clustering data for each participant was entered using the statement reference numbers. For the prioritisation data imputing, statement reference numbers were entered into five columns which represented the different levels of priority (1 = least important; 5 = most important). In the same way, the statement reference numbers were entered in columns to represent how each participant had grouped the statements during the clustering task.
Double data inputting was carried out using two separate computers to reduce the risk of data imputing errors. Ariadne allows for duplicated entries and missing entries to be identified and this further reduced the risk of imputing errors.
The aim of data analysis was to identify the main themes, or concepts, relating to adherence to a GFD and to establish the relative importance of each concept in relation to the others. Data analysis was also used to identify similarities and differences between the perceptions of the three stakeholder groups. Data were analysed using multivariate statistical techniques (multidimensional scaling (MDS) and cluster analysis) to produce interpretable visual concept maps.
The programme calculated how frequently statements were sorted into the same group, or theme, by participants during the clustering task. The output of this was a point map which showed each individual statement as a data point on a two-dimensional plot (point map).
The number of clusters to include on a map can be increased or decreased until an appropriate number which accurately reflects the concepts of the topic are represented. The steering group decided on the most appropriate number of clusters to include on the concept map. The steering group were asked to decide on names for each cluster. The aim was to decide on names
that represented the core or common ideas within the group of statements in the cluster. The cluster name was agreed on by all members of the steering group. In this plot (point map), the similarity between each statement is represented as a geographical distance. Statements that were judged to be similar by participants appear closer together on the two-axis matrix.
The mean priority rating for each statement was calculated by adding together the mean priority ratings (from 1 to 5) and dividing the result by the number of participants who completed the prioritisation task. The mean rating for each cluster was calculated from the mean score of all the statements contained within the clusters. This allowed me to identify which clusters were more or less important.
Identifying differences and similarities between stakeholder groups.
I wanted to find out whether or not there was a difference in medians between stakeholder groups in relation to the level of importance attributed to each of the statements identified from brainstorming. The Kruskal-Wallis test is appropriate for use with ranked data and this test was used to identify significant differences between the three stakeholder groups. Where significant differences were found using the Kruskal-Wallis test, pairwise comparisons among the three groups were made using the Mann-Whitney U test.