Reflection and feeding forward

This section reflects on cycle one and suggests changes going forward into cycle two, which include improvements to the induction process along with changes to the e-Portfolio design and the data collection methods.

The induction process

Participants suggested that one of the reasons for the initial low level of activity was the lack of formal guidance on artifact production and sharing. Rather than the approach to sharing evolving over time, it was suggested that the way to introduce these as community norms was through a more detailed, specific induction with demonstrations and explicit opportunities for artifact construction and commenting. The teaching presence indicator from the community of inquiry model is perhaps less useful where promoting an equality, participatory approach, as the aim of peer participation results in the teaching role being distributed amongst cooperating peers. It does reinforce the importance of an initial moderator role, such as that suggested in Salmon’s e- moderator model. This and the feedback from the participants suggests the need for a stronger induction with more direction on artifact creation and curation,

which is key to making the formative feedback practice clearer and better defined (Strivens et al., 2009).

For the small proportion of students who don’t use, or wish to use social networking sites, a special effort should be made to emphasise the advantages of the learning community approach during this process. There can be participant anxiety over sharing and receiving responses from peers (p. 13), but this can be addressed by emphasising the process of reification and the advantages inherent in participation.

‘Low order’activities that typically feature demonstrations of repetitive learning push students towards the idea that there are single solutions to activities, and that artifacts are solutions to tutor designed activities that reinforce the power relation in each role. Starting with these and then broadening the activity types over time, along with more vivid demonstrations of what can be used to demonstrate learning taking place, should increase the variety of artifacts created. The value of making visible work in progress and community assistance should also be demonstrated with actual exercises during this process. Curation could be encouraged with specific activities, and more emphasis in the use of personal learning environments and personal learning networks should broaden the sources that are used by the participants.

Better immediate use of the analytics in real time may encourage participation levels, which would require changes in the interface. This, and a better description in the induction of what and how data is being collected from the system should address some of the surveillance concerns raised.

The e-Portfolio design

The analytics prove a rich source of raw data, but finding better ways of representing the information to make it more accessible for both participants and the tutor should be possible. During a cycle, there are three key indicators of participation, which are:

• the use of the dashboard;

• the level of artifact creation, view and review of own work; and

• the level of interaction with others, encompassing searching, viewing and

The dashboard is the main page which is displayed after login and is returned to after each activity, which serves as an indicator of the general level of participation. There are opportunities for using both artifacts and activity traces in the dashboard to increase links between the participants, using three mechanisms:

• using the meta data associated with each artifact, for example the tags; • using analytics in real time; and

• using previous artifacts to recommend artifacts that may be of interest.

Tags attached to the artifacts signal both an emerging vocabulary and the types of activities that are being attempted, so it should be possible to reflect this back to the community and the tutor. Similarly, activity data should be able to be used to summarise what a participant has achieved, the rhythm of the community as a whole and to suggest artifacts that would be of future interest. These changes can all be implemented on the dashboard.

Data collection methods and analysis

Graph measures such as number of edges, islands and graph density can be used during the cycle as appropriate signals of the community performance over time, signalling peaks and troughs in class and online activity. Degree distribution is calculated by participant, but is also more useful as a measure of activity in the community, working as a measure of the number of connections being made. Although the use of density measures has been questioned when the network size becomes larger (Toikkanen & Lipponen, 2011), it is a useful measure here as a signal to the tutor of the proportion of connections being made on a week by week basis.

For the measures that are calculated by participant such as degree value, betweenness and eigenvector centrality, highlighting the top and bottom 20% are useful in identifying individuals that are underperforming or “highflying”. These work less well in instances where the number of active participants is lower but does signal where the tutor and/or portfolio is failing to promote connections. Eigenvector centrality is a signal of dissemination, which is applicable to artifact creation and reuse by others, which when evaluated over longer periods may be suggestive of emerging expertise. Betweenness is less expressive in the context of assessment artifacts, as there is no immediate way for brokering across the network to be meaningful as the successful application of the networked learning philosophy should make this redundant.

Sociogram diagrams provided an immediate visualisation of non-participation and the degree of centrality by activity, but can be time consuming to create and require temporal boundaries to be set, which may not align with participant’s actual practices. A more meaningful analysis of the behaviour of closer packed actors requires an analysis of artifact content to see the nature of the collaboration or reuse that is taking place; clique detection is easier to identify through this representation or by calculation, although it has not been applied here because of the smaller sample size. Although valid, the use of analytics has to be used cautiously as individuals may have activity patterns that don’t align with the measures. Here for example, the statistics would report that two participants were not engaged at all, when in reality they had not integrated artifact production into their working practices so the work they were performing was not registered.

The next chapter takes this analysis forward by implementing the changes in a second cycle with postgraduate students on an equivalent module.