The participants in this research were involved in several data collection methods over the period of data collection. These were surveys, interviews, observations and analysis of the paper-based portfolios and ePortfolios. This section will begin with the data collection timeline and will go on to discuss the methods used.
4.5.1 Data collection timeline
The data in this study was collected from July 2013 to March 2015. The initial round of data collection within the ECE setting was conducted in just over a year from July 2013 to August 2014. Interviews with the ePortfolio provider and the ECE settings management were completed in March 2015.
65
Table 4.4 Timeline of data collection
Date Event Focus Tool used
July 2013 Initial Survey -parents and whānau - teachers Paper-based portfolios Survey Monkey or paper-based July 2013 Interviews
- case study families and teachers Paper-based portfolios February 2014 Midway Survey -parents and whānau - teachers Initial impact of ePortfolios Survey Monkey or paper-based August 2014 Final Survey -parents and whānau - teachers Changes to contribution and assessment Survey Monkey or paper-based August 2014 Interviews
- case study families and teachers Changes to contribution and assessment March 2015 Interviews - Management Journey from paper-based to ePortfolios March 2015 Interview - ePortfolio provider Rationale for developing the platform 4.5.2 Methods Surveys
The survey is widely used as a research method across many disciplines, such as education, social sciences, business and sociology (Cohen et al., 2011; Mukherji & Albon, 2015; Zhang, 2000). Using surveys enables data to be gathered at particular points in time meaning that the nature of what was currently occurring can be documented (Cohen et al., 2011). Surveys are useful as they are able to ascertain information on many variables. These include attitudes, beliefs, experiences, opinions and demographic information (Wiersma, 1995). A survey is also a familiar tool. As Wiersma (1995) noted, most adults will have participated in a survey sometime during their life. As this study collected information over time, and is therefore longitudinal, surveys at specific points during the duration of the study collected information regarding participants’ changing perspectives of and experience with portfolios.
66 The participants (adults) in this research undertook three surveys over the data collection period. An Initial Survey was undertaken at the beginning of the research period and could be completed in either online or paper-based format. This survey explored the use of and engagement with the paper-based portfolio system already in place at the early childhood education setting (see Appendix Eight for questions).
A Midway Survey was undertaken six months after the introduction of the ePortfolio system. This survey explored the adult participants’ use of and engagement with the ePortfolios to date. It too, was offered in paper-based and online formats (see Appendix Eight for questions).
The Final Survey was undertaken near the conclusion of the data collection phase. It investigated which system the participants preferred and why. As with the previous two surveys, it was completed by the adult participants and was offered in online and paper-based formats (see Appendix Eight for questions).
Each survey in this study contributed to the overall collective case study of the ECE setting. The surveys also provided insight and contributions to some of the themes that were emerging in the six individual case studies which were nested beneath. The way that the surveys were designed with some open-ended questions (see Appendix Eight) meant that the respondents were able to provide rich and detailed descriptions of their experiences – this form of survey links to the narrative inquiry methodology used, and therefore contributed to the story being told.
Semi-structured Interviews
In fields where qualitative research practices take place, such as education, where this study is based, the semi-structured interview is a valid and important data collection method (Cohen et al., 2011; Gudmundsdottir, 1996; Mukherji & Albon, 2015). Gudmundsdottir (1996) saw interviews as
67 conversations from which rich and meaningful data can be extracted. Semi- structured interviews, with portfolios and ePortfolios as props, were used so that the researcher and participant could engage in a narrative conversation. This links to narrative inquiry where participants tell their stories. For the purpose of this research, the stories told in the interviews will show the participants’ experience with and perspectives about portfolios – paper-based and online.
The adult participants who were part of the six individual case studies were interviewed twice. These participants were parents, whānau and teachers. The children who were also part of the case studies were not interviewed in a formal manner as some were pre-verbal; but some of these children were involved in conversations which have been included in the data set. However their stories were frequently shared by their parents and teachers during the interviews. The Initial Interviews explored the current portfolio use in more detail. As mentioned above, these portfolios were used as props to inform and support the discussion. This allowed both the researcher and the participant to take the lead in the direction of the interview and distributed the power in the relationship (Cohen et al., 2011; Hollingsworth & Dybdahl, 2007; Mukherji & Albon, 2015). The same participants were interviewed again near the conclusion of the data collection phase. As with the Initial Interviews, the Final Interviews were semi-structured in nature and used the new ePortfolio, alongside the paper-based portfolio, as props to encourage discussion and conversation (see Appendix Nine for questions). The interviews were digitally recorded and transcribed, and were sent to the participants for member checking.
The data from the interviews was analysed and used to inform the case studies and the stories being told by the participants through narrative inquiry. The questions used in the interviews (see Appendix Nine) were framed in ways that would elicit stories. They became more like conversations that could in turn take the reader through a detailed, descriptive and contextualised journey.
68 Observations
A number of observations of children interacting and engaging with their portfolios (both paper-based and online) were undertaken during the data collection period in the ECE setting. By observing the children actually working with their portfolios, “live” data were able to be collected from a situation which was not unnatural to them (Cohen et al., 2011; Mukherji & Albon, 2015). Observations of children are commonplace in education so the children in this setting were used to being observed by adults. The observations were taken whenever they children interacted with their portfolios or ePortfolios when the researcher was present. Therefore they were not out of the ordinary happenings but were snapshots of what regularly occurred in the setting. By undertaking observations teachers and researchers are able to make sense of behavioural issues and children’s development. Observations are also useful to discover the effectiveness of practices used in an educational context (Malderez, 2003). For the purpose of this research, observations helped establish how the children used their portfolios. They also showed how adults interacted with the portfolios and who initiated those interactions. These observations were recorded as both running records and anecdotal observations, and are descriptive accounts of what was observed. The analysis and descriptions provided with the observations helped to show the reader the context in which they were recorded. Photos were taken to support the observations, if children were in these photos whose parents had not given consent for observations then their features were disguised.
The observations were included as data for the overarching case study of the ECE setting and added explanation of how children and adults engaged with paper-based portfolios initially and later ePortfolios – although this was definitely in a fledgling stage and only just developing towards the end of the data collection period.
69 Analysis of the portfolios
The portfolios themselves are rich sources of data. Both the paper-based and the ePortfolios of the individual case study children were analysed in the ECE setting. The following themes were examined:
(i) The nature of the assessments. Of particular interest were assessments which could be described as formative assessment, as discussed in the literature review;
(ii) The frequency and nature of the contributions; (iii) Who the contributions were made by;
(iv) Links to learning which were identified by the contributors.
These themes were coded so that when analysing the data a clear picture of the frequency of contributions, the type of contributions and who the contributors were was formed. Examples of the documentation contained in the paper-based portfolios and ePortfolios are provided throughout the findings chapters.
The portfolio analysis added context to the individual case studies. Through including the above data sets, any changes to formative assessment, frequency, nature of the contributions and who was contributing could be established. This data could then be analysed alongside the changes identified by the narrative stories being described in the case studies. This analysis enabled a deeper understanding of changes linked to specific educational practices, participant engagement and use of resources.