The study covered several groups of beneficiaries of the Erasmus+ programme and comparison groups. The principal target groups of this project are:
1. Learners participating in Erasmus+ programme. Groups of students were surveyed shortly (two weeks or less) before their mobility (E+ PRE) as well as after their return (E+ POST). Erasmus(+) graduates with experience of •A narrative literature review was concluded to take stock of the
knowledge base in relation to the Erasmus programme and its effects, and inform various project tasks, including questionnaire design
Literature review
•Large-scale surveys were launched, collecting data through online questionnaires from multiple target groups
Large-scale surveys
•Qualitative case studies were conducted to provide deeper insights on Strategic Partnership projects
May, 2019 26 participation in Erasmus+ or its predecessor Erasmus programme were also
surveyed. The difference between the Erasmus+ POST and graduates is that individuals in the POST group were still studying at the time of the survey83.
2. Staff: academic and non-academic staff, with Erasmus+ or Erasmus experience. 3. Higher education institutions involved in Erasmus+.
4. Coordinators of Erasmus+ Strategic Partnership projects. Several comparison groups were constructed:
1. For learners, these groups included
a) students selected to take part in Erasmus+ but yet more than three months before departure (E+ PRE PRE),
b) students and graduates participating in mobility programmes other than Erasmus(+) (OTHER MOBILE PRE, OTHER MOBILE POST, OTHER MOBILE GRADUATES) and
c) students and graduates who had not participated in international mobility experiences at all (NON-MOBILE STUDENTS, NON-MOBILE GRADUATES). 2. Similarly, for staff following comparison groups were defined:
a) staff members who had participated in mobility programmes other than Erasmus(+) (OTHER MOBILE STAFF)
b) or who had not participated in international mobility at all (NON-MOBILE STAFF).
Another important difference with most previous Erasmus studies is that for some target groups paired data were collected to allow for longitudinal analysis. This was the case of Erasmus+ participants, where –when possible within the time-scale for this project- the same individuals were surveyed twice or even three-times as they moved from the E+ PRE PRE group to E+ PRE or from E+ PRE to E+ POST. This enables a better assessment of individual changes during the period of participation in the programme. Paired data was also collected from some NON-MOBILE STUDENTS. They received the second questionnaire 6 months after the first one, i.e. an average duration of
international mobility periods. The paired data were used primarily to measure the development of attitudes and personality traits using the memo© method.
83 Participants were asked at the beginning of the survey if they had graduated. Those who provided a positive reply are classified as graduates. Those who replied negatively were classified as POST students. The analysis reported for POST respondents do not include graduates.
May, 2019 27 Figure 1. Learners’ target and control groups of the Erasmus+ Higher Education Impact
Study
Source: Visualisation ICF/CHE Consult
For more detail on definitions of these target groups and their characteristics, please see Annex 01.
3.2.2 Questionnaire development
A questionnaire was designed for each of the above-mentioned target and comparison groups. The Erasmus Impact Study (2014) questionnaires were used as a basis in the development of the questionnaires for the present study. However, the project team made substantial changes to the questionnaires used in the 2014 study in order to cover the impact areas defined for the present study and to implement suggestions received from a group of external experts and the European Commission.
The full text of the questionnaires is attached in Annex 06. 3.2.3 Sampling
A census database of Erasmus+ participants (2014-2016) was available to the research team for the purpose of sampling and survey distribution. A randomly selected subset of 70% of the student´s population was used for this project -the other 30% having been provided for an Erasmus+ Mid-Term Evaluation which partly overlapped in time with this study. At the start of the survey in May 2017, a sum of 652,094 cases served as a basis for sampling, including students and staff since 2014.
A random sample of former Erasmus+ participants (both learners and staff, i.e. E+ POST and E+ STAFF) was drawn from the population and invited to take part in the survey. As this sample was selected randomly, it is further considered representative and allows for inference towards the original population. Some individuals who had taken part in the programme in previous years and had graduated already became Erasmus(+) graduates and were included in the graduates’ survey.
As the overall amount of learners who had been selected for participation in the
programme but who had not yet departed for a mobility at the time of the research team receiving the database of participants was too small (cf. Annex 01), all of them were invited (census survey of this group). Based on the distance between the survey date
May, 2019 28 and the scheduled date for the start of their mobility experience, those individuals were assigned to the E+ PRE or E+ PRE PRE groups. The invitations were distributed on an on- going basis bi-weekly from May 2017 to April 2018, taking into account new contacts in the regularly updated database of programme participants.
Higher Education institutions participating in the programme were invited to take part in the survey by the European Commission and other institutional channels –see also next section.
Regarding Strategic Partnerships, projects from 2014 were selected as these projects run under the current programme scheme and had finished by the time of the survey.
Coordinators of all 152 projects selected in that year were invited to the survey (census survey for projects selected that year).
Further details on the sampling process are provided in Annex 01. 3.2.4 Data collection process
The survey questionnaires were distributed online from Spring 2017 until Summer 2018. Erasmus+ participants (both students and staff) were contacted directly via e-mail, thanks to the contact information stored in the Mobility Tool+ database used for programme administration. The same is true also for Strategic Partnership projects coordinators, whose contact information is stored in the project database. Pre-2014 Erasmus participants were contacted in particular through the Erasmus Student Network. Other target groups (other mobile and non-mobile learners and staff) were contacted indirectly through their institutions. Higher Education Institutions were invited to participate by the European Commission and other institutional channels (Erasmus+ national agencies and university networks, such as the UNICA Network and SGroup Network). Invited respondents were reminded once to fill in the survey if they did not respond initially.
Response rates ranged from 19% to 35% for individual groups of Erasmus+ participants. In the case of Strategic Partnership coordinators, the response rate reached 64%.
Although this result is not surprising for an online survey of the target groups for the project, non-response may be a source of bias – if individuals responding to the survey are systematically different from those who did not answer it, e.g. in their feeling about the programme. Other means of data collection, such as face-to-face or phone call surveying could have ensured higher response rate. However, these options were not possible to implement as no contact information was available apart from the e-mail addresses of Erasmus+ participants. The resulting limitations for interpretation stemming from the nature of data collection are discussed at the end of this chapter.