Research design and management

The research comprised a number of elements, of both a quantitative and qualitative nature, focusing on different areas of the given research objectives and the different target groups. The research specification required quantitative results to be produced as much as possible, though the value of qualitative output was also recognised in providing illustrative case study material. To facilitate the efficient conduct of the research activity and its project management, the various research elements were organised into a number of work packages (WPs). These were managed separately and reports delivered on each of them during the course of the project to the DFES. They were:

z a literature and statistical review

z a survey of current students (in a sample of HE and FE institutions)

z a survey of graduates (follow-up of final year students surveyed earlier)

z a survey of parents of students (who had been interviewed earlier)

z a survey of potential students (drawn from schools and colleges linked to institutions in the student survey)

z case study interviews with employers of graduates. A number of these work packages were linked.

This overall organisation of the work worked well. It meant that different elements could run concurrently, helped keep the project to time, and enabled findings to be delivered to the DfES during the project. We had initially planned to also post these early findings on the research project’s website at IES, and thus disseminate externally and get feedback from other researchers during the course of the work. However, the time and cost involved in meeting DfES requirements for publicising research results in this way were far beyond our agreed budget, and so this

did not happen. In future, we recommend that the use of the Internet for dissemination during long projects like this is given more consideration at the outset.

Further details of each of the survey stages are given in the following sections of this appendix. But before turning to them, a number of general points are worth making about the research focus and data on ethnicity, which had a bearing on the way the research was conducted and also on the analysis and interpretation of findings.

z Broad scope: This research was asked to take a very broad

scope, covering flows into, through and out of HE. Each of these phases could have been research studies in their own right. Various issues along the ‘journey’ into, through and out of HE were identified, and important linkages made between the various stages, which had not been done previously on this scale. But many issues relating to each stage were not investigated as fully as we would have liked because of limitations of the overall size of the project and the amount of detailed information on ethnicity that could be generated. z Disaggregating data: By agreement with DfES, the scope was

limited to undergraduate study only (which helped to focus resources better), but we were requested to ensure that the full range of undergraduate study was covered, that is: all types of HE and FE institutions, modes (full-time and part-time) and levels (degree and other undergraduate study, the latter referred to in aggregate for brevity as ‘sub-degree’). Factors affecting different groups of students taking different types of undergraduate study were important to investigate, but contrasts between many of these groups could not be explored as thoroughly as we would have liked because of constraints imposed by ‘small numbers’ issues. This was a particular problem when looking outside of the traditional core of undergraduate study, ie full-time, degree study (which makes up almost two-thirds of the total) to part-time and sub-degree study. If each mode/level group is explored separately by ethnic group, and also say by age and gender, numbers in most cells become very small, and make conclusions unreliable. The small size of some minority ethnic groups (eg Chinese, Bangladeshi) can be particularly problematic. Resource limitations on sample sizes in several stages of the research often prohibited investigation of variations at a detailed level.

z Ethnic Diversity: A related issue is the increasingly diverse

minority ethnic population, in particular by age and gender profile, education background, social status and culture. This means that analyses can become very complex. An aggregate White/non-White breakdown is of much less value nowadays, and we aimed where we could to use an individual ethnic group breakdown (but some very small groups needed to be

aggregated in places). A three-way gender/ethnic/social class breakdown can provide greater insights but generally was not feasible because of insufficient data.

z Defining ethnicity: The research focused on the standard

ethnic groups — ie those used in the Census questions on ethnicity (see Glossary at front of main report and also section 1.2.1). This was decided upon because of the need for compatibility with other data sources (eg national student datasets, produced by HESA and UCAS, on which we based our student survey sample design). It meant though, that we had to give less attention to other aspects of ethnicity or to specific ethnic groups who are not defined by these groupings,

eg refugee groups, asylum seekers, or those from countries of

more recent immigration (eg in Eastern Europe, Middle East). However, we did include questions on religion and country of birth in our student survey (see section 3.2). A point worth noting is that changes made to the standard ethnic groups in the 2001 Census, and adopted by HESA and UCAS from 2001/02, make comparisons over time problematic, and so any trend data (using old [1991 Census] and new [2001 Census] categories) should be treated with caution. The main change in 2001 was the introduction of a new category of mixed ethnic groups (eg Asian/White, Black/White), and recognition of people of Irish descent within the White category.

z Self-identification: A final point to note is that the method of

reporting ethnic group in the Census, UCAS and HESA records (and in our surveys) is by self-identification, ie individuals choosing a group with which they identify the most from a given list. Self-identification or self-classification did not appear to be a problem in any of our surveys and non- reporting in the HESA and UCAS home student data is fairly low overall (though higher in some parts of HE, such as part- time sub-degree study, and higher overall than in the Census 2001). However, problems can arise when combining data from more than one source, even those that use the same ethnicity classification system. There are likely to be some differences in the way the same question is answered, coded, or presented in analysis, leading to uncertainties with data validity. In particular, we have highlighted in the text the difficulties with the calculation of Higher Education Initial Participation Rates (HEIPRs) for individual ethnic groups, which combines HESA and Census data (see section 4.1.1). The respondents in the two surveys are likely to be different and may identify themselves (and others at their address in the case of the Census) with the ethnic groups differently.