Developing Measures of Job Performance for
Support Staff in Housing Services for People
with Intellectual Disabilities
Chris Hatton, Sarah Wigham and Jaime Craig Institute for Health Research, Lancaster University, UK
Accepted for publication 27 March 2008
BackgroundThere is an absence of research concerning the assessment of housing support worker job per-formance, particularly in the development of job perfor-mance measures that reflect the priorities of people with intellectual disabilities and their families.
MethodA worker-oriented job analysis method was used to develop four short job performance measures for direct housing support staff, from the perspective of people with intellectual disabilities (n= 82), family members (n= 38), support staff (n= 122) and service managers (n= 115).
ResultsAll four job performance measures showed ade-quate internal and test–retest reliability and showed very few associations with staff and resident
characteris-tics, although there were no associations between the job performance measures. The service user and man-ager-rated job performance measures showed the widest range of associations with aspects of staff well-being, service quality and service user choice and satisfaction with life, and show the most promise as short, practical measures of the job performance of direct housing sup-port workers.
ConclusionsThese methods of developing job perfor-mance measures show promise, and further investiga-tion of user-defined staff competencies is warranted. Keywords: direct support staff, housing, intellectual dis-abilities, job performance
Introduction
Support staff are the single largest component of reve-nue costs in community-based housing services across many countries of the world, and are crucial in support-ing people with intellectual disabilities to achieve valued lifestyles (Hatton et al. 2004; Stancliffe & Lakin 2005). This recognition of the importance of direct support staff is accompanied by ongoing concerns about the competencies of direct support workers who are often low-paid and in receipt of minimal training, support and supervision (Larsonet al.1998; Testet al.2004).
Within the organizational psychology literature, core competencies are typically derived using various job analysis methods (Visseret al. 1997). Job analysis meth-ods are systematic procedures widely used within orga-nizations to identify the critical aspects of a job and what knowledge, skills, abilities and other characteristics are required to conduct the job successfully. Tradition-ally, job analysis procedures have been job oriented; that
is, they assume that particular jobs have well-defined and stable characteristics. However, such job-oriented job analysis methods are likely to be ill-suited for direct support worker roles, which are often ill-defined, flexi-ble and requiring change over time (Thousand et al. 1986). Instead of attempting to define the essential char-acteristics of the job, worker-oriented job analysis meth-ods assume that job roles are flexible and may vary across individual posts and over time, and attempt to analyse and distil the essential characteristics of workers rather than of specific posts (Visser et al. 1997). Such core competencies of successful workers can go beyond a specific set of skills to include characteristics such as a willingness to learn, positive values concerning people with intellectual disabilities and interpersonal skills.
Over the past 20 years, a small body of research has attempted to identify the broad range of competencies needed by direct support workers in housing and other services for people with intellectual disabilities using various job analysis methods, typically involving a
series of focus groups with direct support workers and other professionals to develop comprehensive typologies of core competencies (Hewitt 1998; Taylor et al. 1996; Thousand et al. 1986). Research within a behavioural paradigm has tended to focus on specific staff skills designed to improve specific domains of a resident’s life, such as engagement in constructive activity or choice-making (Jones et al. 2001; Parsons et al. 1993). Similar sets of staff skills have been reported when using expert opinion and job-oriented job analysis meth-ods with staff and managers in community housing services (Hewitt 1998; Tayloret al.1996; Thousand et al. 1986).
Although typologies of support worker core compe-tencies derived using these methods have great utility for selection, training and evaluating job performance (Testet al.2004), they also have a number of limitations. First, they are based on information from professionals only, which may emphasize staff skills rather than inter-personal, relationship aspects of the direct support worker role. It is currently unclear whether people with intellectual disabilities or family members, for example, would agree with the typologies of core competencies drawn up by professionals. Second, the typologies pro-duced are exhaustive and complex, which may not read-ily translate into short and easy-to-use measures of direct support worker job performance. Finally, they are based on job-oriented job analysis methods or expert opinion, and may therefore emphasize behavioural staff skills at the expense of other potentially important char-acteristics of direct support workers. For example, these methods of conceptualizing staff skills do not explicitly include the values and interpersonal, relationship aspects of the direct support worker role (Hall & Hall 2002; Hastingset al.1995), which have been reported to emerge when using more open-ended methods to gener-ate staff core competencies with community-based staff and managers (Thousandet al.1986). These values-based and interpersonal staff competencies may be particularly important determinants of the quality of services and the quality of life of people living within these services (Burchardet al.1984, 1987, cited in Thousandet al.1986).
The study reported here aimed to use a worker-ori-ented job analysis method, the job element method (Primoff 1975; Primoff & Eyde 1988), with people with intellectual disabilities, family members, direct housing support staff and service managers to develop four sep-arate sets of the core competencies needed by housing support workers. Four relatively short measures of job performance were then developed from these sets of
core competencies, designed for completion by people with intellectual disabilities, family members, housing support staff and managers.
The four job performance measures were examined for their psychometric properties and agreement with each other. Some aspects of divergent and convergent validity were also examined, by investigating associa-tions between the job performance measures and demo-graphic characteristics of support staff and residents, staff well-being (burnout, sickness and turnover), ser-vice quality (person-centred planning processes) and service user experience (choice and satisfaction with life).
Method
Participants and settings
A total of 133 support staff participated in the project. Most staff were female (73.7%), White (95.0%), married⁄ living with a partner (70.0%; single 15.8%; separated⁄ divorced⁄widowed 14.2%) and living with a dependant (54.2%), with a mean age of 40 years (SD = 10.4 years; range: 19–61 years). Support staff had worked in services for people with intellectual disabilities for a mean 8 years 11 months (SD = 72.3 months; range: 1–321 months), in their current organization for a mean 8 years 2 months (SD = 78.8 months; range: 1–336 months) and in their current work location for a mean 3 years 1 month (SD = 40.1 months; range: 1–22 months). Staff had worked longer actual hours in the past week (7.7% 25 h or less; 32.5% 26–35 h; 59.8% 36 h plus) than their contracted hours (23.3% 25 h or less; 37.1% 26–35 h; 39.7% 36 h plus). A minority of staff reported absence through sickness in the past 6 months (45%), with a mean of 3.8 days sickness (SD = 9.3; range: 0–60 days). After 3 months, most staff members were still working in the same house (77.4%); staff were also working elsewhere in the same organization (15.0%) or were on long-term sick leave or had left the organiza-tion (7.5%).
Demographic information was collected on 128 people with intellectual disabilities. Most service users were male (54.7%) and White (93.4%), with a mean age of 43 years (SD = 15.2 years; range: 18–83 years). A minor-ity of service users were reported to be diagnosed as having a current psychiatric disorder (11.3%) or to have shown challenging behaviour causing a serious manage-ment problem for staff in the past 6 months (23.1%). Ser-vice users had lived in their current home for a mean 6 years (SD = 4.9 years; range: 4 months–23 years), and
shared their homes with a mean 2.6 other residents (SD = 1.7 residents; range: 0–9 other residents).
Demographic information was provided by 28 family members. All family members were White and a major-ity were married⁄living with a partner (57.1%; single 10.7%; separated⁄divorced⁄widowed 32.1%), with a mean age of 60 years (SD = 12.3 years; range: 31–76 years).
Demographic information was provided by 113 man-agers. Most were female (68.1%) and all were White, with a mean age of 40 years (SD = 7.9 years; range: 28–58 years).
Participants were recruited from community-based housing services in four areas within North West Eng-land, including a range of statutory, voluntary and inde-pendent sector organizations. While community-based housing services in the four areas were typical of such services throughout England, the representativeness of the staff sample is unknown.
Design
The first phase of the study used a worker-oriented job, task role analysis method (the job element method) to develop measures of job performance with four expert panels (people with intellectual disabilities, family mem-bers, support staff and service managers). The second phase of the study used a survey method to compare the four measures of job performance against each other and against other measures of service processes and out-comes for people with intellectual disabilities living in community-based housing services.
Measures
Development of the job performance measures
Separate job performance measures were developed with four expert panels (people with intellectual disabil-ities, family members, housing support staff and service managers) (see Appendix 1 for a list of items in each measure). For three of these expert panels (family mem-bers, support staff and managers), the job element method (Primoff 1975; Primoff & Eyde 1988) was used in its original form to generate a list of core competen-cies for support staff in community-based housing ser-vices. These core competencies were then used as the basis for the development of behaviourally-anchored rating scales of job performance (Smith & Kendall 1963). For the service user job performance measure, this involved a 3-point rating scale for each competency,
with a behavioural example of good, OK and bad staff behaviour attached to each point of the scale. For the other job performance measures, 7-point rating scales were used, with a behavioural example of superior, acceptable and unacceptable staff behaviour attached to the ends and midpoint of the scale.
Family members expert panel
Seven family members, all parents of a person with intellectual disabilities from across North West England, attended a full-day expert panel meeting. Family mem-bers worked in two small groups with facilitators to identify a list of characteristics of a good housing sup-port worker. Characteristics of a good supsup-port worker could include a knowledge, a skill, an ability, a willing-ness, an interest or a personal characteristic. The expert panel then came together as one group and identified 58 characteristics of a good support worker in a housing service. Facilitators encouraged participants to describe the characteristics in their own words. Although the facilitators sometimes prompted tighter definitions of characteristics, as much of the feedback as possible was recorded in participants’ own words to maximize face validity.
After this meeting, panel members were posted a questionnaire asking them to rate each characteristic on four dimensions: barely acceptable (how many barely acceptable workers have this characteristic), trouble (how much trouble would be caused if a worker did not have this characteristic), superior (how useful is this characteristic for identifying superior workers) and prac-tical (how pracprac-tical is it to expect a new worker to have this characteristic). These ratings were collated and used as the basis for ranking the 58 characteristics in order of their potential utility as core competencies according to equations provided in the job element method (Primoff 1975).
A second full-day meeting was held with the expert panel. At this meeting, the panel chose a final short list of 27 core competencies, based on the job element rank-ings and consideration of duplicate core competencies. The final list was agreed by consensus amongst the expert panel. The panel then split up into two smaller groups to develop behavioural examples of superior, acceptable and unacceptable worker behaviour for each core competency.
Following this meeting, each panel member was posted a questionnaire asking them to rate each behavioural example on a 7-point scale; examples were rewritten when one or more of the expert panel
members did not rate the examples in the required order (superior higher than acceptable higher than unacceptable). On the basis of this, a questionnaire of support staff job performance was developed for use with family members.
Support staff expert panel
Seven support workers in housing services attended the expert panel meeting. After working in small groups, then coming together as one group, the expert panel identified 72 characteristics of a good support worker in a housing service. Following the same procedure as for family members, a questionnaire assessing support staff job performance on 26 core competencies was developed for use with support staff.
Managers expert panel
Fourteen managers in varying roles attended the expert panel meeting. After working in small groups, then coming together as one group, the expert panel identi-fied 58 characteristics of a good support worker in a housing service. Following the same procedure as for family members and support staff, a questionnaire assessing support staff job performance on 23 core com-petencies was developed for use with managers. People with intellectual disabilities expert panel
A modified form of the job element method was used with an expert panel of people with intellectual disabili-ties. Eleven people with intellectual disabilities attended the expert panel meeting; most accompanied by a sup-port worker or advocate. After working in small groups with facilitators, then coming together as one group, the expert panel identified 65 characteristics of a good sup-port worker in a housing service.
Following this meeting, individual interviews were conducted with each member of the expert panel. In these interviews, expert panel members were asked to rank each of the 65 characteristics of a good support worker as ‘very important’, ‘quite important’ or ‘not very important’. The response bias questions used in the Choice Questionnaire (Stancliffe & Parmenter 1999) were used to screen interviewees, all of whom passed this screening task.
This information was collated to rank the 65 charac-teristics, and used as the basis for a second half-day meeting with the expert panel. At this meeting, the panel chose a final short list of 18 core competencies.
The panel also split up into smaller groups to develop behavioural examples of good, OK and bad staff behaviour for each core competency. Following this meeting, further individual interviews were conducted with expert panel members. In these interviews, panel members rated each behavioural example as ‘good’, ‘OK’ or ‘bad’; examples were rewritten when they did not reliably distinguish between ‘good’, ‘OK’ and ‘bad’ ratings. On the basis of this, a structured inter-view schedule of support staff job performance was developed for use with people with intellectual dis-abilities.
Other measures
The following measures were collected by self-report questionnaire and structured interview from support staff:
Demographic data on the member of support staff (age, gender, ethnicity, dependants, marital status, ten-ure in services for people with intellectual disabilities, tenure in the current organization, tenure in the current work location, contracted and actual hours worked in the past week, absence through sickness, number of sickness absences and number of days lost to sickness in the past 6 months).
Demographic data on the person with intellectual dis-abilities (age, gender, ethnicity, diagnosed as having a current psychiatric disorder, shown any challenging behaviour causing a serious management problem for staff in the past 6 months, length of time in current home, number of other residents in the house).
The Maslach Burnout Inventory (MBI) (Maslach et al. 1996), a 22-item self-report measure of three dimensions of staff burnout in human services, emotional exhaus-tion, depersonalization and (lack of) personal accom-plishment. The MBI has been widely used in surveys of support staff in services for people with intellectual dis-abilities (Skirrow & Hatton 2007), and showed adequate psychometric properties in the current sample (alpha EE = 0.88; DP = 0.72; PA = 0.68).
The Person-Centred Planning Processes scale, a 14-item self-report measure completed by staff concerning whether person-centred planning goals have been set in a number of life domains in the past 6 months. This scale has been developed for use in UK evaluation of person-centred planning (Emerson et al. 2004), and showed adequate psychometric properties in the current sample (alpha = 0.81).
Intended Turnover, a 2-item self-report measure com-pleted by staff concerning whether they are intending to
leave their current job, widely used in previous staff research (Allenet al.1990; Hattonet al.2001).
Demographic information was collected by self-report questionnaire from family members (age, ethnicity, marital status) and from managers (age, gender, ethnic-ity).
The following measures were collected by structured and semi-structured interviews from people with intel-lectual disabilities:
The Choice Questionnaire (version 2) (Stancliffe & Parmenter 1999), a 26-item structured interview concern-ing the choices available to people with intellectual disabilities, showed adequate psychometric properties in the current sample (scale alpha = 0.86). The Choice Questionnaire also contains four items to screen for response bias (recency and acquiescence).
The My Life interview, a semi-structured interview where people with intellectual disabilities discuss 14 aspects of their life, which are then rated by the inter-viewer in terms of the positive or negative nature of the interviewee’s responses (Gregory et al. 2001). This showed adequate psychometric properties in the current sample (alpha = 0.70).
The Support Scale (Malamet al. 2003), a 10-item mea-sure of adaptive behaviour designed for completion by structured interview with people with intellectual dis-abilities. This measure has been designed for a national survey of 2,750 adults with intellectual disabilities in England (Malamet al.2003).
Three months after staff completed the measures, organizational records were examined to establish whether staff had remained in their current workplace, moved to another workplace within the same organiza-tion or left the organizaorganiza-tion (actual turnover).
Procedure
Ethical approval for the study was obtained from rele-vant National Health Service Research Ethics Commit-tees in England.
Participants for the expert panels (see above) were recruited through service managers in participating services in North West England; all participants gave their informed consent to take part in the expert panels.
For the second phase of the study, service managers were asked to identify support staff members where information concerning their job performance could potentially be obtained from at least three of four sources: the support staff member themselves; a person with intellectual disabilities being supported by the staff
member; a family member in regular contact with the person with intellectual disabilities being supported by the staff member; and the line manager of the staff member.
Once potential staff members were identified, staff members were approached to give their informed con-sent to take part. If they concon-sented, then self-report questionnaires with follow-up visits from researchers were arranged. The person with intellectual disabilities being supported by the staff member was then approached for informed consent. If informed consent was obtained, interviews were arranged with a researcher. Three people with intellectual disabilities did not pass the response bias screening items on the Choice Questionnaire (Stancliffe & Parmenter 1999), and the information they provided was excluded from all analyses. Family members and managers were posted self-report questionnaires, with an information sheet and consent form attached, and a FREEPOST return envelope for confidential returns direct to the researchers. All potential participants were provided with contact details for the researchers throughout the study.
Because it was important to match information from different sources to the relevant member of staff, all sources of information were required to identify one person to match to (e.g. service users were asked to provide the name of the support worker). This information was given on a tear-off sheet, which was replaced with a code number upon receipt by the researchers.
Results
Analyses of the psychometric properties of the four job performance measures were conducted first, followed by analyses investigating associations between each job performance measure and: characteristics of support staff and service users; staff-rated measures; and service user-rated measures. Due to the skewed nature of many of the measures, non-parametric analyses were con-ducted, with a significance level of P< 0.01 set due to the large number of analyses conducted. Analyses were not conducted if the total sample for the statistical test was less than 20, or if there were fewer than 10 partici-pants in a cell for the statistical test.
Job performance measures
The items contained in each of the four job performance measures are presented in Appendix 1. While the four
job performance measures do not seem to contain con-flicting items, they certainly contain differences of emphasis, with the service user measure focussing more on relational issues and the staff measure focussing more on self-care.
For the purposes of analysis, items were excluded from job performance scales if more than 20% of partici-pants did not complete that item. On this basis, one item was excluded from the service users job performance measure (How honest are they?), and three items were excluded from the family members job performance measure (How aware and respectful are they of sexual-ity issues? To what extent are they aware and respectful of spirituality, religious and cultural issues? To what extent are they an effective advocate?). All items were retained in the support staff and managers job perfor-mance measures.
All four job performance measures showed acceptable internal reliability in terms of Cronbach’s alphas and mean inter-item correlations and acceptable test–retest reliability in terms of intra-class correlations, although all four job performance measures were skewed towards maximum scores (see Table 1). The four job performance measures also did not show any statistically significant associations with each other (there were too few service user–family member pairs for this correlation to be cal-culated).
The combination of skewed item scores, the number of items in each job performance measure and the sam-ple sizes obtained in this study (Hair et al. 1998) meant that robust factor analyses could not be conducted in this study.
Job performance measures and demographic characteristics
Aspects of the divergent validity of the four job perfor-mance measures were investigated by examining associ-ations between the job performance measures and various demographic characteristics of either participat-ing support staff or paired service users, the assumption being that valid job performance measures would show few associations with such demographic characteristics.
Associations were investigated between the four job performance measures and the following staff demo-graphic characteristics: age, length of time in services for people with intellectual disabilities, length of time in the organization, length of time working in the house, number of days of sickness in the past 6 months (all Spearman’s rho); gender, staff with dependant, length of time staff member had known the service user, sickness absence in the past 6 months (all Mann–Whitney U tests); marital status, contracted work hours in past week, actual work hours in past week (all Kruskal–Wal-lis one-way anovas). Only one association was statisti-cally significant, with female staff being rated as showing better job performance by service users (U= 256.5;n= 70;P= 0.008).
Associations were also investigated between the four job performance measures and the following service user demographic characteristics: age, adaptive behav-iour, length of time living in the house, number of other residents living in the house (all Spearman’s rho); gen-der, diagnosis of psychiatric disorgen-der, serious challeng-ing behaviour in the past 6 months (all Mann–Whitney
Table 1 Characteristics of job performance measures
Service user Family member Support staff Manager
No. of sample 82 38 122 115
No. of items in scale 17 24 26 23
Mean (SD) 2.50 (0.27); 6.01 (0.55); 6.33 (0.44); 5.95 (0.69);
Range 1.71–3.00 4.71–6.91 4.58–7.00 3.22–7.00
Cronbach’s alpha 0.84 0.90 0.91 0.96
Mean inter-item correlation 0.24 0.29 0.29 0.51
Test–retest intra-class correlation 0.77;n= 22 0.78;n= 14 0.89;n= 26 0.77;n= 18
P< 0.001 P= 0.005 P< 0.001 P= 0.002 Spearman’s correlations between measures
Family member Not calculated
Support staff 0.20 (n= 67) )0.02 (n= 32)
P= 0.10 P= 0.91
Manager 0.01 (n= 58) 0.31 (n= 28) 0.12 (n= 107)
U tests). Only one association was statistically signifi-cant, with male service users being rated as receiving better job performance by managers (U= 994; n= 113; P= 0.004).
Job performance measures and staff-rated measures
Aspects of the convergent validity of the four job performance measures were investigated by examining associations between the job performance measures and staff-rated measures, the assumption being that valid job performance measures would show associations with lower staff burnout (lower emotional exhaustion, lower depersonalization and higher personal accom-plishment), a greater number of person-centred planning goals set for the person in the last 6 months, staff self-rated intention to quit the job (all Spearman’s rho) and fewer instances of actual staff turnover in the 3 months after completing the measures (Mann–WhitneyUtest).
Higher service user job performance scores were associated with lower staff-rated emotional exhaustion (r=)0.39, n= 70, P= 0.002), and higher support staff self-rated job performance scores were associated with lower depersonalization (r=)0.27, n= 121, P= 0.003) and higher personal accomplishment (r= 0.24, n= 121, P= 0.009).
Job performance measures and user-rated measures
Aspects of the convergent validity of three job perfor-mance measures were investigated by examining associ-ations between the job performance measures and service user-rated measures, the assumption being that valid job performance measures would show associa-tions with greater service user-rated choice and more positive lives as rated by service users (all Spearman’s rho). There were too few service user–family member pairs for analyses to be conducted between relative job performance scores and service user-rated measures.
Regarding the urated choice measure, higher ser-vice user job performance scores were associated with more choice over having a pet (r= 0.45, n= 69, P= 0.001), and less choice over what to cook for dinner (r=)0.39, n= 69, P= 0.001), being alone in the house (r=)0.36, n= 69, P= 0.001) and being late home from a job or day activity (r=)0.41, n= 69, P= 0.002). Higher manager job performance scores were associated with greater service user choice over buying clothes (r= 0.40,n= 68,P= 0.001).
Regarding the service user satisfaction measure, higher service user job performance scores were associated with
not feeling left out (r=)0.44, n= 70,P< 0.001). Higher manager job performance scores were associated with greater service user satisfaction with where they live (r=)0.55, n= 68, P< 0.001), support from services (r=)0.33,n= 68,P= 0.007) and important people in the person’s life (r=)0.32,n= 68,P= 0.008).
Discussion
This study examined the feasibility of using worker-ori-ented job analysis methods to develop job performance measures for housing support workers, and investigated aspects of the reliability and validity of the job perfor-mance measures produced in the study. An important finding of the study is that adaptations of worker-oriented job analysis methods can be used with non-professionals as well as professionals to develop job performance measures. Furthermore, these job perfor-mance measures were reported by study participants to have face validity and to be easy to complete. All four performance measures also demonstrated adequate internal reliability and test–retest reliability.
Separate job performance measures were developed with people with intellectual disabilities, family mem-bers, housing support staff and service managers to allow for different perspectives on support worker job performance to emerge. There were some similar core competencies identified across the four job performance measures, such as reliability, flexibility and honesty. However, although there was little conflict between the measures, there were considerable differences of empha-sis. For example, the service user measure highlighted the importance of personal and interpersonal staff char-acteristics, whereas the support staff measure placed greater emphasis on managing staff stress. These differ-ences of emphasis are reflected in the lack of statistically significant correlations between scores on the measures, although some of these correlations are likely to be underpowered.
In terms of divergent validity, there were very few associations between all four job performance measures and support worker or service user demographic char-acteristics, demonstrating that job performance ratings were being made largely independently of these charac-teristics. It is also worth noting the lack of associations between job performance measures and indicators of staff turnover, supporting the contention that staff per-formance and staff turnover are relatively independent constructs (Hall & Hall 2002).
The sample size of family members in the study was too small to allow a meaningful evaluation of
convergent validity with other measures, while the staff self-rated job performance measure was largely associ-ated with staff burnout. This could be due to the emphasis in the staff self-rated measure on staff stress and its management, staff burnout influencing the way staff completed the job performance measure, or social desirability influencing staff responses to both the burn-out and job performance measures.
Manager’s ratings of better support worker job perfor-mance were associated (P< 0.01) with more service user choice over buying clothes and more service user satis-faction with where they live, the support they receive from services and important people in their life, with trends (0.01 <P< 0.05) in the same direction for another nine service user choice and satisfaction items. This suggests promising convergent validity for the manager job performance measure.
Service users’ ratings of better support worker perfor-mance were associated (P< 0.01) with lower staff emotional exhaustion, more choice over having a pet and not feeling left out, with trends (0.01 <P< 0.05) towards better ratings being associated with having a greater range of person-centred planning goals and more satisfaction with important people in the service user’s life. However, there were also associations between bet-ter service user ratings of staff performance and less choice over what to cook for dinner, being alone in the house and being late home from work or a day activity, with trends in the same direction for a further four choice items. There are a number of potential explana-tions for this counter-intuitive finding. For example, within the Choice Questionnaire (Stancliffe & Parmenter 1999), the midpoint of many of the rating scales for the items contains the anchor ‘I choose with help from staff’; it is possible that service users rate staff performance as better if staff are viewed as helping service users with choices and decisions, rather than not allowing them to make choices at all or leaving them to make choices without support. It is also possible that in some domains of life, service users prefer to have limits imposed on their choices; certainly a recent study found that resident satisfaction with most life domains was not associated with greater resident choice (Gregoryet al.2001).
Although this study has suggested that job perfor-mance measures based on worker-oriented job analyses show promise, there are several limitations of the study. First, there were small sample sizes for some groups, particularly family members, resulting in many analyses being underpowered or not possible. This was due to the relatively small number of family members in regular contact with the service users in the study (cf. Robertson
et al. 2001). Second, as required by the job element method, single expert panels were used to generate the core competencies for each of the job performance measures, unlike the many focus groups used by previ-ous researchers in job analyses (Hewitt 1998; Tayloret al. 1996). Although expert panel members were drawn from different localities and services, it is possible that other expert panels would have generated quite different sets of core competencies. The generalizability of the job performance measures developed in this study is also unclear; to other staff roles, other countries, diverse ethnic and cultural groups and people with severe intel-lectual disabilities. Although most of the expert panels were culturally diverse and were instructed to remember people with severe intellectual disabilities in their delib-erations, it is possible that the job performance measures do not fully reflect the diversity of people with intellec-tual disabilities living in community-based housing services. For example, the set of core competencies required of housing support staff working with people with severe and complex needs may not have the same emphases as the sets of core competencies reported in this study. Finally, the scores on all the performance measures were highly skewed towards the positive ends of the scales, possibly reflecting a sampling bias or a social desirability bias. More research examining these measures in other settings is necessary, in particular investigating their concurrent validity against other ways of assessing support staff job performance and collecting sufficient numbers to attempt to empirically derive subscales through factor analysis.
The job performance measures developed in this study by the job element method are quite different and complementary to the typologies of core competencies produced by previous researchers (Hewitt 1998; Taylor et al.1996). The job performance measures developed for this study are relatively short, simple, and designed to capture the different expectations and perceptions of dif-ferent groups of people. The measures are not designed to be comprehensive typologies of staff competencies that can form the basis of training programmes (Test et al. 2004). It is also important to note that the nature of the core competencies is also different, with a greater empha-sis on the personal qualities of individual workers and less explicit emphasis on the specific skills required of the support worker post, as Thousand et al. (1986) reported when using open-ended methods for generating support staff core competencies. This does not mean that specific staff skills are unimportant; these staff skills are crucial if staff are to achieve many of the higher-level core competencies contained in the job performance
measures (e.g. getting to know the service user well, and listening to them to find out what they want). However, the core competencies developed in this study do present a challenge to existing training schemes based on improving staff skills; can positive staff personal qualities (such as a good sense of humour, or being nice, kind and caring) be improved through training?
Hopefully, this study demonstrates the potential utility of worker-oriented job analysis approaches for identify-ing the core competencies of workers in services for people with intellectual disabilities. The study also dem-onstrates that different groups (in this case service users, family members, support staff and service managers) will have different, but equally legitimate, perspectives on what makes a good support worker. There are several potential applications of such methods, including exam-ining the core competencies involved in other jobs and roles within services (cf. Hewitt et al. 2004; Thousand et al.1986), using such job performance measures in train-ing and staff appraisal, and developtrain-ing core competency measures for use in recruitment and selection. However they are derived, identifying the core competencies of direct support workers only serves to highlight how com-plex and demanding the role of the direct support worker is; such a role demands respect and support from those funding and managing this workforce.
Acknowledgments
The study reported here was supported by the Depart-ment of Health in England, although the views expressed here are those of the authors. The authors would like to thank Emma Gudgeon for her hard work in collecting data, and Nabela Rahim and Julie Stansfield of the North West Training and Development Team for organizing and jointly facilitating the expert panel members. We would also like to thank all the pants in the study and the senior managers in partici-pating services who facilitated the study.
Correspondence
Any correspondence should be directed to Professor Chris Hatton, Institute for Health Research, Lancaster University, Lancaster LA1 4YT, UK (e-mail: chris.hatton@ lancaster.ac.uk).
References
Allen P., Pahl J. & Quine L. (1990)Care Staff in Transition. Her Majesty’s Stationery Office, London.
Burchard S. N., Pine J. & Gordon L. R. (1984)The Relationship of Manager Competence and Program Normalization to Client Satis-faction in Group Homes. Paper presented at the annual meet-ing of the American Psychological Association, Toronto, 1984. Burchard S. N., Pine J., Gordon L. R., Joffe J.M., Widrick G. C. & Goy E. (1987) The relationship of manager competence to program quality in small community residences. In:Traditions in mental retardation,Vol. II(eds J. A. Mulick & R. Antonek). American Association on Mental Deficiency, Norwood, NJ. Emerson E., Robertson J., McIntosh B., Swift P., Sanderson H.,
Hatton C., Oakes P., Routledge M., Knapp M., Romeo R. & Joyce T.(2004) Does person-centred planning make a difference?
Paradigm Annual Conference, Manchester, UK, December 2004.
Gregory N., Robertson J., Kessissoglou S., Emerson E. & Hatton C. (2001) Factors associated with expressed satisfaction among people with intellectual disability receiving residential supports.Journal of Intellectual Disability Research45, 279–291. Hair J. F., Anderson R. E., Tatham R. L. & Black W. C.(1998)
Multivariate Data Analysis5th edn. Prentice Hall, New York. Hall P. S. & Hall N. D. (2002) Hiring and retaining direct-care
staff: after fifty years of research, what do we know?Mental Retardation40, 201–211.
Hastings R. P., Remington B. & Hatton C. (1995) Future direc-tions for research on staff performance in services for people with learning disabilities. Mental Handicap Research 8, 333– 339.
Hatton C., Emerson E., Rivers M., Mason H., Swarbrick R., Mason L., Kiernan C., Reeves D. & Alborz A. (2001) Factors associated with intended staff turnover and job search behav-iour in services for people with intellectual disability.Journal of Intellectual Disability Research45, 258–270.
Hatton C., Rose J. & Rose D. (2004) Researching staff. In:The International Handbook of Applied Research in Intellectual Disabil-ities(eds E. Emerson, C. Hatton, T. Thompson & T. Parment-er), pp. 581–605. Wiley, Chichester.
Hewitt A. (1998)Identification of Competencies and Effective Train-ing Practices for Direct Support Staff WorkTrain-ing in Community Res-idential Services for People with Disabilities. University of Minnesota, Minneapolis.
Hewitt A. S., Larson S. A., Lakin K. C., Sauer J., O’Nell S. & Sedlezky L. (2004) Role and essential competencies of the frontline supervisors of direct support professionals in com-munity services.Mental Retardation42, 122–135.
Jones E., Felce D., Lowe K., Bowley C., Pagler J., Gallagher B. & Roper A. (2001) Evaluation of the dissemination of Active Support training in staffed community residences. American Journal on Mental Retardation106, 344–358.
Larson S. A., Lakin K. C. & Bruininks R. H. (1998)Staff Recruit-ment and Retention: Study Results and Intervention Strategies. American Association on Mental Retardation, Washington, DC.
Malam S., Emerson E. & Davies I. (2003) Learning Difficulties Survey: Dress Rehearsal Report and Proposals for the Main Stage. British Market Research Board, London.
Maslach C., Jackson S. E. & Leiter M. P.(1996)The Maslach Burn-out Inventory Manual3rd edn. Consulting Psychologists Press, Palo Alto.
Parsons M., Reid D. & Green C. (1993) Preparing direct service staff to teach people with severe disabilities: a comprehensive evaluation of an effective and acceptable training program.
Behavioral Interventions8, 163–185.
Primoff E. S.(1975) How to Prepare and Conduct Job Element Examinations. Personnel Research and Development Centre, U.S. Civil Service Commission, Washington, DC.
Primoff E. S. & Eyde L. D.(1988) Job element analysis. In:The Job Analysis Handbook for Business, Industry and Government
(ed S. Gael). Wiley, New York.
Robertson J., Emerson E., Gregory N., Hatton C., Kessissoglou S., Hallam A. & Linehan C. (2001) Social networks of people with mental retardation in residential settings.Mental Retar-dation39, 201–214.
Skirrow P. & Hatton C. (2007) ‘Burnout’ amongst direct care workers in services for adults with intellectual disabilities: a systematic review of research findings and initial normative data. Journal of Applied Research in Intellectual Disabilities 20, 131–144.
Smith P. C. & Kendall L. M. (1963) Retranslation of expectations: an approach to the construction of unambiguous anchors for rating scales.Journal of Applied Psychology47, 149–155. Stancliffe R. J. & Lakin K. C.(eds) (2005)Costs and Outcomes of
Community Services for People with Intellectual Disabilities. Paul H. Brookes, Baltimore.
Stancliffe R. J. & Parameter T. R. (1999) The choice question-naire: a scale to assess choices exercised by adults with intel-lectual disability. Journal of Intellectual & Developmental Disabilities24, 107–132.
Taylor M., Bradley V. & Warren R. Jr (1996) The Community Support Skill Standards: Tools for Managing Change and Achiev-ing Outcomes. Human Services Research Institute, Cambridge. Test D. W., Flowers C., Hewitt A. & Solow J. (2004) Training needs of direct support staff.Mental Retardation42, 327–337. Thousand J. S., Burchard S. N. & Haszi J. E. (1986) Field-based
generation and social validation managers and staff compe-tencies for small community residences. Applied Research in Mental Retardation7, 263–283.
Visser C. F., Altink W. M. M. & Algera J. A.(1997) From job analy-sis to work profiling: do traditional procedures still apply? In:
International Handbook of Selection and Assessment (eds N. Anderson & P. Herriot), pp. 441–454. Wiley, Chichester.
Appendix 1. Job Performance Measure Items
Service user-rated support staff job performance measure
1) How much do they listen to what you want (not what they want)?
2) How much fun can you have with them? 3) How much can you rely on them?
4) How far will they go beyond the call of duty? 5) How much do they do what’s necessary when it suits you (not them)?
6) How nice, kind and caring are they?
7) How much can they do the things you want to do? 8) How happy, or miserable, are they?
9) How flexible are they?
10) How good is their sense of humour?
11) How much will they let you get on with your own life?
12) How much do you like them?
13) How much will they do the things you want to do? 14) How handy are they around the house?
15) How much time do they spend with you listening and talking?
16) How honest are they?
17) How much do they respect your privacy?
18) How much do they give you new ideas, without forcing you to do things?
Family member-rated support staff job performance measure
1) How forward planning and organized are they? 2) To what extent do they work in partnership with par-ents and families?
3) How committed to the job are they?
4) To what extent do they do things ‘with’ people rather than ‘to’ people?
5) How trustworthy, honest and genuine are they? 6) To what extent do they actively promote a healthy lifestyle?
7) How caring are they? 8) How reliable are they?
9) To what extent do they proactively, constructively and imaginatively encourage the person to do what the person wants?
10) To what extent do they introduce new opportunities to the person?
11) How clean and appropriately dressed are they for the support they are providing?
12) How tactful and diplomatic are they?
13) To what extent do they support the person to be clean and dressed in the person’s own style?
14) How willing are they to integrate the person with dignity?
15) To what extent are they able to manage stressful sit-uations?
16) To what extent do they manage finances honestly and in the interests of the person?
17) How much do they enjoy being with people? 18) How well do they get to know the person? 19) How willing are they to learn?
20) How aware and respectful are they of sexuality issues? 21) How aware and respectful are they of spirituality, religious and cultural issues?
22) To what extent are they an effective advocate? 23) How enthusiastic are they about practical tasks? 24) To what extent are they able to seek support? 25) To what extent do they respect the person’s privacy? 26) To what extent are they able to admit mistakes and learn from them?
27) To what extent are they aware of their own preju-dices?
Support staff self-rated job performance measure
1) How responsible are you?
2) How good are your listening skills? 3) How reliable are you?
4) To what extent are you respectful of confidentiality and aware of its limits?
5) How good a team worker are you?
6) To what extent are you aware of your own stress lev-els, and do you take action to manage them?
7) How honest are you? 8) How approachable are you? 9) How calm are you in a crisis?
10) To what extent are you aware of and do you pro-mote service users’ rights?
11) To what extent are you able to delegate when needed?
12) How aware are you of everything that’s happening in the house?
13) How patient are you?
14) To what extent are you self-motivated and aware of your own weaknesses?
15) How able are you to follow policy and procedure? 16) To what extent are you able to plan and organize? 17) How accurately do you share information?
18) To what extent are you able to conduct and review risk assessments?
19) To what extent do you leave work and work and home life at home?
20) How well do you get to know the service users? 21) To what extent do you value the service users you work with?
22) How able are you to work on your own using initia-tive?
23) How much common sense do you have? 24) How confident and assertive are you?
25) To what extent do you treat people as individuals and allow them to take risks?
26) To what extent do you take things personally?
Manager-rated support staff job performance measure
1) How much do they respect the individual and their home?
2) How much common sense do they have? 3) How honest and trustworthy are they?
4) How much do they understand confidentiality and its limits?
5) How friendly and approachable are they?
6) How open minded to change and ongoing develop-ment are they?
7) To what extent do they listen and engage with an individual, and enable communication in any form? 8) To what extent do they promote a positive image of the service user, the service and themselves?
9) To what extent are they willing to learn and reflect from their own experience?
10) How much do they demonstrate a good work ethic? 11) How reliable are they?
12) How person-centred are they, and whole-heartedly focussed around service users rather than their own agenda?
13) To what extent do they take ownership and respon-sibility for their own actions?
14) To what extent do they understand inclusion values and how discrimination can exclude or impact on peo-ple?
15) How non-judgemental are they?
16) How much do they promote independence emotion-ally, socially and practically?
17) How much do they understand their roles and responsibilities?
18) To what extent are they able to work in a team and independently?
19) How much do they manage risk in a safe and responsible way, but also in a way that is empowering and enabling?
20) To what extent are they a ‘can do’ person, motivat-ing self and others to remove barriers rather than create them?
21) How much do they have practical daily living skills and the ability to promote these in others?
22) How flexible are they in their working practices? 23) To what extent do they manage finances in the ser-vice of the person?
