Stage 2: Towards Competency-Based Education (2009-

A mandatory move towards competency-based education was the most significant

aspect of the change from a pedagogical perspective. At the time, use of competency-

based education in the health professions was only emerging, and its use in pharmacy

had been minimal. It was generally restricted to qualified pharmacists (McRobbie,

pharmacists in a hospital environment (Burke et al., 2008), or for future workforce

needs (Neilson, Burke, & Wykes, 2003). The wider movement to competency-based

medical education (CBME) had not yet gathered the momentum evident in practice

today with key publications only recently published (Frank & Danoff, 2007).

Development of an appropriate competency framework was known to be critical, as it

forms the core structure for all elements of a programme including WBA (Carraccio et

al., 2002). Academic staff developed a competency framework for the NPIP with

reference to frameworks available in 2009, piloted and refined it. It consisted of six

domains, with associated competencies divided into ‘clusters’. Behavioural

descriptors were provided for each competency to aid interpretation (Fig. 2.2).

The PSI required interns to be formally signed off as competent by their tutor. It became a

progression requirement in the NPIP, a requirement to sit the professional registration

exam (PRE). This required those completing a 12-month clinical placement to complete

an online workplace-based assessment (WBA) against the competency framework using a

defined rating scale (discussed below) at three points in the academic year. The same

structure applied for each of the three assessments; (1) interns completed a self-

assessment against the competency framework, (2) tutors completed an assessment of the

intern’s performance based on their observations in the workplace and the intern’s self-

assessment which was visible, (3) the intern and tutor met to discuss ratings and plan

development. For interns completing a standard 12-month placement, the first two

assessments were formative (not associated with any decision on the intern’s progression).

At the third and final assessment, interns were either signed off as competent or not. If the

tutor felt that the intern had not demonstrated competence in all relevant behaviours, a

statutory process commenced to identify necessary remediation or facilitate appeal of the

The rating scale used for the assessment of the workplace-based learning was based on

work by the Competency Development and Evaluation Group (CoDEG, 2007). The

scale, consisting of a numerical ‘level’, a single-word frequency-based ‘rating’, a

descriptive ‘definition’, and a ‘percentage expression’ is shown in Table 2.1 below. In

order for the intern to be signed-off, a tutor needed to rate them at ‘Level 4’ in each of

the competencies in the framework as required (PSI, 2008). In the event that due to the

activities in the workplace the intern could not demonstrate competence, a ‘not

applicable’ rating could be given (without penalty to the intern).

Table 2.1. Overview of the CoDEG Framework used in the NPIP 2009-2014

Level Rating Definiton Percentage

Expression 0 Cannot Candidate is not exposed to this standard in

training establishment

n/a

1 Rarely Very rarely meets the standard expected. No logical thought process appears to apply

0-20%

2 Sometimes Much more haphazard than “mostly” 21-50%

3 Mostly Implies standard practice with occasional lapses 51-84%

4 Consistently Demonstrates the expected standard practice with very rare lapses

85-100%

The ratings were entered using an online system designed specifically for the NPIP.

This technology was developed by an external company to facilitate recording of

WBA information. It was Microsoft SharePoint® based and known as the ‘eportfolio’.

A computer-based method was preferred to relying on paper-based records as it

allowed for centralised tracking and management of the 170 intern and tutor pairs who

were geographically dispersed around Ireland. It was designed to do/enable the tasks

● Prompt interns to complete their self-assessment at designated times

● Allow interns to enter their ratings (using the ‘levels’ in the rating scale outlined in Table 2.1 above)

● Record and display the entered ratings so they were visible to the intern and their designated tutor

● Notify the tutor when the intern has completed their self-assessment and prompt them to complete their assessment before a specified deadline

● Allow the tutor to enter their ratings (using the ‘levels’ in the rating scale in Table 2.1) while keeping the intern’s self-assessment visible

● Simultaneously display the intern’s and tutor’s ratings side-by-side, and aggregate these scores across the year to allow progress to be reviewed in a single place.

Figure 2.2. The eportfolio software used 2009-2014.

Figure 2.2 Explanation: The rating scale is visible at the top of the screen (labelled a). The patient care safe dispensing domain heading is visible (labelled b) with the first two competency clusters also shown (labelled c). The first competency ‘1.1 Access patient medication records/notes’ is shown with the relevant behavioural descriptors expanded (labelled d). This intern has completed three self-assessments (denoted as S1, S2, and S3) and the tutor has completed three assessments (denoted as T1, T2, and T3). The initial assessments saw the intern rated at levels 2 or 3, and as the intern

progressed through the year, their ratings increased. In the final assessment, both the intern and tutor rated at a level 4, indicating that the intern is deemed competent

(labelled e). The system highlighted the level 4s in green to enable a quick visual check by the programme administrator.

Faculty development to support tutors in this new model of education was required.

was required, and therefore they had no experience of WBA or assessing trainees

within a competence-based structure. An online programme consisting of 14

interactive, video-based online lectures, assessed via multiple-choice questions

(MCQs) was developed under the guidance of a steering group consisting of

academics, tutors, and training and development specialists. It was designed to

provide tutors with relevant skills in assessment, coaching and feedback. After two

years, attendance at a face-to-face ‘refresher’ training day focusing on practical skills

for tutors (such as dealing with poor performance, interns in difficulty, and providing

feedback) was required.

The implementation of the new programme appeared to go relatively smoothly, with

92% of interns and 88% of tutors reporting that they felt it was good preparation for

future independent practice (Strawbridge et al., 2017). However, the eportfolio

technology was associated with a number of problems. Issues included high cost for

the department and dependency on external company technical support, resulting in

significant expense and delay. It could not be linked to the Moodle virtual learning

environment (VLE) and required a separate login that led to many and ongoing

queries for the programme administrator. Programme evaluations by interns and tutors

frequently included comments about the challenges of using the eportfolio system,

although this was not a specific part of the evaluation instrument. For example:

“It gets confusing as to what is going on” (Tutor, 2010)

“The layout of the competence standards was awkward to navigate” (Intern, 2010)

Notwithstanding these issues, as there was no budget available to make improvements

2.2.3 Stage 3: National Pharmacy Internship Programme (2014-date)