Identifying the feasibility of undertaking a further validation study

Setting

The feasibility study was undertaken at an acute stroke unit in an inner city teaching hospital serving a population of 250,000.

Subjects and sampling

Inclusion criteria: stroke patients with first or recurrent acute stroke early after the event or at a later point in their hospital stay if a marked deterioration in medical status has occurred. Patients who present either with or without a facial hemiparesis, with or without a hemiparesis and with or without a history of recurrent chest infections.

Exclusion criteria: Those with aspiration pneumonia, or those in a state of agitation, confusion or distress.

Ethics

Ethical approval was given (Appendix 27) to undertake testing of the BESST in the clinical setting. Patients who agreed to be approached to participate in the study by the Research SLT were allowed a maximum of 12 hours to consent to inclusion in the study. Consent needed to be gained in a timely fashion owing to the clinical need for intervention regarding nutrition and hydration. Witnessed patient consent and relative assent was sought if the patient was unable to give written consent. This was in keeping with ethical requirements at this time.

Method

Twelve acute stroke patients admitted to the stroke unit were identified. It is difficult to make an assessment regarding the generalisability of the clinical characteristics of such a small sample and the population from which it is drawn. Instead, the purpose of the pilot study was to reflect the full range of management options in the pilot tool; therefore, patients were purposefully selected (by the SLT) to reflect the range of swallow dysfunction and not necessarily stroke severity, per se.

conventional bedside assessment, one stroke specialist nurse (N1) and one research nurse (N2). In order to explore inter-rater reliability patients were rated for swallowing problems at their bedside by the SLT, who used his clinical judgement, and by the two nurses who used the BESST. The nurses were asked to follow the instructions on how to use the tool (available on the back of the tool) to guide use. Neither nurse had experience of assessing dysphagia in patients with neurological problems, and the nurses were therefore required to follow the BESST without being able to draw on previous experience. All used the same four management options of „normal‟, „soft diet‟, „thickened fluids‟ and „NBM‟. The procedure was repeated on all patients on the following day in order to explore intra-rater reliability. In order to minimise the effects of fluctuation in the patients‟ medical state, the SLT (gold standard) identified changes in patients‟ status by discussion with nursing staff and examination of the medical notes and nursing cardex prior to his assessment. Any patients who demonstrably

deteriorated or improved had their data excluded from the assessment of intra-rater reliability.

All raters were asked to record contemporaneous reasons for their decisions as well as to record any potential bias that may have influenced their management decision, e.g. drinks on bedside cabinets or clinical dilemmas, for example whether or not to continue assessment if the patient coughed whilst trying to swallow water (Appendix 31).

In order to minimise the effects of swallow fatigue upon the results from repetitive screens over a 30 minute period, the raters‟ order of assessment was randomised. This was achieved by using blocks of six, allowing each of the raters undertaking the screen to be first in the list once, second in the list once, and third in the list once, i.e. 1, 2, 3; 1, 3, 2; 2, 1, 3; 2, 3, 1; 3, 1, 2; 3, 2, 1. The sequences were placed in opaque

envelopes and shuffled. One nurse selected an envelope (from the block of six) to identify the order of screens for a patient. All raters were blind to each others‟ ratings.

Figure 7.4: Protocol for pilot study

Acute Stroke admission

Inclusion criteria:

Acute stroke (first or recurrent)

Exclusion criteria: Aspiration pneumonia;

Agitated, confused or distressed

Purposive sampling by SLT (gold standard) of 12 stroke patients to reflect stroke severity

CONSENT (within 12 hours)

Patient consent Witnessed consent Relative assent

Day 1: 12 patients

1 SLT „gold standard‟ assessment

1 Stroke specialist nurse assessment using BESST 1 research nurse assessment using BESST

(contemporaneous reasoning)

Day 2:

Same 12 patients if no change in medical state 1 SLT „gold standard‟ assessment

1 Stroke specialist nurse assessment using BESST 1 research nurse assessment using BESST

Results

Patients assessed by the SLT and the Nurses (N)

Table 7.1 shows that according to the SLT there was a reasonable degree of variability of ratings on both days. However, both nurses tended to be more cautious in their ratings and rated many more patients as nil by mouth.

Table 7.1: A summary of ratings made by the SLT and the two nurses on the two days of the study. Each cell shows the number of ratings made by a rater

Day 1 Day 2 SLT N1 N2 SLT N1 N2 Normal 5 3 3 5 2 2 Soft diet 3 0 0 3 0 0 Thickened fluid 1 1 0 1 2 1 Nil by mouth 3 8 9 3 8 9 Total 12 12 12 12 12 12

D1 = day one; D2 = day two.

The diagnostic performance of the tool was assessed by comparing the ratings of the SLT on day one with the nurses‟ ratings on day one. The results were similar on the two days, so the results for day two are not presented. The agreement between the SLT and N1 can be seen in Table 7.2. In the table, cells to the left of the diagonal (descriptor) represent an impact on clinical care that puts the patient potentially at risk. In contrast, cells to the right of the diagonal (descriptor) represent an impact on clinical care that does not put the patient at risk but which might be considered to reduce their quality of life.

Table 7.2: Comparison of SLT ratings with N1 ratings on day one N1

SLT

Normal Soft diet Thickened

fluid NBM Total Normal 3 0 0 2 5 Soft diet 0 0 0 3 3 Thickened Fluid 0 0 1 0 1 Nil by mouth 0 0 0 3 3 Total 3 0 1 8 12

Analysis of these data revealed:

Percentage Agreement = 58.3%; Kappa = 0.42

7.1.3.a Sensitivity, specificity, positive predictive values, negative