2.3.1 Methods
The Glasgow Royal Infirmary Stroke Register which contains baseline and outcome data for 1029 consecutive patients admitted to an urban teaching hospital between 2000 and 2002 was used to develop the predictive model. The mobility outcome used for the analysis was independent walking at 30 days post-stroke as measured by the subsection of the BI. Patients were excluded from the analysis if this data were not available.
The factors identified by the systematic review and present in the data set constituted Model 1. As the review identified a low number of studies
investigating predictors in this early stage post-stroke two other methods were used to identify potentially predictive factors and thus developed two further predictive models; one based on factors selected by clinical opinion (Model 2) and the other using univariate analysis to select factors (Model 3). These
methods of selecting factors for modelling avoid over fitting of the model which is associated with including all patient baseline factors in the model.72
Clinical opinion was obtained from a physiotherapist and a doctor. Baseline factors were listed, along with a description and the intention for
inclusion/exclusion with reason for any exclusion stated, and provided for independent appraisal. Justified disagreement resulted in the factor being subsequently included or excluded accordingly. The reasons for exclusion were classified into three categories: ‘irrelevant’; factors considered unlikely to be associated with the outcome, ‘better measure available’; duplication of factors i.e. diabetic medication was excluded in favour of presence of diabetes. To reduce the number of factors only one blood pressure variable was included.
Diastolic blood pressure was excluded as there is some evidence that diastolic blood pressure is measured less reliably than systolic blood pressure.79 The third category was ‘missing data’ for factors with high levels of missing data (defined as missing for > 20% of patients) which may reflect that they are not easily collected in clinical practice.
Univariate analysis was conducted between the baseline factors and the
outcome to determine inclusion for Model 3.72 80 Categorical factors were either dichotomised (stroke type, side of lesion and housing arrangements on
admission) or the number of groups were reduced (i.e. smoking status, level of disability). This was done to minimise the number of factors entered into the models in order to make the models as parsimonious as possible.80 Continuous factors were categorised (i.e. level of stroke severity and ADL) using
well-established cutoff points if data were not normally distributed and did not have a linear relationship with the outcome. It is recognised that dichotomising data can result in biases and loss of efficiency,81 therefore the recommendation to categorise factors into three groups was adopted.82
Data on age were normally distributed and there was some evidence that the association between age and the outcome was linear. Systolic blood pressure on admission and systolic blood pressure measured between day one and two was dichotomised using the cutoff point ≥ 160 mmHg as there was no evidence that the association between blood pressure and the outcome was linear. The length of time from symptom recognition to admission was dichotomised at the median, again as the association between this factor and the outcome was not linear.
Since the dependent variable in the data set was binary, logistic regression was the technique employed using backward stepwise regression to drop the least significant factors in turn. Odd ratios were calculated from the regression coefficients in order to convert to a natural scale and ease interpretation. A conservative level of significance (p ≤ 0.1) was used to prevent omission bias both on univariate and multivariate analysis.
The performance of a predictive model should be assessed in terms of both calibration and discrimination.83 Calibration was investigated by plotting the actual proportion of patients who walked independently at 30 days against the probability predicted by the model. To do this the prediction scores (calculated from the regression coefficients) were split into deciles and the mean predicted probability calculated for each group. The mean predicted probability was then plotted against the observed proportion along with 95% CI for each predictive score group. These calibration plots were accompanied by the
Hosmer-Lemeshow test.71 Although, calibration may allow patients to be advised of their
chances of walking it does not distinguish between those patients who will and will not walk independently.84 It is the area under a receiving operating
characteristic (ROC) curve which provides an assessment of how good the model is in discriminating between individuals with and without the outcome.72 A ROC curve plots the sensitivity (proportion of true positive predictions) against 1 minus specificity (proportion of false positive predictions). An area of 0.5 implies a prediction that is not informative and considered no better than chance alone.
The higher the area under the curve the better the model is at predicting the outcome.
2.3.2 Results
The patient baseline demographics and clinical factors are presented in Table 2-3. The total number of baseline factors stored in the data set was 102-3. Outcome data were available for 820 patients with 487 patients (59.4%) independently walking at 30 days post-stroke.
Table 2-3 Baseline demographics and clinical factors
n(%)∗ n(%)∗
Number of patients 1029 820
Age (mean, SD) 68.9 (13.2) 69.2 (12.6)
Female 518 (50.3) 406 (49.5)
Stroke risk factors
Hypertension 492 (47.8) 405 (49.4)
Atrial fibrillation 115 (11.2) 98 (12.0)
Coronary heart disease 335 (32.6) 273 (33.3)
Diabetes 178 (17.3) 146 (17.8)
Current Smoker (yes) 365 (35.5) 289 (35.2)
Independent pre-stroke (mRS score 0-2) 823 (80.3) 667 (81.3) Living arrangements on admission
∗ Entries are n (%), unless stated otherwise
mRS: Modified Rankin Score; NIHSS: National Institute Health Stroke Scale; TACS: Total Anterior Circulation Syndrome; PACS: Partial Anterior Circulation Syndrome; LACS: Lacunar Circulation Syndrome; POCS: Posterior Circulation Syndrome; SSS: Scandinavian Stroke Scale (a lower score indicate higher impairment)
Four out of the seven factors (age, stroke type, consciousness level and leg power) identified by the systematic review as significant predictors of mobility post-stroke were recorded in the data set and therefore constituted Model 1.
The clinical experts identified 22 factors as potentially predictive of
independent walking. This clinical appraisal process excluded 78 factors; better measure available (58.4%), irrelevant (27.3%), missing data (14.3%) (Appendix 5).
On univariate analysis 32 factors were significantly associated with independent walking. A summary of the factors entered into the three models is in Appendix 6. The final factors included in Model 1 after stepwise regression analyses were age (p < 0.01), stroke type (p < 0.01), consciousness level (p = 0.04) and leg power (p < 0.01). The output from the model including regression coefficients and p-values is in Table 2-4.
Table 2-4 Results from multivariate analysis
Factor Model
coefficient
95% CI p-value OR
Model 1: systematic review (n=819)
Age -0.04 -0.05, -0.02 0.000 0.97
Stroke type (TACS) -1.49 -1.96, -1.02 0.000 0.23
Consciousness (unaffected) 0.80 0.05, 1.55 0.036 2.23 Leg power (unaffected) 1.09 0.74, 1.44 0.000 2.98 Model 2: clinical opinion/univariate analysis (n=817)
Age -0.02 -0.04, -0.01 0.004 0.98 The dependent variable is independent walking measured at 30 days post-stroke. The
independent factors included in the final multivariate models are listed in the first column. Stroke type, consciousness, leg power, living alone, severity, disability and activities of daily living were assigned a value of 1 if the condition in brackets was satisfied (in the absence of the condition the variable was assigned a 0).
Stroke severity was measured using the Scandinavian Stroke Scale; moderate: 26-42; severe: 0-25. Disability was measured using the Modified Rankin Scale; moderate: 4; severe: 5
Activities of daily living was measured using the Barthel Index; moderate: 3-9; severe: 0-2
Positive regression coefficients and an OR greater than one means the patient is more likely to be walking independently than the patients in the reference category (always coded 0) at one month. For example, the odds of independent
walking for a patient who is fully conscious (coded 1) on admission are 2.23 times the odds of independent walking for a patient who has a reduced consciousness level (coded 0). Similarly, negative coefficients and an OR less than one means the patient is less likely to be walking at one month. For
example, a patient who has experienced a Total Anterior Circulation Syndrome, (TACS) (coded 1) the chances of independent walking are reduced by
approximately 80% compared to a patient who had experienced any other type of stroke (coded 0). When developing models 2 and 3, the factors that remained after stepwise regression were the same (hereon referred collectively as Model 2). For Model 2, the predictors of walking after stroke following stepwise
regression were age (p < 0.01), living alone on admission (p = 0.07), stroke type (p = 0.09), level of severity (overall value = 0.02), level of disability (overall p-value < 0.01) and level of ADL (overall p-p-value = 0.04). In particular, patients with a high level of disability on admission had significantly reduced chances of walking independently at three months (OR 0.04, p < 0.01, 95% CIs 5.41 to -0.10). The CIs are wider than those of any other variable, indicating a degree of uncertainty, probably due to the lower number of patients in this group. The regression coefficients were then used to calculate a predictive score for individual patients (Box 1-1). These scores was then used to create the predictive score groups used to develop the calibration plots.
Box 1-1 Calculating predictive scores using regression coefficients
Adapted from Royston et al, British Medical Journal.72
Calculating a predictive score from regression model to predict the probability of walking for a patient 30 days post-stroke
Predictive score = 2.01 + ((age x -0.04) + (stroke type x -1.50) + (consciousness x 0.80) + (leg power x 1.09))
The value of 2.01 is the intercept and the numbers each predictor is multiplied by is the corresponding estimated regression coefficient. The estimated regression coefficients are the log (odds ratios) for a change of 1 unit in the predictor.
The predicted probability = exponential (predictive score) / (1+ exponential (predictive score))
For example, a patient aged 81 years classified as having a posterior circulation infarct, had reduced consciousness on admission and had normal leg power would have a predictive score of 0.23. Therefore, the predicted probability of independent walking for this patient is 0.56.
The calibration plots for both models (Figure 2-1) show that the predicted probabilities (black squares) generated by the model fit well with the actual data (black circles). In Model 2 the increase in probability of walking that occurs between the predictive score groups five and six is likely to be due to the
number of predictive score groups used or reducing the number of disability classifications. The p-value for the Hosmer-Lemeshow goodness of fit test was 0.31 for Model 1 and 0.83 for Model 2 showing that Model 2 has a better fit with the observed data than Model 1.
Figure 2-1 Calibration plots of predicted and observed probabilities
The predicted probabilities are represented by the black squares. The observed data are represented by the black circles.
When testing the discriminative properties of the model the ROC curve value was 0.80 for Model 1 (Figure 2-2) and 0.87 for Model 2 (Figure 2-3) showing that both models are better than chance alone in predicting independent walking at 30 days and are very accurate in distinguishing patients who will and those who will not walk independently at three months. The curve for Model 2 (Figure 2-1)
shows a steeper incline to the upper left hand corner of the graph indicating that a higher true positive rate can be achieved for the same true negative rate.
Figure 2-2 Receiver operating characteristic curve for Model 1
0.000.250.500.751.00Sensitivity
0.00 0.25 0.50 0.75 1.00
1 - Specificity
Area under ROC curve = 0.7575
Figure 2-3 Receiving operator characteristic curve for Model 2
0.000.250.500.751.00Sensitivity
0.00 0.25 0.50 0.75 1.00
1 - Specificity
Area under ROC curve = 0.8662
Sensitivity is the true positive rate. 1-specificity is the true negative rate.