Summary of key findings: systematic review
This systematic review identified baseline factors assessed within one week post-stroke that may be of value in predicting the recovery of walking in the acute stages post-stroke. Age, the severity of paresis, the degree of leg power, presence of hemianopia, size of brain lesion and type of stroke were predictors of walking within 30 days post-stroke. Age, the severity of paresis and reduced leg power were each shown to be predictive in more than one study. It could be speculated that severity of paresis and the degree of leg power are measuring the same thing. The severity of paresis was classified as either hemiparesis or hemiplegia, while leg power was measured using an ordinal scale or a continuous scale. As it was unclear whether severity of paresis referred exclusively to lower limbs and the same assessment tool was not used it was decided that these two factors be presented using the original definitions. Age has been frequently noted to have a positive association with function in the longer term.59 85 There was disagreement between the included studies as to whether level of
consciousness and stroke type had any association with the recovery of walking.
This may have been due to the differences in the measurement of these baseline factors and categorisation at the analysis stage. For instance, one study
measured and analysed consciousness on a four point scale while another analysed this as a dichotomised factor (alert or not alert).75 77 Also, the time point at which consciousness was assessed after stroke onset may have varied between the studies. One study specified that level of consciousness was
assessed within three days after onset of stroke while the other study conducted baseline assessments during the first five days after stroke.75 77
The most commonly assessed mobility outcome was walking. Walking speed was used to define independent gait 75 and an unnamed five-point scale (ranging from ‘normal’ to ‘bedridden’) was used to define independent gait in the two studies that developed predictive models.77 Two of the association studies were conducted at the same hospital site and described the pattern of recovery in four key areas of mobility (sitting balance, standing balance, 10 steps and 10-metre walk) according to the OCSP.74 78 The remaining association study used the
SSS sub-section for walking.76 Two studies used fixed assessment points with one study assessing patients at seven days and the other study assessing patients at one month post-stroke. The other studies did not have fixed assessment points and recorded the time to achieving a specific mobility outcome within a set follow-up period. 74 76 78
The selection of factors for the two predictive models developed in the studies was based on univariate analysis. While it is common practice to include factors that are significant on univariate analysis it is also important to include those based on clinical opinion or are theoretically associated with the outcome. It has been speculated that variable screening based on statistical significance alone may lead to an unreliable model. For example, the presence of incontinence and balance impairment on admission are frequently cited as potential predictors of reduced mobility in the long-term.86-91 These factors were not investigated or controlled for in any of the included studies. No attempt was made to assess the performance of the predictive models identified by this review by evaluating accuracy, discriminatory power or clinical applicability in other cohorts. In summary, the main limitations highlighted by this review were that factors may have been missed due to the method used to select variables and no attempts were made to validate the models that were developed.
Summary of key findings: predictive modelling
Considering the limitations identified by the review, existing registry data were used to develop and test suitable models to accurately predict independent walking 30 days post-stroke. Two final predictive models were developed in this study; one using the factors identified in the systematic review and available in the data set and a further model which was based on factors that had been selected by both clinical opinion and univariate analysis. Model 1 showed age, stroke type, consciousness and leg power to be predictors of walking one month after stroke. Model 2 showed age, living arrangement, stroke type, stroke severity, disability and ADL to be predictors of walking one month after stroke.
The model based on clinical opinion and univariate analysis (Model 2) showed better agreement between the predicted and observed data than that of the model solely based on the systematic review (Model 1). Both models appeared to
be able to discriminate well between those patients likely to walk and those who were not likely to walk.
The predictive modelling component of this research aimed to overcome some of the methodological shortcomings highlighted by the systematic review. The development of predictive models used a structured approach to variable selection, reported a codebook for the baseline factors used in the model and assessed the performance of the models. The most appropriate method for variable selection is questionable. The factors finally included in Model 2 and Model 3 had been identified by clinical opinion and by univariate analysis suggesting that either one of the approaches is sufficient to select factors for inclusion. The comparison of these two methods with factor selection based purely on systematic review is limited in that not all the factors were available in the data set i.e. lesion size.
The number of factors entered into Model 3 was 32 which could be considered high. Using the EPV ratio of 10:1, as previously explained, at least 320 outcome events would be needed. This would mean that 603 patients would be required for an event rate of 53% (literature estimate92). Therefore, using this general rule the sample size of 817 is sufficient to assess the factors that were entered into the model. A common limitation in predictive modelling is the management of data sets where data on potential predictive factors may be missing. A
standard approach to manage this is to conduct a complete case analysis whereby patients that have missing data are excluded from the analysis or to exclude the factors that have a high degree of missing data. There is little guidance to the extent of data that should be missing before it warrants
exclusion, hence the use of a > 20% cut-off in this analysis. This could lead to the exclusion of a defined subset of patients i.e. for example, unconscious patients, where it has not been possible to ascertain their smoking habits. An alternative approach, multiple imputation, does not only increase the statistical power of the analysis but helps eliminate the bias associated with excluding patients in a complete case analysis or exclusion of factors with a high number of missing data.
Strengths and limitations
The number of studies eventually included in the systematic review from the search output was low (n = 5). The lack of distinction between motor and functional recovery in the neuromedical literature67 and differences in
definitions for mobility may have implications for the indexing of such studies in electronic databases. To overcome this limitation, electronic indexing synonyms were used for key words to ensure the search was sensitive; however, this may have compromised the precision of the search. A few studies specified a global scale to measure function or disability which may have contained a subsection on gait. For these studies, the reviewer pursued full retrieval in case a
breakdown of mobility items was available. The subsection of the BI was reported separately in one study however this was only available six months post-stroke.87 The main reason for exclusion of the retrieved articles was that the outcome or baseline assessment was conducted out with the timescales specified for this review.
Only including studies that assessed baseline factors within seven days of stroke onset may seem stringent, yet the importance of timing in predictive research cannot be overlooked. Baseline factors shown to be highly predictive within the first two weeks may have different predictive properties if measured at a
different timepoint, even a few days has been shown to have an influence on the performance of the model .53 60 Although the updated search did not identify any further studies, one study did investigate the optimal timing of clinical
assessments using an intensive repeated-measures design.93 Veerbeek et al (2011) concluded that accurate prediction of independent walking at six months is feasible within 72 hours post-stroke using two simple bedside tests; sitting balance and muscle power of the affected leg. Furthermore, recent research has suggested the use of neuroimaging to accompany such clinical assessments to improve the accuracy of predicting motor recovery post-stroke.94
An internal validation approach was used here, whereby model performance was tested in the same cohort of patients used to develop the model. There are limitations to this approach in that the generalisability of the model is not challenged; potentially resulting in the model performance being overly
optimistic. It is recommended that performance is best tested in a new cohort of patients as opposed to the original cohort, however, such external validation studies are rare.70 If predictive models are to become more routinely integrated into clinical practice it is important that the performance of the model is
evaluated in a new cohort of patients. The face validity of a predictive tool is important and if it appears to make clinical sense the more likely it will be accepted in clinical practice. For example, some of the factors included in some of the studies identified during the systematic review may not be collected routinely in clinical practice (such as tests of line bisection and constructional apraxia).
Application of predictive models
Being able to predict mobility has important implications for the amount of care needed post-stroke and is of key importance to patients.49 78 More specifically the ability to ambulate independently is often used as a criterion in determining whether a patient is able to live at home or not.95 Complications relating to immobility such as chest infections and deep venous thrombosis account for a high proportion (51%) of deaths in the first 30 days post-stroke.96 Therefore, having knowledge about the patients expected level of mobility may allow planning of preventative measures.
Little is stated in the literature about the real-life implementation of such prediction tools in practice and it would be valuable to establish current levels of understanding and usage by clinicians. The predictive model developed contains factors that can be easily collected in practice therefore increasing its clinical usability. It is acknowledged that algorithms generated from regression models are not always straightforward and accessible for use in clinical practice.
Evaluating the cost-effectiveness of predictive models by comparing the
resource use and outcomes for one group of patients where the predictive tool was applied with another group where the model was not applied are not usually conducted. This would allow the full impact of predictive tools to be assessed in terms of the cost implications and consequences for the patient and family where accurate or inaccurate information is provided.
As the models have not been fully evaluated they cannot yet be recommended for use in clinical practice. Instead, the models could be used in clinical audit to identify patients whose actual outcome differs from that predicted. Reasons for any differences could be identified to inform patient management or improve the predictive models. The models could be used to stratify patients in
rehabilitation trials. This would reduce differences in baseline prediction between the treatment groups. The factors could be used to correct for case-mix in observational studies which would allow patient outcomes from different cohorts (i.e. hospitals) to be compared.
Future direction in predictive research
The use of meta-analysis in predictive research is uncommon due to the
availability of evidence for synthesis. In this review this could not be performed largely due to the shortage of comparable predictive studies investigating the same predictors and mobility outcome. Stratifying patients at baseline and reporting the grouped outcomes as seen in some of these studies presents another challenge for reviewers. The diversity of outcome assessments used to evaluate independent walking or return to walking probably reflects the lack of specific walking tools available.97 The use of individual patient data meta-analysis (IPD MA) may overcome some of these limitations. It is the organisation of multicentre prospective predictive studies adhering to the same protocol collecting the same baseline factors and using the same universally accepted outcome measures which appears to be the favoured approach in this area of research.98
The time taken to achieve certain mobility milestones was the primary outcome in three of the included studies.74 76 78 Proposing timescales for a certain event for different patient types is viewed as useful in goal-setting and as a prompt for further investigation if the patient does not achieve the milestone within the expected timeframe.78 This focus on time to event is even more problematic for meta-analysis, mainly due to the poor reporting of the hazard ratio and often requires a more complex analytical approach.98 The time to event is an
important clinical question and with a reporting guideline equivalent to the Consolidated Standards of Reporting Trials may overcome this limitation and
facilitate future meta-analysis in this area. The Prognostic Systematic Review Methods Group, part of the Cochrane Collaboration,99 aims to improve the
conduct, analysis and reporting of predictive research. This group should be used as a key reference point for research groups conducting future predictive
research in stroke.