Validity, feasibility and reliability

The trade-off between the feasibility of employing methods and the validity of the methods themselves is an important issue that must be considered when assessing physical activity in any population. Validity refers to the extent to which an instrument or tool measures what it is designed to measure (Vincent, 2005), whereas feasibility is concerned with the practicality of employing measures. Figure 3.1 shows an adapted version of the figure presented by Esliger and Tremblay (2007) which demonstrates the trade-off that may occur when various measurement methods are employed. As can be seen in the figure, the most feasible method for measuring physical activity occurs in the form of self-report; however this offers limited validity, especially in children, due to the disadvantages discussed previously.

107 Self-report

PA diaries Pedometers

Accelerometers Heart rate monitors

Direct observation

Indirect calorimetery DLW

• •

•

• •

• •

•

Validity

Alternatively, indirect calorimetery offers highly valid and reliable physical activity data, but is not a feasible method for measuring habitual physical activity. The figure suggests that objective measurements, such as motion sensors, provide the most suitable tradeoffs between feasibility and validity when it comes to acceptable physical activity measurement methods. The decision then comes down to which measurement method is more advantageous in children

Figure 3.1. Trade-off between feasibility and validity of physical activity monitors, modified from Esliger and Tremblay (2007)

Also of consideration is the reliability of a measurement method; reliability refers to the consistency of the method to reproduce similar results under the same circumstances (Vincent, 2005). It can be influenced by many factors, including the characteristics of the test, the subject being measured, and the person taking the measurement (Kohl et al., 2000). In addition, the lack of stability in the measured behaviour itself (PA) may cause a reduction in reliability scores assessed over two time points (Kohl et al., 2000); this final point may be of particular importance when assessing physical activity, which may fluctuate depending on seasonality, school holidays and whether

Feasibility

108 measurement concerns week or weekend day activities. (Trost et al., 2000;

Rowlands, 2007; Tucker & Gilland, 2007; Verbestel et al., 2011).

With respect to the measurement methods discussed above, most have demonstrated adequate reliability (e.g. Carter & Jekendrup, 2002; Goran, Pehlman & Danforth, 1994; Brown et al., 2006; Bauman & Merom, 2002; Tryon et al., 1991; Treuth et al., 2004; Esliger et al., 2011). However, it would seem that self-reported PA may be more effected by factors influencing reliability, despite this, reviews of the literature note that various self-report measures have demonstrated good test-retest reliability (Bauman & Merom, 2002; Brown, Trost, Bauman, Mummery, Owen, 2004). Another measure potentially susceptible to influential factors is direct observation. Here, in order for the observation system to be reliable agreement between raters must be high, in addition, reliability varies depending upon the complexity of the system (McKenzie, 2002), indicating that some systems may be unreliable depending upon the observers and the system itself. Evidence exists for the high reliability of each method of PA measurement; however, a balance must be achieved between reliability, validity and feasibility, in order for successful physical activity measurement to be achieved. A decision table is presented below to summarise these factors for each method.

109 Table 3.3. Summary of PA measurement methods

Method Measurement Outcomes Research

All populations Medium Medium Susceptible to

influence from other sources

110

All populations Medium High Data reduction

methods cause

111 3.6. Method choice summary

The potential for misclassification in physical activity measurement is substantial if inappropriate measures are chosen. Previous literature within the psychological well-being and physical activity domain have mostly chosen to employ self-report measures to assess both physical activity behaviour and PWB, allowing for the possibility of common method bias and misclassification that may lead to an under- or over-estimate of the true association. With consideration to the methods discussed above, the use of an objective measure of physical activity, such as accelerometers, seems to provide the most appropriate solution in children. As PWB must be assessed via self-reported measures, the use of accelerometers overcomes the issue of common method bias, whilst maintaining a balance between feasibility and validity, and in turn allowing for greater precision in the exposure measure. Furthermore, accelerometers are particularly suited to research with children due to their small size, and ease of employment (Freedson et al., 2005; Rowlands et al., 2004). However, accelerometry is not without its disadvantages; the potential for misclassification of intensities is high due to data analysis techniques, wear time compliance and the use of absolute estimates of intensity. Though some methods have employed accelerometry to examine the relationship between PA and PWB, they have failed to take into account the multitude of methodological issues that accompany accelerometer use (Crocker et al., 2006) and result in misclassification of time in activity intensities and different bias being introduced, both of which may alter the relationship between PA and PWB. In order to gain a greater understanding of the relationships between the two constructs, further consideration should be given to how misclassification and bias can impact upon relationships observed.