41 Hypothesis 5: The outdoor programme and solo component induce profound inner transformation
Manifest content open-coding was used to code the data (Elo & Kyngäs, 2008; Graneheim & Lundman, 2004). Coding was done using NVivo 12 Software. Codes from the data were identified and grouped into themes based on the patterns that emerged under each hypothesis. Themes were named and defined and linkages between the implementation of the outdoor phase and the outcomes of the phase were identified (Braun & Clarke, 2006; Patton, 1980). The definition and descriptions of the themes as prescribed by Patton (1980), were elaborate descriptions of the programme and the participant’s experiences with regards to each theme.
42 Chapter 3: Impact Results
This chapter highlights the quantitative results from data analyses performed to respond to the impact evaluation questions.
Outcomes related data was analyzed for 65 individuals, 32 treatment (CA graduates) and 33 control (declined applicants) individuals. Table 4 summarizes the descriptive statistics of the sample.
TABLE 4:
For analysis of the average treatment effect of the programme on its various outcomes, we proposed the use of propensity score weights to balance the treatment and control group on key variables that were shown (in logistic regressions) to be significantly associated with the probability of a candidate being selected into the programme. By applying such weighting, we thus effectively correct potential imbalances in key
43 variables between groups, and thus statistically render the treatment and control groups more equivalent and hence remove potential biases in the outcome analysis. Table 5 presents the results of an assessment of the imbalance checking before and after the use of the IPTW.
TABLE 5:
Covariate Imbalance After Propensity Score Weighting
Before Weighting
Before weights were used, there was covariate imbalance on last grade passed and area of residence. After IPTW weights were applied, covariate balance is achieved.
Evaluation question 1: Do participants who have completed the programme display higher levels of resilience relative to those that do not go through the programme?
Based on the sample used for this analysis, 3-5 years after the programme, on average, youth that do not go through the CA programme have higher resilience scores than those that were on the programme (M = 32.64 SD = 4.11 vs. M = 31.06, SD = 5.03). When comparing average scores obtained on each item of the CD-RISC, CA graduates had obtained slightly higher scores for Item 1 (I am able to adapt to change), Item 2 (I can deal with whatever comes), Item 3 (I try to see the humorous side of problems), Item 7 (I can stay focused under pressure) and item 10 (I can handle unpleasant feelings). Table 6 shows the unweighted average score for each item in the CD-RISC measure of resilience for both the CA graduates and the declined applicants.
44 TABLE 6:
CD RISC: Unweighted Average Scores
Item Group M SD
1. I am able to adapt to change Declined 2.94 .90
CA 3.21 .87
2. I can deal with whatever comes Declined 3.52 .67
CA 3.13 .91
3. I try to see the humorous side of problems Declined 2.70 .88
CA 2.72 1.05
4. Coping with stress can strengthen me Declined 3.18 .98
CA 2.63 1.18
5. I tend to bounce back after illness or hardship Declined 3.10 .86
CA 3.06 1.01
6. I can achieve goals despite obstacles Declined 3.58 .79
CA 3.47 .80
7. I can stay focused under pressure Declined 3.21 .74
CA 3.22 .87
8. I am not easily discouraged by failure Declined 3.42 .75
CA 2.88 .98
9. I think of myself as a strong person Declined 3.76 .50
CA 3.59 .71
10. I can handle unpleasant feelings Declined 3.12 .82
CA 3.16 .92
Total Declined 32.64 4.11
CA 31.06 5.03
Using propensity score weighted regression depicted in Table 7, it is seen that the difference in resilience between CA graduates and declined applicants is not significant, t (64) = -1.66, p >0.05. However, based on this sample, the regression shows that there is a significant difference in the resilience scores for those who went through alternative youth development programmes. Those who went through alternative programmes (n = 7) have, on average, a 2.68 higher resilience score than those that did not (t (64) =2.07, p
<0.05). However, from this regression we can still deduce that declined applicants who went through an
45 alternative programme, on average, have a higher resilience score of 35.72 compared to CA graduates who also attended an additional youth development programme (M=33.42).
TABLE 7:
Summary of IPTW Regression Analysis for Resilience Outcome Variable
β SE p
CA Programme Attendance -2.3 1.40 .10
Attended alternative programme 2.69 1.30 0.04*
Constant 33.03
*p<.05, **p<.01, a p>.05
Using the smaller matched sub-sample, an ANCOVA showed that programme attendance does not have a statistically significant effect on the resilience score obtained, F (1,25) =1.19, p>0.05. The ANCOVA also showed that the probability of being in the CA programme (the propensity score) did not significantly affect the resilience score either. The results of the ANCOVA are depicted in Appendix D.
Evaluation question 1b: Does the level of engagement in the outdoor phase have an effect on participant’s resilience.
Although there was no significant difference between the resilience of treatment and control group participants, it was shown that for those that did go through the CA programme, their level of engagement in the outdoor phase significantly explained 23% of the variability in their resilience scores (R2 =0.23, F (1,30) = 8.76, p <0.01).
TABLE 8:
Summary of Linear Regression to assess effect of Outdoor Engagement on Resilience
Variable Β SE P
Outdoor Engagement 1.47 .50 0.01**
R2 .23
*p<.05, **p<.01, a p>.05
The results of the regression analysis summarized in Table 8, show that a point increase in engagement in the outdoor phase was associated with an increase in the summative resilience score of 1.47.
46 Evaluation question 2: Are participants engaging in productive and positive action?
a. Engagement in employment
Cichello, Leibbrandt and Woolard define being employed as engaging in some form of income generating activity (Cichello, Leibbrandt, & Woolard, 2012). Based on this definition, study participants were classified as employed if they were earning income through either formal, self or casual employment. Based on this definition, proportions shown in Table 9 and a summary of the weighted logistic regressions in Table 10, we see that although there are slightly more employed CA graduates compared to those that were declined into the programme, the higher odds of being employed as a CA graduate are not significantly different from the odds of being employed as a declined applicant (OR Employment = 1.37, p>0.05)
TABLE 9:
Currently in Formal Employment 21(32)
66
19(33) 12
Currently Self-Employed 7 (32)
21
4 (33) 12
Currently in Casual Employment 6 (32)
18
3 (33) 9
The higher odds of being in self or casual employment as a CA graduate relative to a declined applicant are also not statistically significant (OR Self-Employment = 2.39, OR Casual Employment = 2.53)
47 TABLE 10:
Summary of IPTW Logistic Regression for Employment Outcomes
Outcome Variable Odd Ratio SE Β
In the assessment of education outcomes, we focused on two changes. The first was the change in grade after applying. In this case, we compared the highest grade achieved before applying and at data collection.
We then assessed whether there was a significant difference in the odds of CA graduates furthering their high school education after completing the CA programme compared to a declined applicant. We secondly assessed the proportion of those who are currently pursuing tertiary education and those who have post high school diplomas, degrees or certificates.