IX.3 Hypothesis testing
IX.3.2 Hypothesis 2a
Hypothesis 2a sets out to determine whether Business owners who experience enforcement will have higher subscription rate to a safe harbor provision than those who do not experience enforcement. This section is also segmented into four areas of results based on each round of tests and ANOVA analysis.
The tests in this subsection will identify if there is any statistical differences in the subscription rate and the number of enforcement treatments. The dependent variable for
Hypothesis 2a is once more the subscription rate, but the independent variable is the number of enforcement treatments under gone during the experiment.
(4)
where SR is the safe harbor subscription rate and ET is the number of enforcement treatments.
Formula 4 states that the subscription rate is expected to improve as subject encounter higher levels of enforcement.
Table 20 shows the distribution of enforcement treatments during the experiment and a comparison to the expected enforcement rate calculated prior to the experiment. The percent shown in this table can be used as the probability of enforcement as described in Section 8.1.5 (page 45). The table shows a higher number of not selected and “1 enforcement encounter” than previously expected. This shows that participants subscribed to the safe harbor provision at a higher rate than anticipated in the design.
Table 20: Enforcement Treatments Frequency and Percent
Number of Enforcement
Treatments Frequency Probability
Expected Probability No treatment 49 23.9% 15.6% 1 treatment 96 46.8% 31.3% 2 treatments 56 27.3% 31.3% 3 treatments 4 2.0% 15.6% 4 treatments 0 0% 6.2%
IX.3.2.1Round 4 Test Results
Figure 13 shows the distribution of subjects’ decision to subscribe to the safe harbor provision given the number of enforcement treatments (1 thru 4 treatments). As the picture shows, a larger number of participants subscribed to the safe harbor provision. The figure also shows similar distributions for both groups (subscribed and not
who experienced the higher enforcement treatments. The statistical tests for significance are shown below.
Figure 13: Round 4 Distributions between Enforcements Treatments and the Safe Harbor Provision Subscription Rate.
To determine the relationship enforcement treatments and subscription to a safe harbor provision in the fourth round, a Pearson’s χ2 was run. Since the dependent variable is categorical and not binary, the Fisher Exact test was not run for this hypothesis. The results are shown on
Table 21.
Ho: Safe harbor subscription rate and number of enforcement treatments are independent in the fourth round.
Ha: Safe harbor subscription rate and number of enforcement treatments are not independent in the fourth round.
A Pearson’s χ2
was run and the p-value was 0.012, which shows that we can reject the null hypothesis confirming that there is association between enforcement treatments and subscription to the safe harbor provision. This confirms that enforcement treatments and subscription to the safe harbor provision are related at the community-standard 5 percent alpha protection level. These results confirm that a higher number of
enforcement treatments may lead to higher subscription to the safe harbor provision in the fourth round.
Table 21: Hypothesis 2a Round 4 Statistical Tests
Test Value df Asymp. p-value
(2-sided) Pearson’s χ2
10.913 a 3 0.012
Likelihood Ratio 6.182 3 0.103
Linear-by-Linear Association 2.690 1 0.101
a. 4 cells (50.0%) have expected count less than 5. The minimum expected count is .31
As with the prior hypothesis, an effect size test was conducted. Table 22 shows the results from an effect size analysis using η2. The results found an η of 0.231 and an η2
of 0.053. Using Cohen’s interpretation guidelines (Ruscio, 2008), the results show a small effect size between enforcement treatments and subscription to the safe harbor provision. This implies that the enforcement treatments had a small effect on the safe harbor provision subscription rates the fourth round. These results are in line with the Phi and Cramer’s V test results, which are statically significant at the community-standard 5 percent alpha protection level.
Table 22: Hypothesis 2a Round 4 Effect Size Tests
Test Value Approx. p-value
η 0.231 -
Phi 0.231 0.012
Cramer’s V 0.231 0.012
Figure 14 shows the distribution of subjects’ decision to subscribe to a safe harbor provision given the number of enforcement treatments (1 thru 4 treatments). The results in this round are similar to those in round 4 and 6. Some of the similarities are larger number of participants who subscribed to the safe harbor provision, and similar
distributions for within groups (subscribed and not subscribed). However, in this round, those individuals who experienced more than 3 enforcement treatments did not overly subscribe to the safe harbor provision, which contradicts expectations. The statistical tests for significance are shown below.
Figure 14: Round 5 Distributions between Enforcement Treatments and the Safe Harbor Provision Subscription Rate.
To determine the relationship enforcement treatments and subscription to a safe harbor provision in the fifth round, the Pearson’s χ2
was run, and results are show on
Table 23.
Ho: The safe harbor subscription rate and number of enforcement treatments are independent in the fifth round.
Ha: The safe harbor subscription rate and number of enforcement treatments are not independent in the fifth round.
As mentioned before, The Fisher Exact test was not run. On the other hand, the Pearson’s χ2
was run and the p-value was 0.000, which shows that we must reject the null hypothesis confirming that there is association between enforcement treatments and subscription to the safe harbor provision. This confirms that enforcement treatments and subscription to the safe harbor provision are related at the community-standard 5 percent alpha protection level. The results confirm that a higher number of enforcement
treatments may lead to higher subscription to the safe harbor provision in the fifth round.
Table 23: Hypothesis 2a Round 5 Statistical Tests
Test Value df Asymp. p-value
(2-sided) Pearson’s χ2
47.167 a 3 0.000
Likelihood Ratio 23.497 3 0.000
Linear-by-Linear Association 4.981 1 0.026
a. 4 cells (50.0%) have expected count less than 5. The minimum expected count is .33
Furthermore, Table 24 shows the results from an effect size analysis using η2. The results found an η of 0.480 and an η2
of 0.230. Using Cohen’s interpretation guidelines (Ruscio, 2008), the results show a moderate effect size between enforcement treatments and subscription to the safe harbor provision. This implies that the
round. These results are in line with the Phi and Cramer’s V test results, which are statically significant at the community-standard 5 percent alpha protection level.
Table 24: Hypothesis 2a Round 5 Effect Size Tests
Test Value Approx. p-value
η 0.480 -
Phi 0.480 0.000
Cramer’s V 0.480 0.000
IX.3.2.3Round 6 Test Results
Figure 15 shows the distribution of subjects’ decision to subscribe to a safe harbor provision given the number of enforcement treatments (1 thru 4 treatments). The results in this round are similar to those in round 4 and 5:
Larger number of them subscribed to the safe harbor provision
Similar distributions for both groups (subscribed and not subscribed). The statistical tests for significance are shown below.
Higher subscription to the safe harbor provision for those who experienced higher enforcement treatments.
Figure 15: Round 6 Distributions between Enforcement Treatments and the Safe Harbor Provision Subscription Rate.
To determine the relationship enforcement treatments and subscription to a safe harbor provision in the sixth round, the Pearson’s χ2 was run and the results are shown on
Ho: The safe harbor subscription rate and number of enforcement treatments are independent in the sixth round.
Ha: The safe harbor subscription rate and number of enforcement treatments are not independent in the sixth round.
As mentioned before, the dependent variable is not a binary variable and the Fisher Exact test was not run. The results from the Pearson’s χ2 found a p-value was 0.009 in the sixth round, which shows that we can reject the null hypothesis confirming that there is association between enforcement treatments and subscription to the safe harbor provision. This confirms that enforcement treatments and subscription to the safe harbor provision are related at the community-standard 5 percent alpha protection level. The results confirm that a higher the number of enforcement treatments may lead to higher subscription to the safe harbor provision in the sixth round.
Table 25: Hypothesis 2a Round 6 Statistical Tests
Test Value df Asymp. p-value (2-
sided) Pearson’s χ2
11.680 a 3 0.009
Likelihood Ratio 8.367 3 0.039
Linear-by-Linear Association 4.529 1 0.033
a. 2 cells (25.0%) have expected count less than 5. The minimum expected count is .64
Furthermore, Table 26 shows the results from an effect size analysis using η2. The results found an η of 0.239 and an η2
of 0.057. Using Cohen’s interpretation guidelines (Ruscio, 2008), the results show a small effect size between enforcement treatments and subscription to the safe harbor provision. This implies that the enforcement treatments had a small effect on the safe harbor subscription rates in the sixth round. These results are in line with the Phi and Cramer’s V test results, which are statically significant at the community-standard 5 percent alpha protection level.
Although the results were statistically significant at the 0.05 alpha scientific community protection level, we can only generalize these effect sizes to the population if the
statistical power is above 0.8. In this case, the calculated post-hoc statistical power was 0.94. Therefore, we can generalize these results to the population.
Table 26: Hypothesis 2a Round 6 Effect Size Tests
Test Value Approx. p-value
η 0.239 -
Phi 0.239 0.009
Cramer’s V 0.239 0.009
IX.3.2.4Hypothesis 2a ANOVA Test Results
An ANOVA test was run as planned in the proposal. The ANOVA test evaluates the mean difference between the safe harbor subscription rate and number of enforcement treatments.
Table 27 shows, all the p-values were below the community-standard 5 percent alpha protection level. The results confirm a dependency between the safe harbor
subscription rate and the number of enforcement treatments (i.e., 1- 4). Since the number of enforcements had an impact on the safe harbor subscription rate, this corroborates with the idea that subjects with higher enforcement treatments would be more likely to
subscribe to the safe harbor provision.
Table 27: Hypothesis 2a ANOVA tests
Sum of Squares df Mean Square F p-value
Round 4 0.785 3 0.262 3.767 0.012
Round 5 3.587 3 1.196 20.022 0.000