Kolmogorov-Smirnov test
Quick Reference
To request a Kolmogorov-Smirnov test, type: stat=ks
on the tab statement.
The Kolmogorov-Smirnov statistic is a table-level statistic. It may be used to compare the cumulative frequency distributions of two samples to test whether they are from the same population. For example, you might wish to compare frequency of shopping in Safeway with frequency of shopping in Sainsbury to test whether one frequency increases more rapidly than the other.
To request a Kolmogorov-Smirnov test, include the option stat=ks on the tab statement. The table
must have three columns only: the first must be the base column, and the other two columns divide
the sample into the two groups to be compared. For example, in the shopping survey the table would require a base column and a column for each supermarket.
The first row of the table must be the base row, while the other rows represent some ordered classification of the respondents — numbers, numeric ranges, or measurements on some ordered scale — listed in increasing order of magnitude.
Notes for this test are:
• Both the row and column axes must contain only elements which are mutually exclusive. • When the rows comprise numeric ranges, remember that the test is based only on the figures in the table, and therefore the more information there is in the table, the more powerful the test will be. In other words, the more categories the better — you can lose information by
collapsing data too much into a few large categories. The counts in the cells of the table can be small, even zero.
• This test uses the sum of totalizable rows rather than the figures in the base row in its calculation.
Quantum User’s Guide Volume 3 82 / Descriptive statistics – Chapter 4
compare frequency of shopping between a sample of people who shop at Sainsbury and a sample who shop at Safeway, and you wish to know, not whether the average number of visits differ, but whether the distributions themselves differ. A Kolmogorov-Smirnov test is appropriate:
tab freq shop;stat=ks
ttlMonthly frequency of shopping at .... ttlBase: All respondents
l freq
val 157;Base;1–3 times;4–6 times;7–9 times;10 or more times l shop
col 167;Base;Sainsbury;Safeway
produces:
Figure 4.4 Kolmogorov-Smirnov test
The results of this test show a significance level that is close to 1. This indicates that there is little evidence to suggest a difference between the frequency distributions for the two supermarkets.
Monthly frequency of shopping at .... Base: All respondents
Base Sainsbury Safeway Base 605 304 301 Once 54 24 29 Twice 82 46 36 3 Times 129 65 64 4 Times 194 93 101 5-7 Times 91 51 40 8 or more times 55 24 31
KOLMGOROV - SMIRNOV VALUE = 0.120 SIGNIFICANCE LEVEL = 0.942
Quantum User’s Guide Volume 3 Descriptive statistics – Chapter 4 / 83
McNemar’s test
Quick Reference
To request a McNemar test, type:
stat=mcnemar [, element_text] [;options] as an element in the axis.
McNemar’s test is used to test for differences in a variable with just two possible values (for example, yes/no). It is most commonly used to test whether differences between ‘before and after’ measurements on the same sample indicate a real change or are simply due to chance.
To run a McNemar test, you will need a stat=mcnemar element in the axis. This must be preceded
by exactly two basic count elements representing the changes. For instance, one might count those
respondents answering yes before and no after, and the other those answering no before and yes after.
The first element in the axis must be a base element. If several McNemar tests are required in the same axis, each must follow a base element (use n11 if you don’t want to see these extra bases) and
a pair of elements representing the changes.
When looking at your table, you should remember that:
• The McNemar test is not concerned with the number of respondents whose opinions do not change.
• If the two counts are equal, the statistic will have a small but non-zero value.
• In the same way as for the one-dimensional chi-squared test, the sum of the two counts should be at least 10 to avoid giving misleading results.
84 / Descriptive statistics – Chapter 4
Here is an example. To examine whether trying out a washing powder affects respondents’ willingness to buy it, you might write this Quantum program:
tab change ban1
ttlQ9 Likelihood of Buying Washo ttlBase: All Trying Sample
l change n10Base
n01Yes then No;c=c34’12’.and.c48’45’ n01No then Yes;c=c34’45’.and.c48’12’ n03
stat=mcnemar,McNemar Value l ban1
col 12;Base;Male;Female col 15;AB;C1;C2;DE g Sex Social Class
g Base Male Female AB C1 C2 DE
This produces:
Figure 4.5 McNemar test
In this example we have obtained highly significant results for respondents in social classes AB and C1. If we look at the result for respondents in social class AB, we see that 48 changed their mind negatively after trying Washo and decided that they would not buy it. Only 16 respondents in the same social group made the opposite decision. The significant result shows that for respondents in social class AB, trying Washo adversely affects their likelihood of purchasing it.
Q9: Likelihood of buying Washo Base: All Trying Sample
Sex Social Class
Base Male Female AB C1 C2 DE Base 400 184 216 88 96 112 104 Yes then No 96 56 40 48 8 8 32 No then Yes 99 32 56 16 40 8 24
McNemar Value 0.27 6.01 2.34 15.02 20.02 0.06 0.88 0.600 0.014 0.126 0.000 0.000 0.803 0.350
Quantum User’s Guide Volume 3 Descriptive statistics – Chapter 4 / 85