The Quality of AP as Event Data Source

Because AP is the main empirical source for this study and has remained nearly untested as a potential event data source (for an exception, King and Lowe 2003), I investigate in this section that source’s usefulness and validity for event data analysis. The investigation is based on a comparison to Reuters, which has proven to be a rather rich and valid source for event data research (Reeves et al. 2006; Gerner et al. 1994; Huxtable and Pevehouse 1996). A com-parison of the two sources with special consideration of South Africa is possible since both wire services are available for the time period June 1987 to December 1996 and cover roughly the second half of the period under investigation in this study.

Density and Types of Reported Events within South Africa

I first analyze how well the source covers the region studied in this research project. I there-fore analyze the numbers and types of reported inner-South African events and compare them to Reuters’ reporting on the region. Inner-South African events are favored over international events concerning South Africa for this particular analysis since coverage of international events is expected to be more strongly biased in favor of the two news organizations domicile (United States in the case of AP and Britain in the case of Reuters, see next section for a detailed consideration of this potential bias).

Figure 13: Total Monthly Event Counts for South Africa (domestic) for AP and Reuters, 6/1987-12/1996

Figure 13 above shows the total monthly counts of coded domestic events in South Africa (that is, both source and target of the event are of South African origin).

Both time series in Figure 13 include mostly events between the South African gov-ernment and the ANC as its main opposition. Both series are coded from lead sentences using the SAID actor and event coding schemes. It clearly appears that Reuters covers events within South Africa two or three times more intensively than AP. However, the two series evolve quite simultaneously over time (correlation coefficient of 0.75). As a result, events coded from the two sources differ in level (average of number of events per month) but with a rather high covariance over time: when Reuters increases its reporting on South Africa, AP does so too; in months with only a few AP reports, Reuters reports on a lower intensity level as well.

Whether these differences involve coverage of different events or are just a differ-ence of reporting intensity will now be examined with a comparison of the distribution of different event types across the two sources. If coverage varies only in intensity but not in content, the distribution between different event types over a certain period of time is ex-pected to be similar between the two sources.

Figure 14: Event Counts per Event Category and Year for AP and Reuters

Note: Graphs differ in scale.

Figure 14 above illustrates the event distribution across the SAID/CAMEO event categories 01 through 20 comparing AP and Reuters for the selected years of 1988, 1990, 1992 and 1994.

The selected years display both intensively covered years (1990: De Klerk elected South African president, Mandela released from prison; 1994: first race-free democratic elections in South Africa) and rather purely covered years (1988, 1992). As expected, Reuters outpaces AP each year in every category. But although the two sources differ in event totals per category, AP and Reuters seem to report similar information due to a fairly similar distri-bution of the events across the different event categories. Table 13 indicates the Pearson correlation between the AP and Reuters event distribution for the four selected years. It is shown that the yearly event count distribution of AP highly correlates with Reuters’ yearly event count distribution.

Table 13: Correlations of AP and Reuters Event Counts per Event Category for South Africa (domestic) South Africa 1988 South Africa 1990 South Africa 1992 South Africa 1994

Correlation AP-Reuters 0.78*** 0.98*** 0.89*** 0.96***

***significant at the 0.01 level

As a result, AP and Reuters cover domestic political events in South Africa with dif-ferent intensity, but the two news sources generally focus on similar types of events, indicat-ing that the two sources report similar interaction patterns between actors of South African origin. AP therefore proves to be a useful source to monitor events within South Africa, even though the intensity of the coverage is lower than in the case of Reuters.

Density and Types of Reported International Events

As indicated above, it can be assumed that the domicile of a news source has an effect on the source’s focus on international events. International events involving or affecting the domicile country are expected to be more closely covered than events without such linkages to the home country of a particular news source. AP is therefore expected to report the U.S.-South African dyad more closely than Reuters, whereas Reuters covers the British-South African dyad better. Regarding third states such as Germany, Sweden and Switzerland, differences in the two organizations’ reporting are assumed to be less significant.

To test these assumptions, I first compare AP and Reuters regarding their reporting on international events with an involvement of their respective home country. Then, I will analyze the news sources’ coverage of events involving Germany, Sweden and Switzerland to find out how differently the two sources report on neither U.S. nor British international events.

Figure 15 shows that AP and Reuters differ in their reporting on U.S. and British South Africa relations. The two series for each news organization differ significantly in level and variance over time. Whereas the AP series reports on the USA>ZAF dyad on an average of 3.68 events per month (standard deviation 0.38), Reuters covers for the same dyad a mean of 6.42 events per month, on average almost twice as many events (standard deviation 0.48).

For the GBR>ZAF dyad, differences between the two series are even bigger: AP reports on average 0.65 events per month (standard deviation 0.11), whereas Reuters has a monthly average of 3.98 events (standard deviation 0.41).

Figure 15: Monthly Event Counts in USA>ZAF and GBR>ZAF, AP and Reuters 6/1987-12/1996

USA>ZAF GBR>ZAF

Note: Graphs differ in scale.

Consequently, the British Reuters outpaces the American AP not only in its reporting on the British-South African dyad but also—contrary to the initial assumption—in its cover-age of the U.S.-South African dyad. Despite their differences in level (number of events per month), the two sources’ reporting evolves at least somewhat similarly over time (covariance of 0.54 for the USA>ZAF dyad, 0.46 for the GBR>ZAF dyad).

Figure 16 below illustrates the number of events in the German-South African (DEU-ZAF), Swedish-South African (SWE-ZAF), and Swiss-South African (CHE-ZAF) dyads coded from AP and Reuters lead sentences from June 1986 to December 1996. Both sources cover the most events in the DEU-ZAF dyad (monthly mean of 0.75 for AP; 1.81 for Reuters). The SWE-ZAF and CHE-ZAF dyads are significantly less represented in the lead sentences obtained from the two sources. The CHE-ZAF dyad is especially poorly covered by AP lead sentences (monthly mean of 0.29). Coded from Reuters leads sentences, the CHE-ZAF and SWE-CHE-ZAF dyads show similar numbers of events (monthly mean of 0.59 for both dyads).

Figure 16: Total Monthly Event Counts for Selected Dyads for AP and Reuters, 6/1987-12/1996

AP Reuters

Note: Graphs differ in scale.

The statistical significance of the differences between the two sources’ reporting on the particular dyads is tested with a t-test (Table 14). The means for each dyad and source are calculated from normalized event counts to exclude the fact that Reuters shows an overall higher coverage of international events than AP. Normalized event counts are calculated by dividing the total number of events reported per dyad and year by the yearly total of all re-ported events.

Table 14: Mean of Yearly Normalized Event Counts in AP and Reuters (1988-1996)

Mean AP Mean Reuters Mean Diff. 95% Conf. Int. t-statistic

USA .0844 .0428 +.0416*** .0194 .0267 6.4246

DEU .0043 .0074 -.0031** -.0053 -.0008 -3.0920

SWE .0023 .0061 -.0038 -.0078 .0002 -2.1745

CHE .0007 .0033 -.0025*** -.0039 -.0015 -5.5889

GBR .0193 .0315 -.0122*** -.0189 -.0056 -4.2491

ZAF .2515 .2820 -.0142 -.0632 .0022 -2.1501

**significant at the 0.05 level *** significant at the 0.01 level

The comparison of normalized means of event counts per year and dyad (Table 14) confirms the initially presumed U.S. focus of AP. Between 1988 and 1996, around 8.4 percent of all international events concerning South Africa reported by AP include the United States either as source or target of the event. In Reuters, the share of U.S. related events is only 4.2 percent. Apart from the expected difference regarding the reporting of events related to Brit-ain, significant differences between the two sources exist with regard to Germany, Sweden and Switzerland. On these dyads, Reuters reports significantly more events than AP compared to the two sources’ overall reporting on international South Africa events. No significant differences can be found considering the Swedish-South African dyad.

Language

The poor coverage of Germany, Sweden and Switzerland in both AP’s and Reuters’ reporting may not only be due to the fact that these three countries represent less important third states for the two big international news organizations. An English-language bias can have an influ-ence too. I therefore compare the number of South Africa related events with German or Swiss involvement (as AP and Reuters reported them) with German-language news sources.³⁸ The sources of reference are the Neue Zürcher Zeitung (Switzerland) and the Süddeutsche Zeitung (Germany). The New York Times (United States) and The Times (England) are also included in the analysis. All four newspapers are traditionally considered to have an interna-tional focus.

Table 15: Number of South Africa Stories for Germany and Switzerland in different Sources

Germany Switzerland

NZZ SDZ NYT Times AP NZZ SDZ NYT Times AP

Orig. CH D USA UK USA CH D USA UK USA

Lang. Germ. Germ. Engl. Engl. Engl. Germ. Germ. Engl. Engl. Engl.

Type Paper Paper Paper Paper Wire Paper Paper Paper Paper Wire

1991 ^{- 1)} 3²⁾ 9 1 10 ^{- 1)} 1²⁾ 0 5 3

1992 ^{- 1)} 10 5 6 5 ^{- 1)} 2 0 1 1

1993 6 15 6 7 4 30 2 1 2 1

1994 10 39 4 6 5 35 3 0 3 3

1995 7 30 11 4 10 23 1 1 1 5

1996 17 54 9 6 7 28 0 1 1 5

1) not electronically available before 1993 ²⁾ from Feb 11, 1991 Database: Lexis-Nexis

Search strings (title and lead-sentence): “südafrika!” AND “deutsch!” (SDZ); “south africa!” AND “german!” (NYT, Times, AP); “südafrika” AND “schweiz!” (NZZ); “south africa!” AND (“switzerland” OR “swiss”) (NYT, Times, AP); sports stories and stock market reports excluded.

Sources: NZZ=Neue Zürcher Zeitung, SDZ=Süddeutsche Zeitung, NYT=New York Times, Times=Times and Sunday Times (London), AP=Associated Press

Table 15 indicates that the two included German-language newspapers, the Neue Zürcher Zeitung and the Süddeutsche Zeitung, report more South African stories for the Ger-man-speaking countries of Germany and Switzerland than the English-language news sources. It also becomes clear, as expected from the outset, that both the Swiss and German newspapers report significantly better on events involving their respective home countries. AP reports the most Germany or Switzerland related events but still reports on a smaller scale than the two German-language newspapers.

38 For Sweden, unfortunately, no similar newspaper was available for this test.

Conclusion

The results from the validity assessment of AP as a source for event data analysis can be summarized as follows:

AP proves to be a valid source for the coding of events regarding South Africa. The source covers the region less intensively than Reuters, which has proved to be a valuable source for event data analysis in many studies. But the two sources report similar patterns of behavior. That is, their respective reporting on South Africa differs rather in intensity than content.

Reuters seems to report international events regarding South Africa generally more densely than AP. The British Reuters wire produces more events not only for the British-South African dyad but also for the U.S.-British-South African dyad. Reuters also reports more closely on events including other European countries such as the three under study in this research project, namely West Germany, Sweden and Switzerland. When lead sentences are coded, English-language sources miss a significant number of events, including those in non-English-speaking countries such as Germany and Switzerland.

For less prominent dyads, AP shows a weakness when events are coded from lead sentences. Thus, I examine in the next section if full-text coding is able to provide a broader data basis for these so-called small dyads.