Summary of modeling conclusions

We began this appendix by posing the dilemma of how to estimate unobserved (or at least unreported) killings in order to estimate total deaths. The modeling presented in Appendix 2 convinces us that there were probably a little over 10 000 killings of Kosovar Albanians in the period 20 March to 22 June 1999.

The largest direct estimate is comparable to this “best” model estimate, and different models produce similar estimates. We believe that we have made our case about the overall total number of killings and for the pattern of killings during the period in question. It is on this basis that we made the conclusions presented in the body of the report.

Appendix 3: Additional Sources on KLA and NATO Activity

Albanian Human Rights Group Albanian Media

Belgrade Center for Human Rights Center for Peace and Tolerance Daily Telegraph

Danas

Egyptian National Community in Kosovo European Community Monitor Mission European Roma Rights Center

Federal Republic of Yugoslavia (FRY) Ministry of Defense FRY Civil Defense Authorities

FRY Ministry of Foreign Affairs FRY Ministry of Information

FRY, Aide-Memoire on the Use of Inhumane Weapons in the Aggression of the North Atlantic Treaty Organization Against the Federal Republic of Yugoslavia. Belgrade, 15 May 1999

Fund for the Humanitarian Right The Guardian

Human Rights Board of Sandzak

Information Service of Church and National Assembly (Kosovo) International Committee of the Red Cross

International Criminal Tribunal for the Former Yugoslavia Koha Ditore

Kosovapress

Kosovar Media

Kosovo Verification Mission Local Church Councils (Kosovo) Los Angeles Times

NATO Operation Allied Force Update Open Society Institute

Organiztion for Security and Cooperation in Europe Organization of Families of Disappeared

Orthodoxy Press Politika

Report by Bishop Artemije “List of Killed and Kidnapped Serbs.” Republic of Serbia Ministry of Internal Affairs

RTS TV Belgrade Serbian Media

Serbian Orthodox Church Serbian Unity Congress NewsBits SVEDOK-Belgrade weekly Tanjug

United Nations High Commission for Refugees V.I.P. Daily News Report

References

American Bar Association Central and East European Law Initiative and the American Association for the Advancement of Science. 2000. Political Killings in Kosova/Kosovo, March-June 1999. Washington, DC: American Bar Association Central and East European Law Initiative.

Anderson, Margo and Stephen E. Fienberg. 2001a. Who Counts? Census-Taking in Contemporary America. Revised Paperback Edition. New York: Russell Sage Foundation.

Anderson, Margo and Stephen E. Fienberg. 2001b. Counting and estimation:

Methodology for Improving the Quality of Censuses. The U.S. 2000 Census Adjustment Decision. Paper presented at the International Conference on Quality in Official Statistics, Stockholm, Sweden, May 14-15, 2001.

Asher, Jana and Patrick Ball. 2001. Understanding Human Rights Violation Data through the Analysis of Circuits. To appear in the Proceedings of the American Statistical Association (Social Statistics Section).

Asher, Jana and Stephen E. Fienberg. 2001. Statistical Variations on an Ad-ministrative Records Census. To appear in the Proceedings of the American Statistical Association (Government Statistics Section).

Ball, Patrick. 2000a. Policy or Panic: The Flight of Ethnic Albanians from Kosovo, March–May 1999. Washington D.C.: American Association for the Ad-vancement of Science.

Ball, Patrick. 2000b. The Guatemalan Commission for Historical Clarification:

Intersample Analysis. Chapter 11 in Making the Case: Investigating Large Scale Human Rights Violations using Information Systems and Data Analy-sis, edited by Patrick Ball, Herbert Spirer, and Louise Spirer. Washington, DC: American Association for the Advancement of Science.

Belin, Thomas R. and Donald B. Rubin. 1995. A Method for Calibrating False-Match Rates in Record Linkage. Journal of the American Statistical Associ-ation.

Bishop, Yvonne M. M., Stephen E. Fienberg, and Paul H. Holland. 1975. Dis-crete Multivariate Analysis: Theory and Practice. Cambridge, MA: MIT Press.

Converse, N. and F. Scheuren. 2001. Workarounds in Survey Data Handling.

Submitted to the new Journal of Data.

Cressie, Noel and Paul W. Holland. 1983. Characterizing the Manifest Proba-bilities of Latent Trait Models. Psychometrika 48: 129–141.

Cormack, R. 1992. Interval Estimates for Mark-Recapture Studies of Closed Populations. Biometrics 48: 567–576.

Cowan, Charles Douglas. 1984. The effects of misclassifications on estimates from capture-recapture studies. Ph.D. diss., George Washington Univer-sity.

Darroch, John N., Stephen E. Fienberg, Gary Glonek, F.V. Gary, and Brian W.

Junker. 1993. A Three-Sample Multiple-Recapture Approach to Census Population Estimation with Heterogeneous Catchability. Journal of the American Statistical Association 88: 1137–1148.

Fienberg, Stephen E. 1972. The multiple recapture census for closed popula-tions and incomplete ^W contingency tables. Biometrika 59: 591–603.

Fienberg, Stephen E. 1980. The Analysis of Cross-Classified Categorical Data.

Second Edition. Cambridge, MA: MIT Press.

Fienberg, Stephen E., Matthew S. Johnson, and Brian W. Junker. 1999. Clas-sical multilevel and Bayesian approaches to population size estimation using multiple lists. Journal of the Royal Statistical Society, Series A 162:

383–405.

Fienberg, Stephen E., and Michael M. Meyer. 1983. Loglinear models and categorical data analysis with psychometric and econometric applications.

Journal of Econometrics 22: 191–214.

Ford, B. 1983. Hot Deck Imputation. Ch. 14 in vol. 2, part 4 of Incomplete Data in Sample Surveys, edited by William G. Madow, Harold Nisselson, and Ingram Olkin. New York: Academic Press.

Hogan, Howard. 1993. The 1990 Post-Enumeration Survey: Operations and Results. Journal of the American Statistical Association 88: 1047–1060.

Holland, Paul W. 1990. On the sampling theory foundations of item response theory models. Psychometrika 55: 577–601.

Human Rights Watch. 2001. Under Orders: War Crimes in Kosovo. New York:

Human Rights Watch.

Independent International Commission on Kosovo. 2000. The Kosovo Report:

Conflict*International Response*Lessons Learned. New York: Oxford Uni-versity Press.

International Working Group for Disease Monitoring and Forecasting. 1995a.

Capture-recapture and multiple-record systems estimation, I: History and theoretical development. American Journal of Epidemiology 141: 1047–

1058.

International Working Group for Disease Monitoring and Forecasting. 1995b.

Capture-recapture and multiple-record systems estimation, II: Applica-tions in human diseases. American Journal of Epidemiology 141: 1059–

1088.

Marks, E.S., W. Seltzer, and K. J. Kr´otki. 1974. Population Growth Estimation:

A Handbook of Vital Statistics Measurement. New York: The Population Council.

Oh, H. and F. Scheuren. 1983. Weighting Adjustment for Unit Nonresponse.

Chap. 13 in vol. 2, part 4 of Incomplete Data in Sample Surveys, edited by William G. Madow, Harold Nisselson, and Ingram Olkin. New York:

Academic Press.

Organization for Security and Cooperation in Europe. 1999. Kosovo/Kosova As Seen As Told: An Analysis of the Human Rights Findings of the OSCE Kosovo Verification Mission October 1998 to June 1999. Warsaw, Poland: OSCE Office for Democratic Institutions and Human Rights.

Peterson, C. G. J. 1896. The yearly immigration of young plaice into the Lim-fjord from the German Sea. Report of the Danish Biological Station to the Ministry of Fisheries 6: 1–48.

Physicians for Human Rights. 1999. War Crimes in Kosovo: A Population-Based Assessment of Human Rights Violations of Kosovar Albanians by Serb Forces.

Boston: Physicians for Human Rights.

Record Linkage Techniques. 1985. Record Linkage Techniques – 1985 Proceed-ings of the Workshop on Exact Matching Methodologies. Washington, DC:

U.S. Internal Revenue Service, Statistics of Income Division.

Record Linkage Techniques. 1997. Record Linkage Techniques – 1997 Proceed-ings of An International Workshop and Exposition. Washington, DC: Ernst and Young, LLP.

Rubin, Donald B. 1987. Multiple imputation for nonresponse in surveys. New York: Wiley.

Scheuren, F. 1985. Methodologic issues in linkage of multiple data bases.

Record Linkage Techniques – 1985 Proceedings of the Workshop on Exact Matching Methodologies. Washington, DC: U.S. Internal Revenue Service, Statistics of Income Division.

Sekar, C.C. and Deming, W.E. 1949. On a Method of Estimating Birth and Death Rates and the Extent of Registration. Journal of the American Statistical Association. 44:101-115.

Spiegel, Paul B. and Peter Salama. 2000. War and Mortality in Kosovo, 1998–

1999: An Epidemiological Testimony. Lancet 355: 2206–2211.

Splus, Insightful Corporation. 2001. “Generalizing the Linear Model.” Ch.

12 in S-PLUS 6 for Windows Guide to Statistics, Volume 1. Seattle, WA:

Insightful Corp.

Stata Corporation. 2001. Section on generalized linear models in Stata 7 Ref-erence Manual. Vol 1 A-G. College Station, TX: Stata Corporation.

Ball, Betts, Scheuren, Dudukovic, and Asher, 3 January 2002 67

Acknowledgments

This report would not exist without the collaboration of many individuals and organizations. It is based primarily on data provided by the American Bar As-sociation Central and East European Law Initiative (ABA/CEELI), the Ameri-can Association for the Advancement of Science (AAAS), The Center for Peace Through Justice, the Council for Defense of Human Rights and Freedoms, Hu-man Rights Watch, the Organization for Security and Cooperation in Europe, and the International Criminal Tribunal for the Former Yugoslavia. We are very thankful for the willingness of these organizations to share the results of their work with each other. Without this cooperative spirit, this project would not have been possible.

In 1999, Scott Carlson, Director of Central and East European Programs at ABA/CEELI, brought human rights organizations together to share their infor-mation regarding Kosovo and to benefit from their past experience in docu-menting human rights abuses. ABA/CEELI and AAAS published a report enti-tled Political Killings in Kosovo/Kosova, March-June 1999 in 2000.

In April 1999, Patrick Ball and Fritz Scheuren of AAAS, with Fron Nazi of the East-West Management Institute and the Institute for Policy and Legal Stud-ies began a study of the statistical patterns of refugee flows out of Kosovo.

This work was published as Policy or Panic: The Flight of Ethnic Albanians from Kosovo, March-May 1999. Organizations and individuals who contributed time, data, and other assistance to these earlier projects include Physicians for Hu-man Rights, the the HuHu-man Rights Center and the Department of Demography of the University of California-Berkeley, Fred Abrahams, Vasian Cepa, Blerina Kashari, Julia Belanger, Andrea Lako, Eric Stover, Dr. Sandra Eyster, Ilir Gocaj, and many others.

In addition, a number of individuals generously contributed their time, en-ergy, and expertise to this project. Matt Zimmerman wrote the software appli-cation used to perform the inter-system matching. He also designed the layout and provided crucial assistance with graphic design and technical editing. Patri-cia Hawkins provided early programming assistance. Sarah Churchill and Maya Goldstein oversaw the data coding of the ABA and OSCE data. Jason Sanders helped with coding and administrative issues. Jeff Henigson recoded the HRW data from the original interviews.

Support for the project was provided to ABA/CEELI from the Bureau of Democracy, Human Rights, and Labor at the U.S. Department of State and the U.S. Agency for International Development. While U.S. Government support was essential to the project, it should also be emphasized that at no time did U.S. Government personnel seek to infringe upon our independent manage-ment of the project of influence our substantive reporting. ABA/CEELI

struc-tured its relations with the U.S. Government as a “cooperative agreement” to ensure its independence in this respect. Consequently, this report was not sub-mitted for U.S. Government review, and any convergence with the views of the U.S. Government is purely coincidental.

In addition to support via a subcontract with ABA/CEELI, AAAS received support from the Institute for Civil Society and the John D. and Catherine T.

MacArthur Foundation.

Ball, Betts, Scheuren, Dudukovic, and Asher, 3 January 2002 69

Authors and Editors

Patrick Ball programmed the database, managed the quality control, and cre-ated statistical software, wrote portions of each section of the report, and pro-vided overall direction. Wendy Betts wrote the body of the report and coordi-nated the coding of the ABA/CEELI and OSCE data. Fritz Scheuren provided statistical guidance and wrote Appendix 1. Jana Dudukovic managed the data clerks, oversaw the matching process, coded the KLA data, and contributed to Appendix 1. Jana Asher programmed statistical routines and wrote Appendix 2. Patrick Ball and Jana Asher developed the modeling procedures. All of the authors jointly edited the report.

Scholarly Review Panel

A number of people reviewed the report. An international review team was chaired by Dr. Helge Brunborg (Senior Research Fellow, Statistics Norway), and consisted of Dr. Ronald Lee (Professor of Economics and Demography, Uni-versity of California-Berkeley); Dr. Francoise Seillier-Moiseiwitsch (Associate Professor of Statistics and Director of the Bioinformatics Research Center, Uni-versity of Maryland-Baltimore County, and Chair, Human Rights Committee, American Statistical Association), Dr. Jean-Louis Bodin (Past President, Inter-national Statistics Institute); Dr. Carlo Malaguerra (Director General of the Swiss Federal Statistical Office–SFSO); Dr. Philippe Eichenberger (Head of the Department of Statistical Methods, SFSO); Dr. Beat Hulliger (Deputy Head of the Department of Statistical Methods, SFSO). The reviewers provided exten-sive comments on two preliminary drafts of the report.

A number of additional reviewers worked with us less formally. These in-cluded Dr. David Banks (U.S. Department of Transportation), Herbert F. Spirer (Adjunct Professor, Columbia University School of International and Public Af-fairs Human Rights Program), Louise Spirer (independent scholar), and Dr.

Denise Albanese (Department of English, George Mason University).

This report is much stronger as a result of the frank critical assessments made by the reviewers, and we are very grateful to them. Of course, the authors bear sole responsibility for the analysis and opinions expressed in this study.

Authoring Organizations

AAAS Science and Human Rights Program

The Science and Human Rights Program of the American Association for the Advancement of Science (AAAS) seeks to protect the human rights of scien-tists and to bring the methods of science to human rights work. The Program develops and advances methods for human rights documentation and monitor-ing, fosters support for human rights among scientists, and conducts research on a variety of related issues. The Program’s work is based on the premise that respect for human rights is essential to the conduct of science. For more information about the Program and its activities, visit http://shr.aaas.org.

ABA Central and East European Law Initiative

The Central and East European Law Initiative (CEELI) is a public service project of the American Bar Association (ABA). The project is designed to advance the rule of law by supporting the law reform process underway in Central and East-ern Europe and the New Independent States of the former Soviet Union (NIS).

Through various programs, CEELI makes available the legal expertise of Amer-ican and European volunteers to assist emerging democracies in modifying or restructuring laws and legal systems.

The ABA/CEELI War Crimes Documentation Project (WCDP) began in May 1999 with funding from the U.S. Agency for International Development and the U.S. State Department. The WCDP has two objectives: 1) to assist efforts to investigate war crimes and prosecute perpetrators, and 2) to increase public awareness of war crimes, their prosecution, and the role of the International Criminal Tribunal for the former Yugoslavia (ICTY) in the process. On war crimes issues, ABA/CEELI has worked closely with several other nongovern-mental organizations, including the Coalition for International Justice (CIJ), Chicago-Kent College of Law, and The Center for Peace Through Justice. For more information about ABA/CEELI and its activities, visit

http://www.abanet.org/ceeli/.

About the Authors

Patrick Ball, Ph.D., is Deputy Director of the AAAS Science and Human Rights Program. Since 1991, he has designed information management systems and conducted quantitative analysis for large-scale human rights data projects for truth commissions, non-governmental organizations, tri-bunals and United Nations missions in El Salvador, Ethiopia, Guatemala, Haiti, South Africa, Kosovo, and Sri Lanka.

Wendy Betts, M.A., is Co-Director of the ABA/CEELI War Crimes Docu-mentation Project. She has contributed to a number of publications on international and internal conflict, and post-conflict transition.

Fritz Scheuren, Ph.D., is Vice President, Statistics, at the National Opinion Research Center, a research arm of the University of Chicago. He has extensive experience in record linkage both in survey and administrative settings. Currently, he is working full time on Native American issues.

Jana Dudukovic is an independent scholar studying under the tutelage of Louise Spirer.

Jana Asher, M.S., has wide research experience in small area estimation, administrative records, record linkage, and multiple systems estimation.

She is currently pursuing a Ph.D. in statistics at Carnegie Mellon Univer-sity under the guidance of Professor Stephen E. Fienberg.