Making sense of maternal mortality estimates

(1)

For the PDF version of this paper and other related documents, visit

Working Paper Series • Number 11 • November 2010 • WORKING PAPER

Making sense of maternal

mortality estimates

Carla AbouZahr

(2)

Author details

Carla AbouZahr, Health Information Systems Knowledge Hub, School of Population Health, The University of Queensland

About this series

The Health Information Systems Knowledge Hub’s Working Paper Series is the principal means to

disseminate the knowledge products developed by the hub as easily accessible resources that collectively form a lasting repository of the research findings and knowledge generated by the hub’s activities. Working papers are intended to stimulate debate and promote the adoption of best practice for health information systems in the region. The series focuses on a range of knowledge gaps, including new tools, methods and approaches, and raises and debates fundamental issues around the orientation, purpose and functioning of health information systems. Generally, working papers contain more detailed information than a journal article, are written in less-academic language, and are intended to inform health system dialogue in and between countries and a range of development partners.

Many working papers have accompanying products such as summaries, key points and action guides. The full range of documents, as well as other resources and tools, is available on the Health Information Systems Knowledge Hub website at www.uq.edu.au/hishub/publication-tools. The opinions or conclusions expressed in the Working Paper Series are those of the authors and do not

necessarily reflect the views of institutions or governments.

Published by the Health Information Systems Knowledge Hub,

School of Population Health The University of Queensland Public Health Building,

Herston Rd, Herston Qld 4006, Australia Please contact us for additional copies of this publication, or send us feedback: Email: hishub@sph.uq.edu.au Tel: +61 7 3346 4732

Fax: +61 7 3365 5442 www.uq.edu.au/hishub

(3)

Health In

forma

tion S

yst

ems K

no

wledg

e Hub

Acronyms and abbreviations

��

2

Summary ��

3

Introduction

��

5

Sources of maternal mortality data ��

9

Monitoring rare events ��

16

Tracking trends ��

18

Global estimates of maternal mortality

��

21

Resolving the decision-maker’s dilemma ��

24

Conclusions ��

28

References ��

30

(4)

Health In

forma

tion S

ys

te

ms K

no

wledg

e Hub

AIDS acquired immunodeficiency syndrome

DHS demographic and health surveys

GFR general fertility rate

HMIS health management information system

HIV human immunodeficiency virus

ICD International Statistical Classification of Diseases and Related Health Problems

ICD-10 International Statistical Classification of Diseases and Related Health Problems, 10th revision

IHME Institute for Health Metrics and Evaluation

LFR lifetime risk

MMrate maternal mortality rate

MMratio maternal mortality ratio

MDG Millennium Development Goal

NFHS National Family Health Surveys

PMDF proportional maternal among deaths of females

RAMOS reproductive age mortality studies

RAPID rapid ascertainment process for institutional deaths

SRS sample registration system

TFR total fertility rate

UN United Nations

UNICEF United Nations Children’s Fund

UNFPA United Nations Population Fund

WHO World Health Organization

(5)

Health In

forma

tion S

yst

ems K

no

wledg

e Hub

International agencies and academic institutions have developed statistical modelling techniques to improve comparability between data sources. The models can also generate values for countries and periods with little empirical data. These modelled estimates are useful for advocacy, planning, making strategic decisions and identifying research priorities. However, they are not designed for country monitoring or for assessing the effectiveness of measurement methods.

Planners and managers need high-quality, up-to-date data at both national and subnational levels. National data on maternal mortality may hide major disparities between geographic areas, socioeconomic groups and ethnic groups. The extent and persistence of these differences can be estimated by tracking disparities in indicators of maternal health care. Examples of useful indicators are coverage of maternity care; educational attainment and other socioeconomic indicators; and other pregnancy-related outcomes, such as rates of stillbirths and early neonatal mortality.

The following recommendations should improve the understanding and interpretation of maternal mortality data.

• Assess all metadata carefully, to avoid inappropriate

comparisons between different methods and times. Include definitions, data sources and collection methods, margins of uncertainty and statistical methods for adjustment and imputation.

• Remember the hierarchy of data sources for maternal

mortality. Preferred sources are those that generate ongoing, population-based, unbiased estimates. These sources include complete civil registration and sample registration. When these sources are unavailable or are of inadequate quality, household surveys and censuses can be used for population-based estimates. However, such sources only produce data on an occasional basis, and the values contain uncertainties that create problems for monitoring trends. Facility-based data are rarely representative of the total population because not all deliveries take place in health care facilities and not all health facilities report maternal deaths accurately. • Avoid over-interpreting specific values and do not

concentrate exclusively on point estimates. For example, differences in maternal mortality ratios of 600 and 650 per 100 000 live births are not relevant. Remember the confidence intervals or boundaries This working paper aims to improve the understanding

and interpretation of statistics on maternal mortality. The paper does not primarily discuss methods to improve the measurement of maternal mortality, although these methods are mentioned when appropriate. Instead, it shows how to interpret the data that are already available.

Maternal mortality is often described as hard to measure and difficult to monitor. This statement contains some truth, but similar challenges also arise in measuring and monitoring other cause-specific deaths, such as those from acquired immunodeficiency syndrome (AIDS), tuberculosis and malaria. This working paper clarifies the issues associated with measuring maternal mortality, and therefore, makes the interpretation of data from different sources easier. This paper should also increase our understanding of how to assess the reliability of different estimates.

The levels of, and trends in, maternal mortality provide important markers of societal health and development and the performance of health systems. These markers are therefore included among the Millennium Development Goals (MDGs). However, some uncertainties and misunderstandings exist around measurement. Different methods of measurement produce varied results that cannot be compared over time or between countries. Thus, decision-makers face multiple, often divergent values that are difficult to interpret and use. Although other indicators such as child mortality also have this problem, the size of the discrepancies in the data on maternal mortality makes interpretation particularly difficult.

Many reasons underlie these data discrepancies, but they are mainly due to differences in:

• definitions of maternal mortality

• data sources and methods of data collection • time references for the data

• sources and levels of uncertainty around the values

• populations described by the data

• adjustments to the data elements to account for known biases and missing values

• statistical models used to ascribe missing values

• independent variables used to predict maternal

mortality.

(6)

Health In

forma

tion S

ys

te

ms K

no

wledg

e Hub

The only strategies for data collection that yield regular, ongoing, small-area data on maternal mortality are civil or sample registration, with medical certification of the cause of death (or verbal autopsy if certification is infeasible). These systems generate benefits for monitoring mortality and other demographic indicators. The civil registration system can also provide the starting point for confidential maternal and perinatal enquiries, which produce important information on the causes of, and the avoidable factors associated with, maternal mortality.

of uncertainty associated with each set of estimates. For general advocacy purposes, consider using bands. In countries with low mortality, use relatively narrow bands: <20; 21–39; 40–59; 60+. In countries with high mortality, use wider bands: 300–499; 500–699; 700–899; 900+. In countries with intermediate levels of mortality, the bands might be 50–99; 100–199; 200–299; 300+. Any maternal mortality ratio higher than 500 per 100 000 requires action.

• Use the maternal mortality ratio with care, especially when the absolute number of maternal deaths is low. In this case, random fluctuations in the number of deaths produce large yearly variations in the maternal mortality ratio. Use three or five-year moving averages to minimise the variations. Focus on surveillance of individual cases, coupled with facility audits and confidential enquiries, to discover the underlying causes of deaths and the potentially avoidable factors.

• As well as the maternal mortality ratio, use the range of maternal mortality indicators (eg the maternal mortality rate, the proportional maternal among deaths of females [PMDF] and the lifetime risk). Also track the absolute number of maternal deaths. • Assess the reliability of maternal mortality figures by

comparing them with other mortality data such as infant and child mortality, and with other indicators including fertility, coverage of maternal health care, availability of maternal health care services, female education, nutrition, and women’s status in society. • Use estimates developed by external agencies for

comparison or to check values reported by countries. Estimates close to the country-generated data reinforce the overall picture. When the estimates differ radically, consider performing a study to understand why the differences exist; differences may relate to the quality of the reported data, to the assumptions behind the estimates, or both.

• Rather than focusing solely on the maternal mortality

levels, decision-makers should use active surveillance to identify cases of maternal deaths. Use confidential enquiries and audit techniques to learn how to prevent such deaths.

(7)

Health In

forma

tion S

yst

ems K

no

wledg

e Hub

help decision-makers understand their current position and the direction for progress. Key issues addressed are: • definitions of maternal death

• indicators of maternal mortality

• sources of data on maternal mortality

• monitoring rare events

• dealing with uncertainty

• tracking trends

• understanding global estimates • interpreting multiple values.

The working paper also provides a list of additional resources on monitoring maternal mortality.

Definitions of maternal mortality

According to the International Statistical Classification

of Diseases and Related Health Problems, 10th revision (ICD-10; WHO 1992), a maternal death is defined as ‘the death of a woman while pregnant or within 42 days of termination of pregnancy … from any cause related to the pregnancy or its management, but not from accidental or incidental causes’ (Box 1). Maternal deaths are further subdivided into those due to obstetric complications such as eclampsia, obstructed labour, puerperal sepsis and obstetric haemhorrage (ie direct maternal deaths) and those due to existing conditions aggravated by pregnancy or its management (ie indirect maternal deaths). Deaths among pregnant women unrelated to the pregnancy are classified as incidental. While the definition seems straightforward, its

application in practice creates problems, especially when medical certification of the cause of death is unavailable or of inadequate quality, or when deaths occur at home, as is often the case. Some causes of maternal deaths are hard to identify, especially when the death occurs very early in the pregnancy; for example, death due to ectopic pregnancy. Abortion-related deaths are often missed—particularly when abortion is illegal or stigmatised—because a deceased woman may have hidden her pregnancy. Deaths in the late postpartum period are generally less likely to be reported than early postpartum deaths, partly because they cannot be linked to reportable birth outcomes. Lack of awareness of pregnancy at the extremes of maternal age (youngest and oldest) can also result in missed maternal deaths.

The decision-maker’s dilemma

A citizen in Nepal, faced with the set of maternal mortality figures in Table 1, could be forgiven for feeling confused. The situation facing people in Zimbabwe seems even more unclear. Are things getting better or worse? Which of these numbers should be used to help determine policy and guide programmes? What can explain these large differences from year to year? These questions worry national decision-makers under pressure to show progress towards internationally agreed goals and targets such as the MDGs. The demands for data on maternal mortality are particularly acute. A shortage of primary data on the levels of, and trends in, this indicator exists in most developing countries.

Table 1 Maternal mortality data, Nepal and Zimbabwe, selected years

Nepal MMR per

100 000 live births Year Zimbabwe MMR per 100 000 live births Year

539 1993 283 1994 281 2003 695 1999 830 2005 880 2005 88 2007 555 2006 240 2008 725 2007 380 2008 624 2008 229 2009 790 2008 316 2011 329 2011

MMR = maternal mortality ratio

Source: WHO database (2011) (unpublished)

This working paper offers guidance on interpreting and using different estimates of maternal mortality. The guidance is primarily intended for decision-makers and development partners who use the available data. The working paper is not a manual on methods to measure

maternal mortality.1_{Rather, the paper focuses on}

interpreting and using data that are already available. The goal of this paper is to improve the understanding of data to avoid confusion and contradictions. The paper shows how different values arise from variations in definitions, data sources, data collection methods and techniques for statistical imputation. It proposes strategies for interpreting data on maternal mortality to

1 See, for example, www.maternal-mortality-measurement.org

Introduction

(8)

Health In

forma

tion S

ys

te

ms K

no

wledg

e Hub

Until 2010, indirect maternal deaths from HIV were coded to the ICD chapter 1, according to ICD rule 5.8.3 (in volume 2), and were not included in the overall tally of maternal deaths. An amendment to ICD-10 introduces code 098.7 to identify indirect maternal deaths from HIV. These deaths are those in which HIV complicates the pregnancy or the delivery. However, incidental deaths from HIV, in which the women were also pregnant, are still excluded from the maternal

mortality ratio.2_{The difficulties of rigorously applying}

the ICD definitions explain the well-documented fact that many maternal deaths are misclassified as non-maternal. This misclassification occurs even in countries with complete, high-quality systems of civil registration and medical certification, including France (Horon 2005), the United Kingdom (Lewis 2007) and the United States of America (Deneux-Tharaux 2005). Often, maternal deaths are misclassified because the person certifying the death is unaware that the woman was pregnant (or had recently been pregnant) when she died. Therefore, the World Health Organization (WHO) advises that death certificates include a specific question about the pregnancy status of deceased women aged between 15 and 49 years.

In recognition of the difficulty of correctly determining cause of death in developing countries, where most deaths occur outside medical facilities, the ICD

introduced an additional category of pregnancy-related death. This definition specifies a time of death rather than a cause of death, because the specific cause of death need not be determined. In most settings, only small differences exist between pregnancy-related and maternal deaths. However, where HIV prevalence is high, the difference can be substantial. This difference is important from a measurement perspective, and is an issue to which we return later.

No clear consensus exists on whether pregnancy-related deaths, as recorded by surveys or censuses, overestimate or underestimate ‘true’ maternal deaths. In theory, pregnancy-related deaths should overestimate maternal deaths because they include deaths from incidental causes. However, pregnancy-related deaths may be underreported if household respondents are unaware of the pregnancy status of the deceased woman, particularly in deaths related to unsafe abortion. Therefore, reported pregnancy-related deaths are often

2 Given the ongoing clinical uncertainty around the relationship between maternal and HIV-related causes of death, these deaths are likely to remain a source of misclassification.

Box 1

Maternal death: ‘… the death of a woman while pregnant or within 42 days of termination of

pregnancy…from any cause related to the pregnancy or its management, but not from accidental or incidental causes’

Maternal deaths are classified as: • direct — obstetric causes

• indirect — existing conditions aggravated by pregnancy or its management

• incidental — unrelated to pregnancy.

Late maternal death: the death of a woman from direct or indirect obstetric causes, more than 42 days but less than one year after termination of pregnancy.

Pregnancy-related death: death of a woman while pregnant or within 42 days of terminations of pregnancy, irrespective of cause of death.

Source: WHO (1992)

Miscoding of maternal death occurs more frequently when the death results from an indirect cause such as cerebrovascular or cardiovascular disease. Medical practitioners responsible for certifying deaths may not adequately understand the ICD rules related to the underlying causes of death. Maternal deaths may also be wrongly coded to avoid blame and litigation, or because of a desire to protect confidentiality and avoid stigmatizing conditions such as abortion or HIV. Differentiating between indirect and incidental causes of death can also be difficult and is, to some extent, subjective. For example, suicides and homicides among pregnant women may be correctly classified as indirect causes of maternal death (Oates 2003). Deaths in pregnant women who are HIV-positive are generally assumed to be caused by advancing disease and related comorbidities such as tuberculosis and malaria. However, evidence shows that HIV infection might also increase the risk of maternal death by increasing the risk of dying from direct obstetric complications. Compromised immune status and disease interactions provide an immediate biological explanation for the increased risk of maternal death in pregnant women who are HIV-positive (NCCEMD 2008).

(9)

Health In

forma

tion S

yst

ems K

no

wledg

e Hub

in relation to each pregnancy, in other words, the obstetric risk.

A close relationship exists between the number of maternal deaths and overall fertility. As fertility declines and fewer live births occur, fewer maternal deaths also occur, even if the obstetric risk remains high. The maternal mortality rate captures this relationship between maternal mortality and fertility. The maternal mortality rate is defined as the number of maternal deaths divided by the population at risk (ie the number of women of reproductive age), multiplied by 1000. A simple mathematical relationship links the ratio and the rate: the maternal mortality ratio is the maternal

mortality rate divided by the general fertility rate (GFR).3

Most women become pregnant more than once, and they face the obstetric risk for each pregnancy. The adult lifetime risk of maternal mortality is an indicator that reflects this combination of obstetric risk and fertility. This indicator summarises the risk of maternal death over a woman’s reproductive life span.

One other indicator is the ‘proportional maternal among deaths of females’ (PMDF) of reproductive age. This is calculated as the number of maternal deaths divided by the total deaths among women of reproductive age. Reproductive age is usually defined as females aged

3 General fertility rate = Number of live births x 100

Number of women aged 15–49

assumed to approximate true maternal deaths. However, the extent of the trade-off or how that trade-off varies in different contexts, particularly when levels of abortion or HIV are high, is arguable.

Modern life-sustaining procedures and technologies can prolong the lives of women with severe birth complications and, therefore, some deaths occur beyond 42 days postpartum. Because these deaths do not strictly meet the ICD definitions, they are often excluded from the tally of maternal deaths in civil registration systems. The ICD has added the definition of ‘late maternal death’ to capture deaths occurring between six weeks and one year postpartum (Box 1). Some countries, particularly those with more developed systems of health care and civil registration, use this definition.

Indicators of maternal mortality

The most commonly used indicator of maternal mortality is the maternal mortality ratio—that is, the number of maternal deaths divided by the number of pregnancies, with a multiplier of 100 000 (Table 2). Conventionally, the number of live births is used in the denominator because it is easier to measure than the number of pregnancies. The ratio reflects the risk faced by women

Table 2 Maternal mortality indicators

Indicator Definition

Maternal mortality ratio (MMratio): expresses obstetric risk Number of maternal deaths x 100 000 Number of live births

Maternal mortality rate (MMrate): risk of maternal death among women of

reproductive age Number of maternal deaths x 1000 Number of women aged 15–49 years The relationship between the MMrate and the MMratio MMratio = MMrate

General Fertility Rate The lifetime risk (LTR) of maternal death: chances of a woman dying from

maternal causes over her 35-year reproductive life span LTR = 1 – (1 – MMRatio/100 000) TFR

The proportion of maternal deaths in females (PMDF) among women of

reproductive age Number of maternal deaths x 100 Total deaths in women aged 15–49 years TFR = total fertility rate. This is the average number of children that would be born to a woman over her lifetime if she were to experience the exact current age-specific fertility rates through her lifetime, and were to survive from birth through the end of her reproductive life. The TFR is obtained by summing the single-year age-specific rates at a given time.

(10)

Health In

forma

tion S

ys

te

ms K

no

wledg

e Hub

Tracking the absolute numbers of deaths is also important, especially in small countries or places where maternal mortality levels are low. A simple distribution of the number of deaths by time of occurrence (during pregnancy, during the intrapartum period and postpartum) provides valuable information for policymaking and programming.

15–45 years, but the age range used can vary. The PMDF is particularly useful when information on the number of live births or the number of women of reproductive age is not readily available; for example, in hospital-based studies of maternal mortality or studies of reproductive age mortality (see below). Note that the PMDF reflects the distribution of mortality by cause rather than the overall level of mortality and is affected by competing causes of death among women of reproductive age, particularly AIDS. Extending the upper end of the age range to 49 years can have a marked impact on the PMDF.

Each of the indicators described above reflects different aspects of the level of maternal (or pregnancy-related) mortality. The maternal mortality ratio has received the most attention among decision-makers, program managers and the donor community. However, using more than one indicator is advisable when analysing levels of, and trends in, maternal mortality. For example, valuable insights into maternal health can be gleaned by examining the relationship between maternal mortality and fertility. In settings where the maternal mortality ratio (ie the obstetric risk) is high, the maternal mortality rate (the risk per woman of reproductive age) may decline because of falling fertility. A decline in the absolute number of births results in fewer maternal deaths, even without improvements in the uptake of maternal health interventions. Likewise, the PMDF may change substantially if the cause of death structure is altered (eg due to AIDS mortality). Thus, trends in maternal mortality should be interpreted in light of both the risk per woman and the risk per birth, and take account of changes in fertility and the distribution of deaths by cause. Ideally, measures of maternal mortality should reflect:

• the annual risk of maternal death per woman

(MMrate)

• the obstetric risk (MMratio)

• the overall level of fertility (general or total fertility rates)

• the overall level of mortality in the population and its distribution by age, sex and cause (PMDF).

(11)

Health In

forma

tion S

yst

ems K

no

wledg

e Hub

results. Verbal autopsy techniques that involve interviews with family members of the deceased are used to determine probable cause of death (Box 2) (WHO 2007). However, the accuracy of the diagnosis of the cause of death tends to vary with this technique, as does the ability to detect trends in cause-specific mortality.

Box 2

Verbal autopsy (ie interviews with family or community members) is used when medical certification of the cause of death is unavailable to assign a cause of death. Records of births and deaths are collected periodically among defined populations, and interviews are conducted with family members (and health care providers when possible) to identify signs and symptoms before the death. These signs and symptoms are then classified into ICD-compatible, medical causes of death. Physicians traditionally do the classification, but more recently automated algorithms are used to determine the cause of death. These algorithms appear to work well. Verbal autopsy is a useful tool where most deaths occur outside health care facilities, but it has limitations.

• The causes of deaths in reproductive-aged women may be misclassified.

• The method may not identify maternal deaths that occur in early pregnancy (eg ectopic pregnancy or abortion-related deaths) and indirect causes of maternal death (eg malaria). • The accuracy of information on the cause of

death depends on the extent of family members’ knowledge of the events leading to the death, the skill of the interviewers and the competence of the physicians who make the final diagnosis and do the coding.

• The WHO standard verbal autopsy tool is quite complex to administer and requires extensive inputs from well trained medical practitioners. Research is underway to develop statistical methods that can diagnose the cause of death from patterns of reported symptoms, which would remove the need for expensive and time-consuming reviews by physicians.

Civil registration

The best routine source of data on maternal deaths is a civil registration system that assures the ongoing, permanent, compulsory and universal recording of the occurrence and characteristics of vital statistics,

including births and deaths.4_{A civil registration system}

also provides for the medical certification and coding of causes of death according to the ICD rules. Civil registration has dual purposes: administrative and legal on the one hand; and statistical, demographical and epidemiological on the other. For individuals, the civil records of birth and death provide essential legal documentation for many purposes. From a population perspective, birth and death records provide important information on public health. Vital statistics derived from civil registration are the only nationally representative, continuously available source of information on cause-specific mortality.

The record of deaths among women of reproductive age derived from civil registration is often the first step in conducting a confidential enquiry into maternal deaths. However, as already noted, even when civil registration is complete, maternal deaths may be misclassified. Active surveillance measures, such as record matching and introducing a pregnancy checkbox on the death certificate, help minimise misreporting (MacKay et al 2000, Chang et al 2003, Hoyert 2007).

South Africa uses a separate Maternal Death Notification Form that supplements the standard death certificate with more detailed information about the circumstances of a maternal death (Pattinson 1999).

Sample registration with verbal autopsy

A comprehensive system of civil registration and vital statistics may not be feasible in most developing countries in the near future. However, some countries (notably India and China) have introduced sample enumeration systems for vital statistics, which work fairly effectively although they do not include the issuance of legal documents certifying birth or death (Mari Bhat 2002, Yang et al 2005, Jha et al 2006). These systems use longitudinal tracking of demographic events in randomly selected areas, thus generating nationally representative

4 For more details, please refer to the Principles and Recommendations

for a Vital Statistics System, revision 2 (United Nations publication, Sales No. 01.XVI.10).

(12)

Health In

forma

tion S

ys

te

ms K

no

wledg

e Hub

Sample size requirements are significantly reduced when sisterhood or sibling survival methods are used to indirectly measure maternal mortality in household surveys (Stanton et al 1997, Hill et al 2006). With this method, a representative sample of respondents is interviewed about the survival of their adult sisters to determine: the number of ever-married sisters; how many are alive; how many are dead; and how many died during pregnancy, delivery or within six weeks of pregnancy. There are two variants of sisterhood methods. The original indirect method, which has been used; for example, in multiple indicator cluster surveys supported by the United Nations Children’s Fund (UNICEF), uses quite small sample sizes. However, this method is not appropriate in settings in which fertility levels are low (ie a total fertility rate <4) or in which substantial migration or other social dislocation has occurred. The direct sisterhood method, used in the demographic and health surveys (DHS) supported by the United States Agency for International Development, collects more information than the indirect method (eg the age of all siblings, age at death and year of death of those deceased). However, this method requires larger sample sizes, more questions and a more complex analysis (WHO 1997).

Household surveys

Population-based household surveys are widely used to generate data on maternal mortality in many developing countries. In addition to providing data on child and maternal mortality, surveys produce information on fertility, contraception, maternal health, nutrition, use of services, and the knowledge and practices related to maternity care. Maternal deaths are identified using either direct or indirect methods. Direct methods involve asking respondents about recent deaths in the household and, when deaths are identified in women of reproductive age, asking extra questions about the timing of the death in relation to pregnancy. These methods can generate estimates with a reference period of about 2–3 years before the survey, which is acceptable for monitoring purposes. However, these methods require large sample sizes to produce reliable estimates, and the estimates of maternal mortality will have very wide confidence intervals, making it difficult to monitor changes over time. For example, a 2007 household survey in Ghana involved 240 000 households and produced estimates with a confidence interval of ± 30 per cent (see below) (Ghana Statistical Service et al 2009). By comparison, typical confidence intervals for estimates of child mortality are about ± 10 per cent.

0 200 400 600 800 1000 1200 1400 1993 2003 1992 1999 1992 2000 2004 2000 2005 1996 2004 2000 2005 1996 2001 2005 Upper bound Lower bound Point estimate

India

Nepal Malawi Rwanda Tanzania Uganda Zambia

Figure 1 Confidence intervals for estimated values of maternal mortality produced from selected household surveys

Maternal mortality, selected countries, 1992-2005

Number of ma

ternal de

(13)

Health In

forma

tion S

yst

ems K

no

wledg

e Hub

In principle, a census allows the identification of deaths in a household in a relatively short reference period (1–2 years), and thereby provides estimates of recent maternal mortality (Stanton et al 2001). However, the results must be adjusted for the completeness of births and deaths declared in the census, and for distortions in age structures, to produce reliable estimates (WHO et al 2010). Techniques for demographic adjustment compare fertility and mortality data between the current and the most recent census. This technique introduces a distortion into the final adjusted values because any adjustment relates to a period midway between the two censuses. A further disadvantage is that censuses are usually conducted at 10-year intervals, which limits their use for monitoring maternal mortality. Because of these limitations, WHO advises that a census should be viewed as a source of additional comparative data, rather than a primary data source, for estimates of maternal mortality. Currently, no methods can quantify the uncertainty of estimates of maternal mortality derived from census data. Uncertainty may arise from a combination of under or over-reporting of adult deaths, of maternal deaths as a proportion of all deaths of women of reproductive age or of live births. Underreporting occurs because a census will miss deaths in single-person households or because a mother’s death may result in breakup of the household. Overreporting may occur if deaths are reported as pregnancy-related to avoid mentioning potentially stigmatising conditions such as HIV/AIDS.

Because a census involves visiting all households in the country, sampling errors are not an issue and calculating confidence intervals, as is commonly done for survey-based estimates, is not an option. To date, uncertainty has been expressed by documenting the adjustment factors required for measuring adult female mortality and live births after evaluating the data. No formal means exist to evaluate the proportion of maternal deaths in females of reproductive age. Results vary across countries, but generally underreporting is more common than overreporting. Adult female deaths tend to be more underreported than live births. Several counties in Africa, Asia and Latin America that have evaluated their census-based data on maternal mortality found sizeable adjustment factors, particularly for adult female deaths, which suggests substantial uncertainty in the maternal mortality ratio (Hill et al 2009).

The questions used in a census identify pregnancy-related rather than true maternal deaths. True estimates All household survey methods actually measure

pregnancy-related mortality, rather than maternal mortality. Sisterhood methods are relatively cost effective because the sample sizes are smaller than those of surveys using direct methods. However, the wide confidence intervals make trend analysis difficult (Figure 1). Also, sibling survival methods produce retrospective rather than current estimates of maternal mortality—around 5–7 years before the survey for direct sisterhood methods and 10–12 years before the survey for indirect methods. Sibling survival methods (and censuses) may underestimate overall mortality because of inherent biases in survey data (eg survival and recall bias) (Obermeyer et al 2010). Statistical methods that correct for these biases have been developed, but are not yet widely used (Gakidou and King 2006). Whatever the statistical method used to produce estimates of pregnancy-related (or maternal) mortality, remember that the data are only as good as the survey itself. While DHS are generally considered of high quality, with strong systems for training, supervision and statistical analysis, the implementation in the field may be less than ideal. Confidence intervals can account for sampling variation. However, overall survey quality is a more fundamental issue. Keep this potentially complicating factor in mind when interpreting the values derived from household surveys.

Census

A national census, with the addition of a limited number of questions, can produce estimates of maternal mortality. This approach eliminates sampling errors because the entire population is surveyed, but the non-sampling errors remain (see below). This approach allows a more detailed analysis of the results, including time trends, geographic subdivisions and social strata. The training of enumerators is crucial because census activities collect information on a range of topics that are unrelated to maternal deaths. Information on household deaths collected in a census should identify:

• all deaths in the household within a specified period

• the age and sex of each deceased person

• the timing of adult female deaths relative to pregnancy, childbirth and the postpartum period.

(14)

Health In

forma

tion S

ys

te

ms K

no

wledg

e Hub

population-based estimates of maternal mortality. Health service data may, however, provide useful information on trends over time and, in particular, on geographic regions and the relative importance of various diseases and causes of death. At the facility level, identifying maternal deaths is often the first step in conducting a detailed audit or a review of the causes and circumstances surrounding the deaths. Such audits often identify weaknesses in the health care system, which allows the development of strategies to resolve them.

In principle, all maternal deaths that occur in health facilities should be reported in the routine health management information system (HMIS). In practice, however, not all deaths occurring in facilities are of maternal mortality can be generated by conducting a

followup study among the households reporting deaths; verbal autopsy is used to discover the cause of death. Depending on the numbers involved and the resources available, the followup study is either conducted on all deaths identified during the census or on a random sample. This approach was used in the 2007 census in Mozambique, and generated estimates of maternal mortality for all provinces and for local areas (Figure 2) (Instituto Nacional de Estatistica 2009). One of the major advantages of using census data is that estimates are generated at the subnational level, which enables the identification of regions or provinces with particularly high maternal mortality. However, the logistic and analytical challenges of conducting such followup surveys should not be underestimated.

Health facility reporting

In most developing countries, only a proportion of all births take place in health care facilities. Unless nearly all women deliver in health care institutions, facility-based data (or data derived from systems for management of routine health information) are rarely sufficient to make

0 100 200 300 400 500 600 700 800 900 Nationa l Maput o C. Maput o P. Gaza Inhambane Sofala Manic a Tete Zambe zia Namphua C. Delg ado Niassa Ma te rnal mort ality r atio

Figure 2 Maternal mortality ratios of Mozambique provinces, generated from the 2007 census results

(15)

Health In

forma

tion S

yst

ems K

no

wledg

e Hub

Reproductive age mortality studies

Reproductive age mortality studies (RAMOS) involve systematic efforts to combine data on maternal deaths from multiple sources. The starting point is usually to list all deaths of women of reproductive age. These deaths are then investigated using verbal autopsy and medical records (when available) to identify maternal deaths (RAMOS, no date; WHO 1987). The sources of information on deaths in women of reproductive age vary. Where feasible, the initial list is drawn from civil or sample registration records. However, when registration of deaths is incomplete, other methods have to be used to identify deaths among women of reproductive age. These methods include reviewing hospital records, captured in the HMIS (Figure 3). In most countries, the HMIS covers only public sector facilities and, even within the public sector, not all facilities regularly report to the HMIS. Also, at the facility level, maternal deaths may be missed or misclassified for the reasons cited above. Frequently, the HMIS only records deaths that occur in obstetric wards; deaths in specialist units or emergency wards are most likely to be excluded. Tools are available to facilitate the identification of unreported deaths within health facilities, in particular the Rapid Ascertainment Process for Institutional Deaths (RAPID) developed by IMMPACT at the University of Aberdeen (University of Aberdeen 2007).

Capturing maternal deaths

Figure 3 Identifying all maternal deaths

Maternal deaths reported in routine HMIS Maternal deaths in all wards Maternal deaths in public and private facilities

Maternal deaths in all facilities and community deaths

discussions with traditional birth attendants, examining funeral records, interviewing religious and community leaders, and even visiting schools (Smith and Burnham 2005).

After the complete listing of deaths in women of reproductive age has been compiled, verbal autopsy methods and reviews of health records (where available) can be used to determine whether the death was due to maternal causes. The use of RAMOS is not restricted to developing countries; they have also been used in high-income countries to identify maternal deaths that may have been missed through the routine registration system.

By their very nature, these mortality studies are complex and in developing countries are mostly carried out in small areas (often at the district level). If properly conducted, RAMOS can generate a reliable estimate of maternal mortality. However, they are complicated, time-consuming and expensive, particularly on a large scale. Also, RAMOS do not generate complete data on live births to calculate the maternal mortality ratio, especially in settings in which most women deliver at home—a major weakness. Therefore, the PMDF from such studies is often applied to an independent external source of data on live births (eg by calculating expected births using natality data extrapolated from the most recent census).

A specific example of RAMOS is the United Kingdom Confidential Enquiry into Maternal Deaths, which is conducted at regular intervals to identify avoidable factors underlying maternal deaths and to learn lessons to prevent such deaths (Drife 2006). Egypt, Jamaica and South Africa have also conducted confidential enquiries (NCCEMD 2000). The United Kingdom enquiry usually finds that a substantial proportion of maternal deaths were incorrectly classified to other causes (Lewis 2004). Similar studies in other countries estimated adjustment factors for misclassification of maternal mortality in data from death registrations of 0.9–3.2, with a median value of 1.5 (NCCEMD 1998, Schuitemaker et al 1998, WHO et al 2010).

Comparison of data sources

Four key conclusions emerge from this brief review of definitions and methods (Table 3). First, different data sources and methods of data collection tend to yield

(16)

Health In

forma

tion S

ys

te

ms K

no

wledg

e Hub

different measures of maternal mortality with their own strengths and weaknesses. Thus, one method does not suit every situation. Different data collection strategies generate different indicators—maternal mortality or pregnancy-related mortality. Civil registration systems in which deaths are medically certified produce data on maternal mortality, whereas sibling-based methods and censuses measure pregnancy-related mortality. However, the results are very often called ‘maternal’ deaths, regardless of the definition of the recorded event. Second, every data source produces estimates that have a degree of uncertainty. In sample surveys, confidence intervals quantify the uncertainty associated with sampling methods, but not the uncertainty around the accuracy of respondents’ recall. When a census is used to generate the data, precise values for uncertainty cannot be calculated. However, the degree of adjustment required for the data provides an indication of

uncertainty. Even complete civil registration systems with medical certification of the cause of death have a degree of uncertainty because some deaths may be incorrectly classified as nonmaternal.

Third, the time reference for maternal mortality figures differs with the data source and collection method. As already noted, for technical reasons, direct sisterhood methods generate values for a period some 5–7 years before the survey. The ongoing systems of civil and sample registration can generate the most recent data. In practice, delays occur in compiling and disseminating the results; even a sophisticated civil registration system usually issues figures with a delay of one or two years. Many countries are introducing automated systems for data coding that are designed to speed up the process (and also reduce coding errors). Household surveys that direct methods can generate recent estimates, but publication is often delayed by the need to clean and analyse the data.

Finally, the various data sources and collection methods offer different opportunities to gather other important data alongside that required to measure maternal mortality. These differences have important implications for the efficiency and cost–benefit of the measurement approaches.

Hierarchy of data sources

When multiple data sources are available, and assuming that collection methods are correctly implemented, a

hierarchy exists to assess the resulting data on maternal mortality.

At the top of the hierarchy are methods that involve a full count of events and generate unbiased population-based values. These methods are civil registration with medical certification of cause of death (assuming high rates of completeness), followed by sample registration with verbal autopsy (assuming that the sample sites are representative of the total population) and that verbal autopsy methods are aligned with international standards.

Next is longitudinal surveillance in specific sites. This method includes a full count of events and verbal autopsies to establish cause of death, but is limited to the population under surveillance. The sites are not randomly selected and are not nationally, or even locally, representative. Studies of mortality in women of reproductive age try to establish a full count of events by reconciling data from various sources (registration, health facilities, cemeteries, religious institutions, etc), but are rarely conducted at national level.

Household surveys are valuable for generating estimates on broad scales (eg the national level). However, sample sizes make them inefficient instruments for generating subnational data and they can be problematic for monitoring trends because of wide confidence intervals. A census can generate data at the subnational level and identify differences between population groups. However, for technical reasons, the estimates may be biased and incomplete. Moreover, because censuses are only conducted about every 10 years, they are not useful ongoing monitoring. Therefore, use a census as an adjunct to other data sources, not as a stand-alone source.

Data from health facilities do not produce population-based estimates of maternal mortality unless all women deliver in health facilities, all maternal deaths are correctly identified and all facilities report maternal deaths. Nonetheless, this source can be of value if sustained efforts are made to ensure complete reporting by all facilities (public and private), and if complementary mechanisms are used to identify deaths in the

community. Even where incomplete, facility data can be used as a starting point for audits and case reviews that evaluate the quality of care, describe the causes and circumstances associated with each death, and identify locally relevant, avoidable factors.

(17)

Health In

forma

tion S

yst

ems K

no

wledg

e Hub

Table 3 Summary of maternal mortality data sources and data collection methods

Method Event measured Precision and uncertainty Reference period

Civil registration with medical certification of cause of death

Maternal mortality Total count; maternal deaths may be misclassified; evidence indicates ~ 50 per cent underreporting

Previous year Sample registration with

verbal autopsy Maternal mortality Representative count; maternal deaths may be misclassified; extent of misclassification in verbal autopsy not known so uncertainty cannot be calculated

Previous year

Household survey with

direct estimation Pregnancy-related mortality Depends on sample size; generally wide Usually 1–2 years before survey, depending on recall period Household survey with

direct sisterhood method Pregnancy-related mortality Uncertainty arises from sampling errors (20–30 per cent) and from misreporting of pregnancy status of deceased women

Around 5–7 years before survey

Household survey with indirect sisterhood method

Pregnancy-related

mortality Uncertainty arises from sampling errors (~ 30 per cent) and from misreporting of age and pregnancy status of deceased siblings

Around 10–12 years before survey

Census Pregnancy-related

mortality Total count but estimates require adjustment using demographic techniques; also misreporting of age and pregnancy status of deceased women

Reference period for maternal deaths usually 1–2 years before census, depending on recall period. Demographic adjustments generate values at the midpoint between two censuses

Health facility reporting Maternal mortality HMIS generally covers only public health facilities; captures maternal deaths occurring in obstetric wards; maternal deaths in emergency and specialist wards often missed

Usually recent reference period

RAMOS Combination of

maternal and pregnancy-related mortality

Depends on the ability of investigators to identify all maternal/pregnancy-related deaths, and on the quality of the medical records and verbal autopsies

Usually covers multiple years

(18)

Health In

forma

tion S

ys

te

ms K

no

wledg

e Hub

Monitoring rare events

Many of the problems associated with monitoring maternal mortality arise from the fact that maternal deaths are relatively rare—only about 5 per cent of child deaths. The small numbers often cause unstable national and, especially, subnational figures, particularly when mortality levels are low and household surveys are used as the data source. In countries with small absolute numbers of maternal deaths, changes of one or two deaths in the numerator disproportionately affect the maternal mortality ratio. For example, the maternal mortality ratio in a country with about 4000 live births annually and 4–6 maternal deaths in a given year will fluctuate between 100 and 150. Therefore, WHO advises countries to use a 3–5 year moving average to illustrate trends, rather than annual values.

The historical data on maternal mortality for England and Wales during the period 1850 to 1980 vividly illustrates this point (Figure 4). In the chart on the left, the yearly values fluctuate markedly, despite a high maternal mortality ratio during that time (over 600 per 100 000 live births). If a moving average is applied to these data,

0 10 20 30 40 50 60 70 1960 1970 1950 1940 1930 1920 1910 1900 1890 1880 1870 1860 1850 0 10 20 30 40 50 60 70 1960 1970 1950 1940 1930 1920 1910 1900 1890 1880 1870 1860 1850

Source: Registrar General for England and Wales, Decennial Supplements (1850-1970)

Figure 4 Using moving averages to smooth stochastic variations in annual data

the long-term trend becomes apparent (the chart on the right).

When the absolute numbers of maternal deaths are small (as they commonly are), there are limitations on the kinds of disaggregations possible; for example, by age, parity or region. The use of 100 000 births as the multiplier in the calculation of the ratio often causes disquiet in countries with few births such as small island countries in the Pacific and the Caribbean. In such settings, simply tracking the numerators and carefully investigating each maternal death may be more appropriate than monitoring the ratio, which is subject to random variations associated with small numbers. The expected number of maternal deaths varies in settings with differing population sizes, expected numbers of live births and levels of maternal mortality (Table 4). For example, with a population of 1 million people, about 40 000 expected live births annually and a very high maternal mortality ratio of 500 per 100 000 live births, only about 200 maternal deaths would be expected each year.

Maternal mortality in England and Wales, 1850-1980

Annual dea th r at e per 1 0 00 t ot al births

(19)

Health In

forma

tion S

yst

ems K

no

wledg

e Hub

Table 4 Expected number of maternal deaths with differing population sizes and maternal mortality

Maternal mortality (per 100 000

live births) populationTotal live birthsExpected

Expected maternal deaths 100 100 000 4000 4 500 20 1000 40 100 1 000 000 40 000 40 500 200 1000 400 100 20 000 000 800 000 800 500 4000 1000 8000

The problem of small numbers is illustrated by experiences with household surveys, notably DHS (Table 5). These are relatively large surveys with interviews conducted in about 20 000 households. The small absolute numbers of maternal deaths do not allow subnational analyses or the identification of high-risk groups. Even the Indian 1999 household survey, with a sample size of 100 000 households, identified only 65 deaths throughout the whole country; in some states, no maternal deaths were identified (International Institute for Population Sciences 1999).

Table 5 Number of maternal deaths from

demographic and health surveys in selected countries

Year of survey Country maternal deathsNumber of

2005 Ethiopia 197 2004 Lesotho 92 2007 Liberia 127 2008 Sierra Leone 97 2006 Uganda 151 2007 Zambia 105

(20)

Health In

forma

tion S

ys

te

ms K

no

wledg

e Hub

Tracking trends

The uncertainty inherent in measuring maternal mortality makes definitive statements about trends difficult. Household surveys generate estimates with wide margins of uncertainty (see Figure 1), which often overlap. Hence, stating unequivocally that the observed estimates reflect real changes over the 10-year period is impossible. In Malawi, for example, while the confidence intervals of the 1992 and 2000 surveys do not overlap, the estimates for 2000 and 2004 are statistically indistinguishable.

When wide confidence intervals make it impossible to determine whether the observed trends are real or simply artefacts of the data, other data are needed to support the interpretation of the observed trends over time. Even when confidence intervals do not overlap and the point estimates presumably reflect a real trend, other data should be reviewed to reinforce the conclusions. More than one kind of indicator is usually needed to explain trends. For example, Bangladesh appears to have achieved a steady decline in maternal mortality, from 574 per 100 000 live births in 1981 to 322 per 100 000 live births in 2001 and 194 per 100 000 in 2010. This decline has occurred despite the fact that only a minority (26 per cent) of pregnant women deliver with the assistance of a trained health care provider—generally considered the most important interventions to reduce maternal mortality. A detailed analysis of the trends in maternal mortality from the longitudinal demographic surveillance site, Matlab, and its control area identified two key explanatory factors for the striking decline in maternal

mortality. Fertility reduction resulted in declines in higher risk, high-parity births. At the same time, there was increased use of medical care, both for normal deliveries and among women experiencing complications (Government of Bangladesh 2011).

If interpreting trend data from the same data sources in different years is challenging, interpreting trends in data derived from different sources is even more difficult. Table 6 and Figure 5 show Indian maternal mortality data from two sources. The National Family Health Surveys (NFHS) in 1991–92 and 1998–99 used direct estimation of mortality. The Indian Sample Registration System (SRS) 1999–2001 and 2001–03 monitors vital events on an ongoing basis, coupled with verbal autopsies for deaths among women of reproductive age. The confidence intervals for the SRS estimates of maternal mortality are much narrower than those of the estimates derived from the NFHS (although the SRS uncertainty interval does not take into account uncertainties inherent in determining the cause of death by verbal autopsies). Nonetheless, until 2001, the SRS estimates fell within the confidence interval of the first NFHS. The SRS functions on a vast scale, with sample sizes of more than 4 million women. The survey identifies about 1500 maternal deaths in each round (Office of the Registrar General, India 2006). Because the SRS has narrower confidence intervals and the perceived quality of the data is better than that of the NFHS, official Indian estimates of maternal mortality are now drawn solely from the SRS (Government of India 2007; Office of Registrar General, India 2009). However,

Table 6 Estimates of maternal mortality in India from two data sources

Reference year Sample size Point estimate (confidence interval)Uncertainty bounds Data source Method

1989–91 90 000

ever-married women 424 324–524 NFHS I Household survey; recent deaths in household

1996–97 91 000

ever-married women 540 428–653 NFHS II Household survey; recent deaths in household

1997–98 4 562 000

women 398 378–418 SRS Sample registration with verbal autopsy

1999–2001 4 839 000

2001–03 5 039 000

2004–06 5 348 441

(21)

Health In

forma

tion S

yst

ems K

no

wledg

e Hub

Source: Government of India (2007), Office of Registrar General, India (2009)

Figure 5 Estimates of maternal mortality in India from different sources

0 100 200 300 400 500 600 700

Point estimate Lower bound Upper bound

large variations in maternal mortality occur at the subnational level, both in terms of point estimates and the size of the confidence intervals.

In Thailand, data sources for estimates of maternal mortality are the routine civil registration system, safe motherhood monitoring, studies of mortality in women of reproductive age (RAMOS) and a special study that verifies maternal deaths by comparing civil registration and facility data (Figure 6). Routine systems such as registration and safe motherhood monitoring appear to systematically underestimate maternal mortality, compared with estimates based on occasional special studies such as RAMOS. This result is understandable, given that routine systems are essentially passive whereas special studies, by definition, use active case-finding to identify all maternal deaths. However, special studies require considerable effort and resources and, therefore, are not used routinely for monitoring, although they are valuable for evaluation. Routine systems are less resource-demanding, but appear to miss a significant proportion of maternal deaths. The solution is to combine routine systems with occasional special studies or surveillance to gauge the extent to which the routine system underreports maternal deaths.

0 10 20 30 40 50 60 70 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

BPS BHP RAMOS TDRI BHP-BPS Lancet 2010 WHO 2010

BHP = Bureau of Health Promotion, Department of Health; BPS = Bureau of Policy and Strategy; RAMOS = reproductive age mortality studies; TDRI = Thailand Development and Research Institute, 2007

Figure 6 Maternal mortality data for Thailand from various sources

Maternal mortality, India, 1989-91 - 2004-06

Maternal mortality, Thailand, 1990-2008

Number of ma ternal de aths Ma ternal mort ality r atio

(22)

Health In

forma

tion S

ys

te

ms K

no

wledg

e Hub

Above, we compared maternal mortality estimates from different sources in both India and Thailand. However, different estimates of maternal mortality can also be derived from a single source. Household surveys can apply multiple techniques in the same data collection exercise; for example, direct questions to the household followed up by verbal autopsy, and sibling survival methods. This procedure was used in the 2007 Ghana Maternal Health Survey (Ghana Statistical Service et al 2009), which was carried out in two phases. Phase I consisted of a very short household questionnaire administered in about 240 000 households. The questionnaire recorded the number of deaths in the household in the past five years and, for female deaths, the age at death. Three additional questions were asked if the death was that of a woman aged from 12 to 49 years: (a) whether she was pregnant at the time of death; (b) whether she died during childbirth; and (c) whether she died within two months of a pregnancy or childbirth.

Households that reported one or more deaths of women of reproductive age (12–49 years) were then revisited in Phase II to administer a verbal autopsy questionnaire. This questionnaire recorded signs and symptoms observed before the death and an open-ended narrative of the circumstances surrounding the death. A total of 4203 verbal autopsy questionnaires were completed. Phase II of the survey also included a full sibling history similar to a standard DHS. The Ghana survey thus generated three distinct estimates of pregnancy-related mortality (Table 7, Figure 7). For technical reasons, confidence intervals were not calculated in the Phase I survey, and the true maternal mortality ratio could not be calculated from the verbal autopsy information.

Table 7 Maternal mortality in Ghana

Definition Value Range PMDF Reference date Method

Pregnancy-related mortality 469 366–633 14.1 0–4 years before survey Sibling history Pregnancy-related mortality 378 249–505 15.9 5–9 years before survey Sibling history

Pregnancy-related mortality 580 Na 14.0 5 years before survey Reported deaths and verbal autopsy PMDF = proportional maternal among deaths of females

0 100 200 300 400 500 600 700

Upper bound Lower bound Point estimate 7 years prior to survey 5 years prior to survey

Figure 7 Pregnancy-related/maternal mortality ratio in Ghana

These examples illustrate the challenge of interpreting values for maternal mortality that are derived from various sources and by different methods of data collection. Different data sources generate data points that are not comparable. Scientific methods and learning tools are needed to help countries grapple with these multiple estimates and to support decision making. This challenge provides the underlying impetus for developing global estimates of maternal mortality, an issue that we discuss next.

Maternal mortality in Ghana, 2007

Pr

egnancy r

ela

ted dea

(23)

Health In

forma

tion S

yst

ems K

no

wledg

e Hub

Global estimates of maternal mortality

The rationale for global estimates

A group of United Nations (UN) agencies—WHO, UNICEF and the United Nations Population Fund (UNFPA)— and the World Bank have been producing global and country estimates of maternal mortality since 1996 (WHO and UNICEF 1996, AbouZahr et al 2001, AbouZahr and Wardlaw 2004, WHO 2005). These efforts were driven by the need to monitor global trends, given the variety of definitions, data sources and methods of data collection being used to measure maternal mortality at country level. The most recent UN estimates, issued in 2010 for the year 2008, include point estimates and, for the first time, country-by-country trends from 1990 to 2008 (WHO et al 2010). In 2010, the Institute for Health Metrics and Evaluation (IHME) at the University of Washington in Seattle produced global estimates of maternal mortality levels and trends between 1980 and 2008 (Hogan et al 2010). The IHME issued an updated set of estimates for the period 1990–2010 in 2011 (Lozano et al 2011). These exercises were driven by the need to show global and regional trends, hence the importance of comparability. Such estimates are not primarily intended to serve as country values.

Developing global estimates requires addressing two related challenges: how to predict maternal mortality ratios for countries and periods for which empirical data are unavailable; and how to combine, in a common format and using similar sets of assumptions, data for multiple periods and from diverse sources to generate a set of figures comparable across countries and over time. This upsurge in the availability of estimates for all countries that cover multiple years presents users— especially those working at country level—with a dilemma. What are the differences between these different estimates? Is one superior to the others? How do the estimates compare with reported, empirically generated country data? Which values should country decision-makers use to guide policy development and program implementation?

Global estimates compared

The methods used to develop the three sets of estimates were similar in outline although important details differ. While we do not provide a detailed comparison here, essentially, the key steps below were followed to develop both sets of estimates:

• All available data were complied, with a focus on

extracting the PMDF from each data source, rather than the maternal mortality ratio.

• The observed PMDF values were adjusted to account for misreporting of maternal deaths and survival bias (for data from DHS or similar surveys). The precise adjustments differed, but the results generally adjusted the PMDF upwards for data from civil registration systems and downwards for data from sources that used sibling survival methods. • A statistical model of the relationship between the

adjusted PMDF and a number of covariates was developed. The covariates of the UN model were Gross Domestic Product per capita, fertility and skilled birth attendant. The covariates of the IHME model were Gross Domestic Product per capita, fertility, neonatal mortality, female education and HIV prevalence.

• The model coefficients were applied to predict PMDF values for countries and periods with no data. • The predicted PMDF values were applied to an

envelope of total deaths in women aged 15–49 years to generate the number of maternal deaths. The two sets of estimates used different envelopes of total deaths in women of reproductive age that were derived from their own life tables.

The UN agency estimates extend only up to 2008. The IHME has predicted estimates for 2010 and 2011. In terms of the global numbers for 2008, the differences between the IHME 2010 and UN estimates are small, about 4 per cent in absolute numbers. The IHME estimated about 342 900 maternal deaths, compared with the UN estimates of 358 000. This difference is not statistically significant. The IHME estimates for 2011 are lower than those it issued in 2010, amounting to some 273 500 maternal deaths. The 2011 estimates were derived using a different (and more complex) modelling strategy and updated country input data so the 2011

(24)

Health In

forma

tion S

ys

te

ms K

no

wledg

e Hub

results are not strictly comparable with the 2010 IHME estimates.

The UN estimates have much wider uncertainty ranges than those produced by IHM