Jeremy Rudd (Federal Reserve Board)
Introduction
This paper summarizes some work that Federal Reserve staff have done on quantifying the measurement error in the US Consumer Price Index (or “CPI bias”). (See Lebow and Rudd, 2003, for a comprehensive treatment of many of these topics.)
Ideally, we would like both a point value and “confidence range” for our estimates of CPI bias; both, of course, are relevant for policymakers. Before we begin, however, we need to specify:
•
an “ideal” that we want the index to approach; and•
a methodology for assessing measurement error.Defining the goal of the CPI
We cannot assess the measurement error in the CPI unless we have a view about what the index is supposed to be measuring. For the CPI, the Bureau of Labor Statistics (BLS) state:
“Although the CPI cannot be said to equal a cost-of-living index, the concept of the COLI provides the CPI’s measurement objective and the standard by which we define any bias in the CPI.”
(BLS Handbook of Methods, chapter 17)
The benchmark we used for our own work, therefore, was that of a “conditional COLI,” that is, a COLI defined holding constant nonmarket goods or environmental factors (such as govern-ment-provided goods or crime). In computing CPI bias, we also took as given:
•
the CPI’s scope (out-of-pocket expenditures by consumers); and•
the CPI’s use of “plutocratic” (as opposed to “democratic”) weights.Methodologies for assessing CPI bias
Broadly speaking, we want to compare the CPI to an (unobserved) conditional COLI.
There are other ways to proceed, however (to obtain either a total bias measure or a value for a component of overall bias). For example, we could:
•
estimate the COLI from a fully specified demand system;•
use survey data on living standards (Nordhaus, 1998; Krueger and Siskind, 1998); or•
use other indirect evidence on living standards based on observed expenditure patterns (Hamilton, 2001; Bils and Klenow, 2001).The particular method employed by the Federal Reserve staff involves identifying various ways in which the actual CPI might fall short of a conditional COLI, and quantifying the likely differences.
A similar approach was taken by the Advisory (“Boskin”) Commission, as well as by Shapiro and Wilcox (1996) and earlier Federal Reserve work; naturally, a large amount of judgement informs these sorts of bias estimates. That said, given that we need to take a stand on the accu-racy of the CPI, it is preferable to base our assessment on a systematic review of the available evidence. (This can also highlight just where the existing evidence is weakest.)
Sources of bias in the US CPI
Our study identified four broad sources of bias in the CPI. Three of these are very familiar.
1. Substitution: As a fixed-weight Laspeyres index, the CPI tends to overstate increases in the cost of living by ignoring changes in consumer demand in response to relative price changes.
For the US CPI, substitution bias comes in two forms: upper-level substitution bias (which refers to substitution across the CPI’s roughly 8000 item-area strata); and lower-level substi-tution bias (which refers to substisubsti-tution within these strata).
2. Quality change/new goods: The CPI might not properly measure quality improvements in existing goods, or benefits to consumers from the introduction of new goods.
3. Outlet changes: When new retail outlets are rotated into the CPI, the BLS assume that price differentials between old and new outlets reflect quality differences. Some or all of these dif-ferences could reflect true (quality-adjusted) price change, however.
In addition, we identified a fourth, less-familiar source of bias:
4. Weighting: The CPI’s expenditure weights come from survey data, which could be inaccurate.
Let us now consider in turn how each source of bias can be quantified.
Upper-level substitution bias
The standard way of measuring upper-level substitution bias (ULSB) involves comparing the existing (Laspeyres) CPI to a “superlative” version of the index. There are, however, some com-plications with this approach.
•
Typically, a “chained” superlative is used. However, each link of the “chain” will involve a different reference indifference curve.•
Sampling error in the underlying price data can bias superlative indexes down in practice (c.f. Greenlees, 2001); a correction is therefore needed.•
Bradley (2001) has argued that ULSB does not really exist (little scope for substitution across item–area strata) – instead, finite–sample bias in Laspeyres is corrected by a superlative.Lower-level substitution bias
Expenditure data are not available (even with a lag) to measure within-stratum substitution bias (also known as lower-level substitution bias, or LLSB). The usual way LLSB is measured involves comparing the existing index with an index whose within-stratum prices are aggregated using a geometric mean. In 1999, the US CPI moved to geometric-means aggregation for about three-fifths of the index. Geo-means aggregation implies a (near) unit elasticity of within-stratum sub-stitution; however, if – as seems plausible – actual elasticities of substitution are larger than unity, then some LLSB could remain in the US CPI. (There is not much evidence on this.)
New-outlet bias
Changes in consumer buying patterns suggest that not all of the price differential between old and new-outlets reflects quality. There is very little evidence on the magnitude of this effect.
•
Reinsdorf (1993) found a 0.25 pp per year price differential at incoming outlets for certain foods and gasoline. Some, all, or none of this could reflect true price change.•
More recently, Hausman and Leibtag (2004) find an estimate for food at home of 0.32–0.42 pp per year (but this also includes some correction for substitution bias).•
Lebow et al. (1994) judged that new-outlet bias is probably relevant for about 40 percent of the overall CPI.Weighting bias
The weights in the CPI are derived from the BLS’s Consumer Expenditure Survey (CEX).
Surveys can be affected by reporting bias (e.g., underreporting of alcohol or tobacco purchases), recall bias, or lack of knowledge about total household expenditures. If items with weights that are too large tend to display above-average price increases, then the CPI will be biased upward.
JEREMY RUDD
PROCEEDINGS IFC WORKSHOP – SESSION 7
•
We therefore estimate “weighting bias” as the average difference in the growth rates of the published and PCE-weighted CPIs.Quality-change and new-items bias
A COLI must try to capture quality changes and any benefits from the introduction of new goods. The BLS devote considerable effort to quality adjustment; nevertheless, many analysts believe that unmeasured quality improvement is a source of significant upward bias in the CPI.
This is probably the most controversial single issue surrounding the topic of CPI bias, both because quality-change bias estimates tend to be large, and because a large degree of judgement is needed in order to quantify them. There are also some important unresolved issues regarding conventional methods for dealing with quality change (c.f. Hobijn, 2003). Even less can be said about the magnitude of new-goods bias.
Our study reviewed the evidence on quality-adjustment bias for each category of expenditure. In some cases, useable studies (e.g. hedonic analyses) exist to inform these estimates. In most cases, however, the evidence is extremely thin. We categorized the degree of evidence as follows.
1. Estimates based on at least a moderate degree of evidence: These prices account for 7 percent of the CPI and 5 percent of our total quality-change/new-goods bias estimate.
2. Estimates based on a small or inadequate degree of evidence: Accounts for 39 percent of the CPI and 68 percent of our bias estimate.
3. Estimates almost totally subjective: These represent 54 percent of the CPI and 27 percent of our bias estimate.
In addition, several other interesting points arise from a consideration of this source of bias.
•
First, it is important to avoid double-counting (e.g. a log-linear hedonic estimate should be compared to a CPI series with geo-means aggregation).•
Second, most relevant studies (e.g. hedonic estimates) cover short periods of time that might not be representative.•
Finally, it is important to choose studies whose baseline price-change estimates are computed in a manner that is comparable to the existing CPI (many are not).Aggregating each source of bias
We can combine each individual source of bias to obtain an aggregate bias estimate. When doing this, it is again important to avoid “double counting.”
•
For example, for a “prospective” estimate of bias, the estimates of quality-change bias for each good should be aggregated using very recent expenditure weights.•
Similarly, if “weighting bias” is an issue, the individual quality-change bias estimates should be aggregated using the preferred weights.When we combine our individual estimates, we obtained an overall (prospective) bias figure of 0.9 percentage point per year. Most of this reflects quality-change/new-goods bias (0.4 percentage point per year) and upper-level substitution bias (0.3 percentage point per year).
Construction of a confidence interval
Following Shapiro and Wilcox (1996), we can also compute a confidence interval for our bias estimate. This requires us to specify distributions for each bias component; if there is a range of estimates for a particular component, then this can be used to inform its assumed distribution.
We might also have priors about the distribution’s “shape.” (In most cases, however, the assumed distributions will be purely subjective.) The distributions can then be combined – with assumptions about how the sources of bias are correlated with each other – to yield an aggregate distribution.
Under our assumptions, we obtained a 90 percent confidence interval around our overall bias estimate that ranged from 0.3 to 1.4 percentage points per year.
JEREMY RUDD
A note about PCE bias
The PCE chain price index is an important alternative US price statistic. What can we say about this measure’s bias?
•
First, since individual CPIs are used in the PCE index, quality-change/new-good bias is still present (its magnitude can be different inasmuch as PCE and CPI weights differ).•
Weighting bias is absent, however (by construction), as is upper-level substitution bias (the PCE index uses a superlative formula).•
Finally, other sources of bias (outlets, lower-level substitution) are also present.•
In addition, we also need to take a stand on the broader scope of the PCE price measure. One assumption we might make is that the PCE’s scope is appropriate for that index, and that no quality-change bias is present in nonmedical, nonmarket PCE prices.Concluding thoughts
This methodology provides a useful benchmark estimate of CPI bias (in particular, it provides a helpful guide to areas where future work is needed). However, this approach is uncomfortably subjective. (Under the circumstances, that might be the best we can do.) Related to this is a con-cern that quantification of a bias estimate (and its confidence interval) might imply a degree of certainty that we simply do not have. Finally, this method tells us little or nothing about the nature of time variation in overall CPI bias, which might be of particular interest to a monetary policymaker.
References
Bils, Mark and Peter Klenow. 2001. “Quantifying Quality Growth.” American Economic Review 91, 1006–1030.
Bradley, Ralph. 2001. “Finite Sample Effects in the Estimation of Substitution Bias in the Consumer Price Index.” Journal of Official Statistics 17, 369–390.
Greenless, John. 2001. “Random Errors and Superlative Indexes.” Bureau of Labor Statistics Working Paper Number 343.
Hamilton, Bruce. 2001. “Using Engel’s Law to Estimate CPI Bias.” American Economic Review 91, 619–630.
Hausman, Jerry and Ephraim Leibtag. 2004. “CPI Bias from Supercenters: Des the BLS Know That Wal-Mart Exists?” NBER Working Paper Number 10712.
Hobijn, Bart. 2003. “On Both Sides of the Quality Bias in Price Indexes.” Mimeo, Federal Reserve Bank of New York.
Krueger, Alan and Aaron Siskind. 1998. “Using Survey Data to Assess Bias in the Consumer Price Index.”
Monthly Labor Review (April), 24–33.
Lebow, David and Jeremy Rudd. 2003. “Measurement Error in the Consumer Price Index: Where Do We Stand?” Journal of Economic Literature 41, 159–201.
Lebow, David; John Roberts and David Stockton. 1994. “Monetary Policy and ‘The Price Level’.” Mimeo, Federal Reserve Board of Governors.
Nordhaus, William. 1998. “Quality Change in Price Indexes.” Journal of Economic Perspectives 12, 59–68.
Reinsdorf, Marshall. 1993. “The Effect of Outlet Price Differentials on the U.S. Consumer Price Index.” In Price Measurements and Their Uses, M. Foss, M. Manser, and A. Young, editors. Chicago: University of Chicago Press.
Shapiro, Matthew and David Wilcox. 1996. “Mismeasurement in the Consumer Price Index: An Evaluation.” In NBER Macroeconomics Annual, B. Bernanke and J. Rotemberg, editors.