In our home price example, we found two regression equations, one for the relationship between price and house
size, and one for the relationship between price and distance. What will the equation for the threeway relationship between price, the dependent variable, and the two independent variables, house size and distance, look like?
The regression equation in our housing example will have the form below: house size and distance each have their own coefficients, and they are summed together along with the constant coefficient a.
In general, the linear equation for a regression model with k different variables has the form below. Since the coefficients we obtain from the data are just estimates, we must distinguish between the idealized equation that represents the "true" relationship and the regression line that estimates that relationship. To express that even the
"true" equation does not fit perfectly, we include an error term in the idealized equation.
Running the regression gives us coefficients for house size and distance: 252 and 55,006, respectively. We can use this multiple regression equation to predict the price of other houses not in our data set. To predict a house's price, we need to know only its size and its distance to downtown.
Silverhaven Real Estate Data Silverhaven Real Estate Regressions
Suppose "Windsor" is a modest mansion of 3,500 square feet, located in the outer suburbs of Silverhaven,
approximately 11 miles from downtown. Based on our regression equation, how much would we expect Windsor to sell for?
Silverhaven Real Estate Data Silverhaven Real Estate Regressions
We simply enter Windsor's square footage and distance to downtown into the equation, and calculate an expected selling price of $699,938.
Let's take a closer look at the coefficients in the housing example, focusing on the distance coefficient: 55,006. This coefficient is substantially different from the coefficient in the original simple regression: 39,505. Why is it so different?
The coefficients in the simple regression and the coefficients in the multiple regression have very different meanings. In the simple regression equation of price versus distance, we interpret the coefficient, 39,505, in the following way: for every additional mile farther from downtown, we expect house price to decrease by an average of
$39,505.
We describe this average decrease of $39,505 as a gross effect it is an average computed over the range of variation of all other factors that influence price.
In the multiple regression of price versus size and distance, the value of the distance coefficient, 55,006, is different, because it has a different meaning. Here, the coefficient tells us that, for every additional mile, we should expect the price to decrease by $55,006, provided the size of the house stays the same.
In other words, among houses that are similarly sized, we expect prices to decrease by $55,006 per mile of distance to downtown. We refer to this decrease as the net effect of distance on price. Alternatively, we refer to it as "the effect of distance on price controlling for house size".
Two houses are similar in size, but located in different neighborhoods: "Shangri La" is five miles farther from downtown than "Xanadu." If Xanadu is valued at $450,000, how much would we expect Shangri La to cost?
Silverhaven Real Estate Data Silverhaven Real Estate Regressions
Since the two houses are the same size, we use the net effect of distance on price,
$55,006/mile, to predict the expected difference in their selling prices. Shangri La is 5 additional miles form downtown, so its price should be $55,006/mile * 5 miles = $275,030 less than Xanadu's, or $450,000 $275,030 =
$174,970.
"Valhalla" is another house located 5 miles farther from downtown than Xanadu. We have no information about the relative sizes of the two homes. If Xanadu's selling price is $450,000, what would we expect Valhalla's selling price to be?
Silverhaven Real Estate Data Silverhaven Real Estate Regressions
Since we cannot assume that the sizes of the two houses are equal, we should not control for size. Thus we use the gross effect of distance on price,
$39,505/mile, to predict the expected difference in the two homes' selling prices. Valhalla is 5 additional miles from downtown, so its price should be $39,505/mile * 5 miles = $197,525 less than Xanadu's, or $450,000 $197,525 =
$252,475.
Let's try to build our intuition about the difference in the distance coefficients in the simple and multiple
regressions. The coefficients are different because they have different meanings. But what exactly accounts for the drop from 39,505 to 55,006?
In the multiple regression, by essentially considering only houses that are of equal size, we separate out the effect of house size on price. We are left with a distance coefficient that is net relative to house size.
In the simple regression, the gross effect of distance, $39,505/mile, represents an average over the range of house sizes. As such, it also captures some of the effect that house size has on price. Let's take a closer look at the distance and house size data.
Calculating the correlation coefficient between sizes of homes and their distances from downtown Silverhaven, we see that there is a slight positive relationship, with a correlation coefficient of 31%. In other words, as we move farther from downtown, houses tend to be larger.
We have seen that two things happen as we move farther from downtown — housing prices drop because the commute is longer, and house size increases. The fact that house size increases with distance complicates the pricing story, because larger houses tend to be more expensive.
Longer distances from downtown translate into two different effects on price. One effect of distance on price is negative: as distance increases, commute times increase and prices drop.
A second effect of distance on price is positive: as distance increases, house size increases, and larger houses corresponds to higher prices.
Running a multiple regression with both size and distance as independent variables helps tease out these two separate effects. When we control for house size, we see the net effect of distance on price: prices drop by $55,006 per additional mile.
When we don't control for house size, the effect of distance alone on price is confounded by the fact that house size tends to rise as distance increases. The "real" effect of distance on price is diminished by the relationship between price and house size.
When we look at the net relationship between distance and price, we consider only similarly sized houses. Now we assume that as distance from downtown grows, house size stays the same. If house size didn't increase as we moved farther out, prices would drop more sharply: by $55,006 rather than $39,505 per additional mile.
Let's analyze the house size coefficient in a similar fashion. In the multivariable regression model of home prices, the house size coefficient is net relative to distance. The coefficient of 252 tells us to expect prices of homes equally distant from downtown to increase by an average of $252 for each additional square foot of size.
The gross effect of house size on price is $167 per square foot, considerably less than $252, the net effect. When we do not control for distance, house size "picks up" some of the negative effect of distance: the fact that larger houses are typically located farther from the city offsets some of the effect of increased house size.
We should always be careful to interpret regression coefficients properly. A coefficient is "net" with respect to all variables included in the regression, but "gross" with respect to all omitted variables. An included variable may be picking up the effects on price of a variable that is not included in the model — school district, for example.
Finally, we should note that these coefficients are based on sample data, and as such are only estimates of the coefficients of the true relationship. For each independent variable, we must inspect its pvalue in the regression output to make sure that its relationship with the dependent variable is significant.
Since the pvalue is less than 0.05 for both house size and distance to downtown, we can be 95% confident that the true coefficients of the two independent variables are not zero. In other words, we are confident that there are linear relationships between each independent variable and house price.
There are four steps we should always follow when interpreting the coefficients of an independent variable in a multiple regression:
Summary
We use multiple regression to understand the structure of relationships between multiple variables and a
dependent variable, and to forecast values of the dependent variable. A coefficient for an independent variable in a regression equation characterizes the net relationship between the independent variable and the dependent variable: the effect of the independent variable on the dependent variable when we control for the other independent variables included in the regression.