27 Omitted Variable Bias

If you think that I have spent too much time already trying to nail into your heads that correlation is not causation, then you might not like the chapter to come – which is all about that topic.

In the previous chapter, we saw that when we add additional explanatory variables to a model, the coefficient on the existing variables can change. For example, in the model with just rainfall the coefficient was 0.057, and once the count of days above 30 °C was added it fell to 0.005.

If we want to interpret the coefficient in a model as the effect of an additional mm of rain on yields, then both of these coefficients cannot be right. One – or perhaps both – must be wrong or, more precisely, biased. In this case, it is biased because we left out a variable that is correlated with both rainfall and yield – this is called omitted variable bias.

27.1 Defining omitted variable bias

Let’s be a bit more precise about what we mean by omitted variable bias. Suppose we want to estimate $\beta_1$ in the following regression:

\[ y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \varepsilon \]

but we fit a model with $x_1$ only:

\[ y = \tilde{\beta}_0 + \tilde{\beta}_1 x_1 + v \]

Note that I write $\tilde{\beta}_1$ in the second regression because it will not necessarily be the same as $\beta_1$ from the first regression. Intuitively, if $x_2$ is related to both $x_1$ and $y$, then leaving it out of the model will cause the slope on $x_1$ to absorb some of what $x_2$ was doing. In the previous chapter, we saw that omitting heat days from the model caused the coefficient on rainfall to be much larger than it should have been, because heat days were correlated with rainfall.

The second model has no $x_2$ in it, but that does not make $x_2$ go away. Everything we leave out ends up in the error term, so $v = \beta_2 x_2 + \varepsilon$. That is the whole problem: $x_2$ is correlated with $x_1$, so the error term is correlated with the variable we are trying to interpret.

To see what the slope picks up, write the relationship between the two variables as a regression of its own:

\[ x_2 = \tilde{\delta}_0 + \tilde{\delta}_1 x_1 + u \]

where $\tilde{\delta}_1$ is the slope from regressing the omitted variable on the included one – $x_2$ on $x_1$. Substituting this into the true model and collecting the terms that multiply $x_1$:

\[ y = \underbrace{(\beta_0 + \beta_2 \tilde{\delta}_0)}_{\tilde{\beta}_0} + \underbrace{(\beta_1 + \beta_2 \tilde{\delta}_1)}_{\tilde{\beta}_1} x_1 + \underbrace{(\beta_2 u + \varepsilon)}_{v} \]

which gives us the relationship between $\beta_1$ and $\tilde{\beta}_1$:

\[ \tilde{\beta}_1 = \hat{\beta}_1 + \hat{\beta}_2 \tilde{\delta}_1 \tag{27.1}\]

The second term, $\hat{\beta}_2 \tilde{\delta}_1$, is the bias. It is a product of two things: how much $x_2$ matters to $y$, and how strongly $x_2$ moves with $x_1$.

What this means is that if we omit $x_2$ from the regression then $\tilde{\beta}_1 = \hat{\beta}_1$ only if either:

$\hat{\beta}_2=0$ – which means that $x_2$ has no effect on $y$, or
$\tilde{\delta}_1=0$ – which means that $x_2$ is uncorrelated with $x_1$.

Neither is something we can check, because we do not have $x_2$ – if we did, it would be in the model. This is an argument you make from what you know about the setting, not something the regression output will tell you.

An example

Let’s see how omitted variable bias works in practice. Using the data farm_income.csv which holds data from 400 simulated farms. There are two variables of interest:

farm_income_data <- read_csv("data/farm_income.csv")

head(farm_income_data)
cor(farm_income_data$years_educ, farm_income_data$grade12)

> farm_income_data <- read_csv("data/farm_income.csv")
Rows: 400 Columns: 4
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (1): farm_id
dbl (3): grade12, years_educ, farm_income

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
> 
> head(farm_income_data)
# A tibble: 6 × 4
  farm_id grade12 years_educ farm_income
  <chr>     <dbl>      <dbl>       <dbl>
1 A001       75.5          3        99.6
2 A002       73.8          1       101. 
3 A003       82.8          2       119. 
4 A004       85.2          2       134. 
5 A005       81.3          1       127. 
6 A006       89.9          2       101. 
> cor(farm_income_data$years_educ, farm_income_data$grade12)
[1] 0.3652751

years_educ is years of post-secondary schooling, grade12 is the operator’s grade 12 average in percent, and income is in thousands of dollars. What we want to do is estimate the effect of an additional year of education on farm income. However, this is a notoriously difficult problem in labour economics.

A very simple approach is just to measure the incomes of those who went to school and compare them with those who have not – this is essentially what a univariate regression does, with years of education as the only explanatory variable. The issue is that people who go to post-secondary education (e.g., you!) are quite different from those who don’t. In particular, it is likely that those who go on to post-secondary education are, on average, smarter and harder-working than those who do not. Note, that this is just an average effect – many people who don’t go to University are extraordinarily smart and hard-working (and some who go to University are not). But this means that those who go to University would likely have had higher incomes even if they had not gone to University. So there are two explanations for why University graduates earn more than non-graduates: 1) they are smarter and harder-working, and 2) University education has a causal effect on their earnings. If we just compare the incomes of graduates and non-graduates, we are confounding these two explanations.

To see this in practice, let’s first run a regression of farm income on years of education only.

income_edu_reg <- lm(farm_income ~ years_educ, data = farm_income_data)
summary(income_edu_reg)

> income_edu_reg <- lm(farm_income ~ years_educ, data = farm_income_data)
> summary(income_edu_reg)

Call:
lm(formula = farm_income ~ years_educ, data = farm_income_data)

Residuals:
    Min      1Q  Median      3Q     Max 
-49.264 -11.667   0.591  11.938  46.247 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  88.1417     1.3664   64.51   <2e-16 ***
years_educ    7.8114     0.6142   12.72   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 16.72 on 398 degrees of freedom
Multiple R-squared:  0.289,	Adjusted R-squared:  0.2872 
F-statistic: 161.8 on 1 and 398 DF,  p-value: < 2.2e-16

Farm income on years of education only (adjusted $R^2$ = 0.2872, n = 400)
Term	Estimate	Std. Error	t value	p value
Intercept	88.1417	1.3664	64.51	<0.001
Years of post-secondary education	7.8114	0.6142	12.72	<0.001

The regression tells us that an additional year of education is associated with an increase in farm income of $7,811 per year. I say associated because we cannot say that this is the effect of a year of education – it is correlation and not causation.

Let’s see what happens when we put in a student’s grade 12 average as a second explanatory variable. This is a measure of the student’s ability and work ethic before they went to University. When we include this variable, we are controlling for the fact that smarter and harder-working students are more likely to go to University.

income_edu_grade_reg <- lm(farm_income ~ years_educ + grade12, data = farm_income_data)
coef(income_edu_grade_reg)

> income_edu_grade_reg <- lm(farm_income ~ years_educ + grade12, data = farm_income_data)
> coef(income_edu_grade_reg)
(Intercept)  years_educ     grade12 
 -57.112179    4.380461    1.928458

Farm income on years of education and grade 12 average (adjusted $R^2$ = 0.6493, n = 400)
Term	Estimate	Std. Error	t value	p value
Intercept	-57.1122	7.2202	-7.91	<0.001
Years of post-secondary education	4.3805	0.4628	9.47	<0.001
Grade 12 average (%)	1.9285	0.0950	20.30	<0.001

We see that the slope on years of education has now fallen to $4,380 per year, and the slope on grade 12 average is $1,928 per percentage point. This means that at least some of the apparent effect of education on income was actually due to the fact that students who performed better in high school are more likely to attend University.

We can now completely capture Equation 27.1 by running an auxiliary regression of the omitted variable (grade 12 average) on the included one (years of education).

delta <- lm(grade12 ~ years_educ, data = farm_income_data)
coef(delta)

> delta <- lm(grade12 ~ years_educ, data = farm_income_data)
> coef(delta)
(Intercept)  years_educ 
  75.321289    1.779097

Consistent with the story above, we see that students who go further in school also had better grade 12 averages. So University educated students were better students to begin with, which explains some of the association between University education and farm income.

Returning to the formula for omitted variable bias, we can see that the coefficient in our first regression was equal to the coefficient in our second regression plus the product of the slope on the omitted variable and the slope from the auxiliary regression, which we can call the bias term:

\[ \tilde{\beta}_1 = \underbrace{\hat{\beta}_1}_{\text{Adjusted estimate}} + \underbrace{\hat{\beta}_2 \tilde{\delta}_1}_{\text{Bias}} = 4.38 + 1.93 \times 1.78 = 7.81 \]

Note that $\hat{\beta}_1$ is the estimate adjusted for grade 12 marks, not the true effect itself. It would be the true effect only if grade 12 marks controlled for all the other differences between students who go to University and those who do not. In practice, there are likely many other differences between these two groups of students that we cannot measure, so this is a best-case scenario.

27.2 Which direction?

In the above example, we were able to compute the bias term. Generally we cannot compute the bias term, because we don’t have data on the omitted variable (if we did we would put it in the model). However, when building a model we should always think through the direction and possible magnitude of the bias.

These depend on two factors: $\hat{\beta}_2$ and $\tilde{\delta}_1$. Or, in other plain English 1) the effect of the omitted variable on the response variable, and 2) the correlation between the omitted variable and the included variable. If we know the sign of each of these, we can determine the sign of the bias term.

	$\text{Corr}(x_1, x_2) > 0$	$\text{Corr}(x_1, x_2) < 0$
Omitted variable has a positive effect on the response variable ($\beta_2>0$)	Positive bias	Negative bias
Omitted variable has a negative effect on the response variable ($\beta_2<0$)	Negative bias	Positive bias

Positive bias means $\tilde{\beta}_1$ is too high – it overstates the effect of $x_1$ on the response variable. A negative bias means the estimate is too low.

Take the education case. Better grade 12 marks raise income, so $\beta_2 > 0$. Stronger students also go further in school, so the correlation is positive. Top-left cell: positive bias, the estimate is too big. That is what we found, and we could have said so before running anything.

Here are a list of other examples of omitted variable bias:

Fertilizer and yield, omitting soil quality. Farmers apply more fertilizer to their better land ($\text{Corr}(\text{fertilizer},\ \text{soil quality}) > 0$), and better land yields more on its own ($\beta_2>0$). Both terms are positive, so the fertilizer coefficient is biased upward.
Pesticide application and yield, omitting pest pressure. Spraying happens where the pests are present ($\text{Corr}(\text{pesticide},\ \text{pest pressure}) > 0$), and pests cut yield ($\beta_2<0$). This creates a negative bias, meaning that pesticide application can look like it reduces yield when it is actually helping.
Advertising and sales, omitting seasonality. Firms advertise hardest in the Christmas season, which is the quarter when people are already buying ($\text{Corr}(\text{advertising},\ \text{seasonal demand}) > 0$), and that demand raises sales on its own ($\beta_2>0$). Both terms are positive, so the advertising coefficient absorbs the Christmas that was coming anyway.
Organic premiums and profit, omitting operator experience. Organic growers who stay organic are the ones who are good at it; the ones who struggle revert to conventional ($\text{Corr}(\text{organic},\ \text{experience}) > 0$), and experience raises profit whatever you grow ($\beta_2>0$). Both terms are positive, so the premium looks more reliable than it would be for a new entrant.
Hospital quality and mortality, omitting case severity. The best hospitals take the sickest patients, because that is where those patients get referred ($\text{Corr}(\text{hospital quality},\ \text{case severity}) > 0$), and sicker patients are more likely to die ($\beta_2>0$). Both terms are positive, so mortality is biased upward for the best hospitals and they can look worse on a raw comparison.

27.3 All models are imperfect

It is important to note that almost all models that we develop will have some issues – all models are imperfect representations of reality. They are inevitably simplifications of complex processes. The question is not if they are perfect, but if they are useful. Of course useful is in the eye of the beholder.

When we fear an omitted variable may be biasing our results there are a few different things we can try.

Find a proxy variable. In our example, we used grade 12 average as a proxy for the unobserved ability and work ethic of students. This is imperfect – some students may have had a bad year in grade 12, or went to a school that just gave lower grades.
Sign the bias you expect and treat your coefficient as a lower/upper bound. If we believe that the omitted variable is positively correlated with both the explanatory variable and the response variable, then we can expect the coefficient of the included variables to be biased upward. We could then interpret the coefficient as an upper bound on the true effect of the explanatory variable.

For example, suppose that I am testing whether a new variety outperforms an older variety. I fear that the new variety is more likely to be adopted by farmers who are also more forward-thinking in terms of other management practices. This would cause an upward bias (variety adoption is positively correlated with other management practices, which are themselves positively correlated with yield).

If I find that the variety had a very small or no effect on yield (without controlling for other management practices), then I can conclude that the variety is not likely to be better than the older variety. Including the other management practices will, if anything, reduce the estimated effect of the new variety.

27.4 Beyond regression

In this section, we have talked exclusively about omitted variable bias in the context of regression. However, the same logic holds for any association that we find in observational data. Part of the work of this course is to help you think more critically about these associations. Consider some of the following examples I’ve taken from recent news reports:

Does cycling to work make you live longer? A 2024 Scottish study found that people who commuted by bicycle had a 47% lower risk of dying over the study period than non-active commuters. Cycling could certainly improve health. But people who are already healthy enough to cycle to work are also different from people who cannot. An existing illness might both prevent someone from cycling and increase their risk of dying. If we do not adequately account for health before the choice of commuting method, some of the apparent benefit of cycling could reflect who was able to get on a bicycle in the first place.
Do school phone bans improve students’ mental health? The 2025 SMART Schools study compared 30 English secondary schools and found no evidence of better mental wellbeing among students at schools with restrictive phone policies. Consider a possible omitted variable: problems that existed before the ban. Schools experiencing more serious problems might be more likely to introduce restrictions. If so, students could benefit from a ban and still have no better wellbeing than students elsewhere, because they started from a worse position. We would want to know how the schools differed before their policies were introduced.
Will learning AI skills raise your salary by 56%? PwC’s 2025 Global AI Jobs Barometer reported an average wage premium of 56% for jobs requiring AI skills, compared with jobs in the same occupation that did not require them. That sounds like a good reason to take an AI course. But jobs within the same occupation can still differ in seniority, technical difficulty, and employer. If AI skills are particularly sought by high-paying firms or for more demanding positions, then some of the wage gap could reflect those differences.

Being able to recognize the potential for omitted variable bias is an important skill for life – not just for a data analysis course. I hope that, even if you don’t run another regression after taking this course, you will still be able to think critically about the associations you see in the world around you.