19  Correlation

The last chapter was about looking. This one puts a number on what you saw: how strongly do two variables move together, and in which direction.

Learning Objectives

By the end of this chapter you should be able to:

  1. Compute and interpret a correlation coefficient.
  2. Explain what correlation does not measure.
  3. Distinguish correlation from causation, and name the ways they come apart.

19.1 Covariance and Correlation

The Intuition

Suppose you have two variables — say, fertilizer rate and yield — measured across 100 fields. You make a scatter plot. What does it look like?

  • If the points form an upward-sloping cloud, yield tends to increase with fertilizer.
  • If they form a downward-sloping cloud, yield tends to decrease with fertilizer.
  • If they are a shapeless blob, there is no obvious relationship.
  • If they form a tight line, the relationship is strong.
  • If they are widely scattered, the relationship is weak.

The correlation coefficient turns this visual intuition into a single number between -1 and +1.

The Formula

First, the covariance between two variables \(X\) and \(Y\), for a sample of size \(n\):

\[ \text{Cov}(X, Y) = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y}) \]

The covariance is positive when \(X\) and \(Y\) tend to be above (or below) their means together, and negative when one is above while the other is below. But its magnitude depends on the units of both variables, which makes it hard to interpret directly.

The (Pearson) correlation coefficient normalizes the covariance by the standard deviations:

\[ r = \frac{\text{Cov}(X, Y)}{s_X \, s_Y} \]

The result is always between -1 and +1, regardless of units.

  • \(r = +1\): perfect positive linear relationship.
  • \(r = 0\): no linear relationship.
  • \(r = -1\): perfect negative linear relationship.
  • \(|r|\) between 0 and 1 measures the strength of the (linear!) relationship.

In Excel and R

Excel:

=CORREL(A2:A101, B2:B101)

R:

cor(trial$fertilizer_kg_ha, trial$yield_bu_acre)

Or every pair at once:

cor(trial[, c("fertilizer_kg_ha", "rainfall_mm", "yield_bu_acre")])

That last one is worth running on the trial data. Yield correlates with fertilizer, and it correlates with rainfall – but fertilizer and rainfall also correlate with each other, at about 0.47. Hold on to that number; it is the whole subject of Section 20.6.

What Correlation Does Not Measure

Correlation measures linear relationships. If the true relationship between \(X\) and \(Y\) is a curve — say, \(Y\) rises with \(X\) and then falls — the correlation can be zero even though there is a very strong relationship. This is why you should always plot your data before computing a correlation. A single scatter plot tells you more than a single number.

Anscombe’s quartet in Section 18.3 makes the point concretely: four datasets with identical correlations that look nothing alike.

19.2 Correlation Is Not Causation

If you remember nothing else from this course, remember this sentence. Correlation does not imply causation.

There are several reasons two variables can be correlated without one causing the other:

  1. Coincidence. If you look at enough pairs of variables, some will be correlated by chance alone. There is a famous website (tylervigen.com) that collects absurd correlations: per-capita cheese consumption correlates with deaths by bedsheet entanglement. Neither causes the other. They are both just trending over time.
  2. Reverse causation. You observe that people who exercise more have lower weights. Does exercise cause low weight, or do people with low weight exercise more? Usually a bit of both.
  3. Confounding. A third variable causes both. Ice cream sales correlate with drowning deaths. Does ice cream cause drowning? No. Both are caused by summer.
  4. Selection. The dataset was assembled in a way that induces a spurious correlation. (We will see examples of this when we get to sampling.)

To establish causation, you generally need either a controlled experiment (randomly assign treatment and control) or some clever reasoning about the source of variation. This is the domain of causal inference, which you will meet in AREC 262.

For AREC 261, the point is: any time you see a correlation, the first question should be “is there a plausible causal story, and what else could explain this?” The second question should be “could it be the reverse direction?” And the third should be “what could be confounding this?”

19.3 Test Bank Sample

  1. (Concept.) You find a correlation of 0.6 between two variables. What does that mean? What doesn’t it mean?
  2. (Formula.) Write the formula for the Pearson correlation coefficient.
  3. (Limits.) Give an example of two variables with a strong relationship but a correlation near zero.
  4. (Correlation vs causation.) Give an example of two variables that are correlated but where neither causes the other.
  5. (Confounding.) In the trial data, fertilizer and rainfall are themselves correlated. Why does that complicate the claim that fertilizer raises yield?

19.4 Practice Exercises

  1. Compute the correlation between fertilizer_kg_ha and yield_bu_acre in the trial data. Interpret it in a sentence.
  2. Compute the correlation between rainfall_mm and yield_bu_acre. Which predictor is more strongly related to yield?
  3. Compute the correlation between the two predictors themselves. What does that tell you about the trial?
  4. Compute the correlation of each pair in Anscombe’s quartet, then plot them. What is the lesson?