7  Probability and Random Variables

Learning Objectives

By the end of this module you should be able to:

  1. Define probability and distinguish between frequentist and subjective interpretations.
  2. Compute basic probabilities using addition and multiplication rules.
  3. Define a random variable and distinguish discrete from continuous ones.
  4. Work with probability mass functions (PMFs) and probability density functions (PDFs).
  5. Compute expected value and variance for simple random variables.
  6. Recognize the normal, uniform, binomial, and Poisson distributions.

7.1 What Is Probability?

Probability is the mathematics of uncertainty. You need it whenever you want to make a statement about something that hasn’t happened yet, or a statement about something whose outcome you don’t know.

There are two main ways to interpret probability, and although the mathematics is the same either way, it helps to be aware of both.

Frequentist interpretation: the probability of an event is the long-run fraction of times it occurs if you repeat the experiment many times. “The probability of rain tomorrow is 30%” means: in a large number of days with conditions like tomorrow’s, it rains on about 30% of them.

Subjective (Bayesian) interpretation: the probability of an event is a measure of your degree of belief that it will occur. “The probability of rain tomorrow is 30%” means: I am 30% confident that it will rain tomorrow. This works even for one-off events where “long-run frequencies” don’t literally make sense (like “the probability that this particular tractor will break down next month”).

For almost everything in this course the two interpretations will give the same answer, so you can pick whichever feels more intuitive. Just know that there are debates between statisticians about which is “right,” and when you encounter them in the wild, you now know what they are arguing about.

7.2 Basic Rules of Probability

We write \(P(A)\) for the probability of event \(A\). The basic rules:

  1. Probabilities are between 0 and 1: \(0 \leq P(A) \leq 1\).
  2. Certain events have probability 1: \(P(\text{something happens}) = 1\).
  3. Impossible events have probability 0: \(P(\text{nothing happens}) = 0\).
  4. Complement: \(P(\text{not } A) = 1 - P(A)\).
  5. Addition for disjoint events: if \(A\) and \(B\) cannot both happen, \(P(A \text{ or } B) = P(A) + P(B)\).
  6. General addition: \(P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)\).
  7. Multiplication for independent events: if \(A\) and \(B\) are independent (one doesn’t affect the probability of the other), \(P(A \text{ and } B) = P(A) \cdot P(B)\).
  8. Conditional probability: \(P(A \mid B) = P(A \text{ and } B) / P(B)\). Read “\(P(A\) given \(B)\).”

Worked example: you roll two fair six-sided dice. What is the probability the total is 7?

  • The sample space has \(6 \times 6 = 36\) equally likely outcomes.
  • Outcomes that sum to 7: (1,6), (2,5), (3,4), (4,3), (5,2), (6,1) — six of them.
  • \(P(\text{sum} = 7) = 6/36 = 1/6\).

Worked example: a field has an 80% chance of getting enough rain in June. The chance of enough rain in June and July, if those months are independent, is \(0.8 \times 0.8 = 0.64\). In practice the months are probably not independent — a drought in June often means a drought in July — so this calculation is optimistic.

7.3 Random Variables

A random variable is a number whose value depends on the outcome of a random process. We usually write them with capital letters: \(X\), \(Y\), \(Z\).

Examples:

  • \(X\) = the yield of a randomly selected field, in bu/ac.
  • \(Y\) = the number of calves born alive to a randomly selected cow in her next calving.
  • \(Z\) = the price of canola futures at a random moment next week.

Random variables come in two flavors: discrete and continuous.

Discrete random variables take values from a countable set — usually integers. Examples: number of calves, number of rainy days in a month, number of fields with a yield above 50.

Continuous random variables can take any value in some range. Examples: yield in bu/ac (in principle can be any real number), rainfall in mm, temperature.

The distinction matters because the math is different.

7.4 Discrete Random Variables

For a discrete random variable \(X\), we describe it with its probability mass function (PMF):

\[ p(x) = P(X = x) \]

The PMF gives the probability that \(X\) takes each particular value. It has to satisfy two things:

  1. \(p(x) \geq 0\) for all \(x\).
  2. \(\sum_x p(x) = 1\) (the probabilities sum to 1 over all possible values).

Example: \(X\) = the result of rolling a fair die. \(p(1) = p(2) = \ldots = p(6) = 1/6\). Sum is 1.

Expected Value and Variance

The expected value (or mean) of a discrete random variable is:

\[ E[X] = \sum_x x \cdot p(x) \]

This is the long-run average if you repeated the experiment many times. For a fair die, \(E[X] = 1/6 \cdot (1 + 2 + 3 + 4 + 5 + 6) = 21/6 = 3.5\).

The variance is:

\[ \text{Var}(X) = E[(X - E[X])^2] = \sum_x (x - E[X])^2 \cdot p(x) \]

And the standard deviation is \(\sqrt{\text{Var}(X)}\). Same units as \(X\).

The Binomial Distribution

Suppose you do \(n\) independent trials, each with a probability \(p\) of “success.” The number of successes is a random variable called the binomial distribution, written \(\text{Binomial}(n, p)\).

Example: you plant 10 seeds, each with a 90% chance of germinating. The number that germinate follows \(\text{Binomial}(10, 0.9)\).

The PMF is:

\[ P(X = k) = \binom{n}{k} p^k (1-p)^{n-k} \]

where \(\binom{n}{k} = \frac{n!}{k!(n-k)!}\) is the binomial coefficient (“\(n\) choose \(k\)”).

You don’t need to compute this by hand. In R:

dbinom(8, size = 10, prob = 0.9)   # P(X = 8)
pbinom(8, size = 10, prob = 0.9)   # P(X <= 8)

In Excel: =BINOM.DIST(8, 10, 0.9, FALSE) for the PMF, TRUE for the CDF.

The mean of \(\text{Binomial}(n, p)\) is \(np\), and the variance is \(np(1-p)\). So for 10 seeds with 90% germination, you expect \(10 \times 0.9 = 9\) to germinate, with variance \(10 \times 0.9 \times 0.1 = 0.9\) (standard deviation \(\approx 0.95\)).

The Poisson Distribution

If events happen at an average rate \(\lambda\) per unit (time, area, etc.), and each event is independent of the others, the number of events in a unit follows the Poisson distribution, \(\text{Poisson}(\lambda)\).

Example: weed seeds in a field arrive at an average density of 5 per square meter. The number of seeds in a randomly chosen square meter follows \(\text{Poisson}(5)\).

The PMF:

\[ P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!} \]

The mean and variance of \(\text{Poisson}(\lambda)\) are both \(\lambda\).

In R: dpois(3, lambda = 5), ppois(3, lambda = 5). In Excel: =POISSON.DIST(3, 5, FALSE).

Poisson distributions show up whenever you are counting rare-ish events in a time or space interval.

7.5 Continuous Random Variables

For a continuous random variable the mass function becomes a probability density function (PDF), written \(f(x)\). The key difference: for a continuous variable, \(P(X = x) = 0\) for any specific value (there are infinitely many possible values, so each one gets zero probability). Instead, probabilities come from areas under the PDF:

\[ P(a < X < b) = \int_a^b f(x) \, dx \]

The PDF must satisfy:

  1. \(f(x) \geq 0\) for all \(x\).
  2. \(\int_{-\infty}^{\infty} f(x) \, dx = 1\).

Don’t worry if the integral notation is unfamiliar. Conceptually, the PDF is a curve, and the probability that \(X\) falls in an interval is the area under the curve over that interval. You will compute these with tables or with R functions; we will not do integration by hand.

The cumulative distribution function (CDF) is:

\[ F(x) = P(X \leq x) = \int_{-\infty}^{x} f(t) \, dt \]

It gives the probability that \(X\) is at most \(x\). It always starts at 0 (for very negative \(x\)) and ends at 1 (for very positive \(x\)).

The Normal Distribution

The most important continuous distribution is the normal distribution, also called the Gaussian. You have probably seen its bell-shaped curve.

A normal distribution is specified by two parameters: the mean \(\mu\) and the standard deviation \(\sigma\). We write \(X \sim N(\mu, \sigma^2)\) (the second parameter is the variance, not the standard deviation, by convention).

The PDF is:

\[ f(x) = \frac{1}{\sigma \sqrt{2\pi}} \exp\left(-\frac{(x - \mu)^2}{2\sigma^2}\right) \]

You do not need to memorize this. What you need to know:

  • The curve is symmetric around \(\mu\).
  • About 68% of the area lies within \(\mu \pm \sigma\) (one standard deviation).
  • About 95% within \(\mu \pm 2\sigma\).
  • About 99.7% within \(\mu \pm 3\sigma\).

This “68-95-99.7 rule” is worth memorizing. It tells you, for any normal distribution, roughly how likely extreme values are.

The standard normal distribution is \(N(0, 1)\): mean zero, standard deviation one. Any normal can be converted to a standard normal by the transformation:

\[ Z = \frac{X - \mu}{\sigma} \]

This is called a \(z\)-score. It measures how many standard deviations above (or below) the mean an observation is. \(Z\)-scores are useful for comparing values from different normal distributions, and they are the basis of a lot of classical inference (which you will meet in AREC 262).

In R:

dnorm(2, mean = 0, sd = 1)     # PDF at x = 2
pnorm(2, mean = 0, sd = 1)     # CDF at x = 2 (P(Z <= 2))
qnorm(0.95, mean = 0, sd = 1)  # inverse CDF: value where CDF = 0.95
rnorm(100, mean = 0, sd = 1)   # 100 random draws from the distribution

In Excel: NORM.DIST, NORM.INV, NORM.S.DIST (for standard normal), etc.

The normal distribution is ubiquitous in statistics for two reasons. First, it shows up naturally in many real-world measurements (heights, test scores, measurement errors). Second, and more importantly, the Central Limit Theorem says that the sample mean of many independent observations — regardless of what distribution they came from — is approximately normal for large samples. We will come back to this in Module 10.

The Uniform Distribution

\(\text{Uniform}(a, b)\) has constant density between \(a\) and \(b\) and zero outside. It is the “nothing is more likely than anything else” distribution on an interval. Useful for simulation.

7.6 In Practice

You do not need to memorize any formulas beyond the conceptual understanding. In this course, you will use R (or Excel) to compute probabilities and percentiles. The point of the formal definitions is to understand what those functions are doing under the hood.

The key intuitions to keep:

  • A random variable is a number whose value depends on a random process.
  • Discrete variables have PMFs; continuous ones have PDFs.
  • Expected value is the long-run average; variance measures spread.
  • The binomial counts successes in independent trials.
  • The Poisson counts rare events per unit time/area.
  • The normal distribution shows up everywhere, and its standard form uses \(z\)-scores.

7.7 Test Bank Sample

  1. (Probability rules.) A field has a 70% chance of surviving a drought and a 90% chance of surviving hail. Assuming these are independent, what is the probability it survives both?
  2. (Discrete.) A hen lays eggs at a rate of 6 per week. Modeled as Poisson, what is the probability she lays exactly 5 in a given week?
  3. (Continuous.) Canola yields in a region are approximately normal with mean 45 and standard deviation 8 bu/ac. What is the probability that a randomly selected field has a yield above 55?
  4. (Interpretation.) What does it mean for a random variable to have expected value 50?
  5. (R/Excel.) Write the R or Excel function call that computes \(P(Z < 1.5)\) where \(Z\) is standard normal.
  6. (Concept.) Explain the 68-95-99.7 rule and give an example of when it is useful.

7.8 Practice Exercises

  1. Simulate 1000 rolls of two dice in R or Excel. Plot a histogram of the sums and verify that the distribution peaks at 7.
  2. For \(\text{Binomial}(20, 0.3)\), compute the mean, variance, and the probability of at most 5 successes.
  3. For a \(N(100, 15^2)\) distribution, find the value \(x\) such that \(P(X < x) = 0.9\).
  4. [TBD: an ag-relevant probability problem.]