The t-Test

Students who used a revision app scored an average of $76.5$, and students who didn’t scored $69.9$. Is the app really better, or could a gap like that easily happen by chance with small groups? When you want to compare the means of two populations using a sample from each, you use a two-sample t-test. It follows the same steps as every test in introduction to hypothesis testing; your GDC does the calculation, and your job is to set it up and interpret it.

Key ideas

What the test compares

You have two independent (unpaired) samples, one from each population, such as plants grown in two soils, or students in two different classes. Let $\mu_1$ and $\mu_2$ be the two population means. The null hypothesis is always that they’re equal:

$$H_0: \mu_1 = \mu_2$$

The alternative hypothesis depends on the question:

Question asks whether the means are…	$H_1$	Type of test
different	$\mu_1 \ne \mu_2$	two-tailed
$\mu_1$ is greater	$\mu_1 \gt \mu_2$	one-tailed
$\mu_1$ is smaller	$\mu_1 \lt \mu_2$	one-tailed

The idea behind the test statistic

The test asks: is the gap between the two sample means $\bar{x}_1 - \bar{x}_2$ large compared with how much the samples vary? The t-statistic measures the gap in units of its standard error. A $t$ close to $0$ means the gap is small compared with the natural variation; a $t$ far from $0$ (positive or negative) is evidence that the population means differ.

If $H_0$ is true, $t$ follows a t-distribution with

$$\nu = n_1 + n_2 - 2$$

degrees of freedom, where $n_1$ and $n_2$ are the sample sizes. A t-distribution is bell-shaped and symmetric about $0$, like the normal curve, but with slightly heavier tails.

For a two-tailed test with $t = 2.93$ and $13$ degrees of freedom, the p-value is the total shaded area in both tails: $0.0117$.

Doing the test with technology

In IB exams you always use technology for the t-test. On your GDC, choose the two-sample t-test, then:

1. Enter the data as two lists, or enter the summary statistics: each sample’s mean $\bar{x}$, standard deviation $s_{n-1}$ and size $n$.
2. Choose the alternative hypothesis: $\ne$, $\gt$ or $\lt$.
3. Choose pooled: yes.

The GDC gives $t$, the p-value and the degrees of freedom. Then compare the p-value with the significance level and write your conclusion in context.

If you enter summary statistics, use the sample standard deviation $s_{n-1}$ (often shown as $s_x$ on a GDC), the one that divides by $n - 1$.

Assumptions

The t-test is only valid if:

• the two samples are independent and randomly chosen (at SL, the samples are always unpaired: different individuals in each group)
• the variable is normally distributed in both populations
• the two populations have equal variances. That’s what “pooled” means: the GDC combines the two samples to estimate one common variance. In IB exams, assume the variances are equal and use the pooled test.

The population variances are unknown, which is exactly why you use a $t$-test rather than a normal distribution: the sample standard deviations are estimates.

Worked examples

Example 1: From summary statistics

A researcher records the nightly sleep of $15$ students at each of two schools.

	$n$	Mean (hours)	$s_{n-1}$ (hours)
School P	$15$	$7.40$	$0.90$
School Q	$15$	$7.85$	$0.78$

Assuming sleep times are normally distributed with equal variances, test at the $5\%$ significance level whether the mean sleep time is different at the two schools.

Solution. Let $\mu_P$ and $\mu_Q$ be the population mean sleep times.

$$H_0: \mu_P = \mu_Q \qquad H_1: \mu_P \ne \mu_Q$$

“Different” means a two-tailed test. Enter the summary statistics into the GDC’s two-sample t-test (pooled, $\ne$):

$$t = -1.46, \qquad p = 0.154 \text{ (to 3 s.f.)}, \qquad \nu = 15 + 15 - 2 = 28$$

$0.154 \gt 0.05$, so do not reject $H_0$. There is insufficient evidence at the $5\%$ significance level that the mean sleep time is different at the two schools.

Example 2: From raw data

A biology class grew sunflower seedlings in two types of soil and measured their heights (in cm) after two weeks.

Soil A	$12.4$	$14.1$	$13.2$	$15.0$	$12.8$	$13.9$	$14.6$	$13.5$
Soil B	$11.8$	$12.9$	$12.1$	$13.4$	$11.5$	$12.6$	$13.0$

Assume heights are normally distributed with equal variance. Test at the $5\%$ significance level whether the mean height is different for the two soils.

Solution. Let $\mu_A$ and $\mu_B$ be the population mean heights.

$$H_0: \mu_A = \mu_B \qquad H_1: \mu_A \ne \mu_B$$

Enter the two lists and run the pooled two-sample t-test with $\ne$. The sample means are $\bar{x}_A = 13.7$ cm and $\bar{x}_B = 12.5$ cm (to 3 s.f.), and

$$t = 2.93, \qquad p = 0.0117 \text{ (to 3 s.f.)}, \qquad \nu = 8 + 7 - 2 = 13$$

$0.0117 \lt 0.05$, so reject $H_0$. There is sufficient evidence at the $5\%$ significance level that the mean height of seedlings is different in the two soils. (The figure above shows this p-value as the two shaded tails.)

At the $1\%$ level, the conclusion would be different: $0.0117 \gt 0.01$.

Example 3: A one-tailed test

A teacher believes a revision app increases mean test scores. Ten students used the app and nine didn’t.

With app	$72$	$81$	$68$	$77$	$85$	$74$	$79$	$70$	$83$	$76$
Without	$70$	$65$	$74$	$68$	$71$	$77$	$63$	$69$	$72$

Assuming scores are normally distributed with equal variances, test the teacher’s belief at the $5\%$ significance level.

Solution. Let $\mu_A$ and $\mu_W$ be the population mean scores with and without the app. “Increases” has a direction, so the test is one-tailed:

$$H_0: \mu_A = \mu_W \qquad H_1: \mu_A \gt \mu_W$$

Enter the lists (app first) and run the pooled two-sample t-test with $\gt$. The means are $76.5$ and $69.9$ (to 3 s.f.), and

$$t = 2.86, \qquad p = 0.00546 \text{ (to 3 s.f.)}, \qquad \nu = 10 + 9 - 2 = 17$$

$0.00546 \lt 0.05$, so reject $H_0$. There is sufficient evidence at the $5\%$ significance level that the mean test score is higher for students who use the app.

This shows the scores are higher, not necessarily that the app caused it. If students chose whether to use the app, the more motivated ones might have picked it. Randomly assigning students to the two groups would make a causal claim much stronger.

Example 4: No significant difference

Six batteries of brand X and six of brand Y were tested, with these lifetimes (in hours).

Brand X	$18.2$	$19.5$	$17.8$	$20.1$	$18.9$	$19.3$
Brand Y	$18.8$	$19.9$	$18.4$	$19.2$	$20.4$	$18.6$

• (a) State the assumptions needed for a two-sample t-test.
• (b) Test at the $10\%$ significance level whether the mean lifetimes are different.

Solution.

(a) The lifetimes of each brand are normally distributed, the two populations have equal variances, and the two samples are independent random samples.

(b) $H_0: \mu_X = \mu_Y$, $H_1: \mu_X \ne \mu_Y$. The pooled two-sample t-test gives

$$t = -0.528, \qquad p = 0.609 \text{ (to 3 s.f.)}, \qquad \nu = 10$$

$0.609 \gt 0.10$, so do not reject $H_0$. There is insufficient evidence at the $10\%$ significance level that the mean lifetimes of the two brands are different.

The sample means are $18.97$ h and $19.22$ h. A gap of $0.25$ h is small compared with how much individual batteries vary, so it could easily be due to chance.

Common mistakes

Choosing the wrong alternative hypothesis. Read the question for direction words. “Different” or “changed” means $\ne$ (two-tailed); “higher”, “faster”, “improves” mean $\gt$ or $\lt$ (one-tailed). And make sure the order on your GDC matches: if $H_1$ is $\mu_1 \gt \mu_2$, enter group $1$ as the first list.

Forgetting to choose pooled. IB exam questions expect you to assume equal variances and use the pooled test. The unpooled version gives slightly different values of $t$, $p$ and the degrees of freedom.

Using the wrong standard deviation. When entering summary statistics, use the sample standard deviation $s_{n-1}$, not $\sigma_n$.

Using a two-sample t-test on paired data. If the same people are measured twice (before and after), the samples aren’t independent. The SL two-sample test is for two separate groups.

Ignoring the normality assumption. The t-test assumes each population is normally distributed. For strongly skewed data (like household incomes) with small samples, the p-value can’t be trusted.

Concluding that the means are equal. A large p-value means there’s insufficient evidence of a difference, not proof that the means are the same. Write “do not reject $H_0$”, with the significance level and the context.

Practice

1. (Warm-up) Write $H_0$ and $H_1$ for each test, defining your symbols.

• (a) Is the mean time to finish a puzzle different for students who listen to music and students who don’t?
• (b) Do tomato plants given fertilizer have a greater mean yield than plants without it?

Solution

(a) Let $\mu_M$ and $\mu_N$ be the population mean times with and without music. $H_0: \mu_M = \mu_N$, $H_1: \mu_M \ne \mu_N$ (two-tailed).

(b) Let $\mu_F$ and $\mu_N$ be the population mean yields with and without fertilizer. $H_0: \mu_F = \mu_N$, $H_1: \mu_F \gt \mu_N$ (one-tailed).

2. (Warm-up) A two-sample t-test comparing the mean heights of two varieties of sunflower gives $p = 0.034$. Write the conclusion at

• (a) the $5\%$ significance level
• (b) the $1\%$ significance level.

Solution

(a) $0.034 \lt 0.05$, so reject $H_0$. There is sufficient evidence at the $5\%$ significance level that the mean heights of the two varieties are different.

(b) $0.034 \gt 0.01$, so do not reject $H_0$. There is insufficient evidence at the $1\%$ significance level that the mean heights are different.

3. (Core) The fuel consumption (in L/100 km) of two car models was measured.

Model 1	$7.8$	$8.2$	$7.5$	$8.0$	$8.4$	$7.9$
Model 2	$8.3$	$8.6$	$8.1$	$8.9$	$8.4$	$8.7$	$8.2$

Assuming normal distributions with equal variances, test at the $5\%$ significance level whether the mean fuel consumption of the two models is different.

Solution

$H_0: \mu_1 = \mu_2$, $H_1: \mu_1 \ne \mu_2$, where $\mu_1$ and $\mu_2$ are the population mean fuel consumptions.

The pooled two-sample t-test gives

$$t = -2.94, \qquad p = 0.0135 \text{ (to 3 s.f.)}, \qquad \nu = 6 + 7 - 2 = 11$$

$0.0135 \lt 0.05$, so reject $H_0$. There is sufficient evidence at the $5\%$ significance level that the mean fuel consumption of the two models is different. (Model 1’s sample mean, $7.97$, is lower than Model 2’s, $8.46$.)

4. (Core) A researcher thinks that drinking coffee reduces mean reaction time. Reaction times (in milliseconds) were recorded for two groups.

Coffee	$248$	$262$	$239$	$255$	$251$	$244$	$258$	$247$
No coffee	$259$	$270$	$252$	$266$	$249$	$263$	$271$	$258$

Test the researcher’s claim at the $1\%$ significance level, assuming normal distributions with equal variances.

Solution

Let $\mu_C$ and $\mu_N$ be the population mean reaction times with and without coffee. “Reduces” gives a one-tailed test:

$$H_0: \mu_C = \mu_N \qquad H_1: \mu_C \lt \mu_N$$

The pooled two-sample t-test (coffee as the first list, $\lt$) gives

$$t = -2.70, \qquad p = 0.00871 \text{ (to 3 s.f.)}, \qquad \nu = 14$$

$0.00871 \lt 0.01$, so reject $H_0$. There is sufficient evidence at the $1\%$ significance level that the mean reaction time is lower for people who drink coffee.

5. (Core) Two classes wrote the same test.

	$n$	Mean	$s_{n-1}$
Class A	$12$	$64.2$	$5.1$
Class B	$14$	$60.5$	$4.6$

Assuming normal distributions with equal variances, test at the $5\%$ significance level whether the mean marks of the two classes are different.

Solution

$H_0: \mu_A = \mu_B$, $H_1: \mu_A \ne \mu_B$.

Entering the summary statistics into the pooled two-sample t-test:

$$t = 1.95, \qquad p = 0.0636 \text{ (to 3 s.f.)}, \qquad \nu = 12 + 14 - 2 = 24$$

$0.0636 \gt 0.05$, so do not reject $H_0$. There is insufficient evidence at the $5\%$ significance level that the mean marks of the two classes are different.

6. (Core) Explain why the SL two-sample t-test is not appropriate in each case.

• (a) Twelve students’ marks are compared before and after a revision course.
• (b) The annual incomes of $8$ people in each of two towns are compared. Income data are strongly right-skewed.

Solution

(a) The same students are measured twice, so the two sets of marks are paired, not independent. The two-sample test assumes two separate, independent groups.

(b) The t-test assumes the variable is normally distributed in each population. Strongly skewed data like incomes aren’t normal, and with only $8$ values per group the p-value wouldn’t be reliable.

7. (Core) Two groups of $7$ plants were grown under different lights, and their growth (in cm) recorded.

Light A	$5.2$	$6.1$	$5.8$	$6.4$	$5.5$	$6.0$	$5.7$
Light B	$5.0$	$5.6$	$5.3$	$5.9$	$5.1$	$5.8$	$5.4$

Assuming normal distributions with equal variances, find the p-value and state the conclusion at the $5\%$ level for

• (a) $H_1$: the mean growth is different under the two lights
• (b) $H_1$: the mean growth is greater under light A.

Solution

In both cases $H_0: \mu_A = \mu_B$, the pooled test gives $t = 1.88$ (to 3 s.f.), and $\nu = 12$.

(a) Two-tailed: $p = 0.0851$ (to 3 s.f.). $0.0851 \gt 0.05$, so do not reject $H_0$. There is insufficient evidence at the $5\%$ level that the mean growth is different under the two lights.

(b) One-tailed: $p = 0.0426$ (to 3 s.f.), half the two-tailed value. $0.0426 \lt 0.05$, so reject $H_0$. There is sufficient evidence at the $5\%$ level that the mean growth is greater under light A.

The one-tailed test only “uses up” the significance level in one direction, so it’s easier to reject $H_0$ in that direction. That’s why the choice must be made from the question, before seeing the data.

8. (Challenge) Using the data from question 7, a student tests $H_1$: the mean growth is less under light A. Find the p-value and explain why it’s so large.

Solution

The pooled test with $\lt$ gives $p = 0.957$ (to 3 s.f.), so do not reject $H_0$.

The p-value is the area in the lower tail, below $t = 1.88$. Since $t$ is positive (light A’s sample mean is larger), almost all of the distribution is below it: $p = 1 - 0.0426 = 0.957$. The data point in the opposite direction to $H_1$, so they give no evidence at all for it.

9. (Challenge) Two groups have sample means $52.0$ and $47.0$, and both have sample standard deviation $6.0$. Assume normal distributions with equal variances.

• (a) Test at the $5\%$ level whether the population means are different if each group has $6$ members.
• (b) Repeat with $15$ members in each group.
• (c) Explain why the conclusions differ.

Solution

$H_0: \mu_1 = \mu_2$, $H_1: \mu_1 \ne \mu_2$.

(a) Summary statistics with $n = 6$: $t = 1.44$, $p = 0.180$ (to 3 s.f.), $\nu = 10$. $0.180 \gt 0.05$, so do not reject $H_0$: insufficient evidence at the $5\%$ level that the means differ.

(b) With $n = 15$: $t = 2.28$, $p = 0.0303$ (to 3 s.f.), $\nu = 28$. $0.0303 \lt 0.05$, so reject $H_0$: sufficient evidence at the $5\%$ level that the means differ.

(c) The gap between the sample means ($5.0$) and the spread are the same in both cases. But sample means from larger samples vary less from sample to sample, so a gap of $5.0$ is much less likely to be due to chance with $15$ in each group than with $6$. Larger samples make it easier to detect a real difference.