Terminology and Process of Hypothesis Testing

Learning Objectives

Understand key terminology and concepts in hypothesis testing
Understand the process of hypothesis testing

Why This Matters

Every time a pharmaceutical company submits a new drug to the FDA, every time Netflix decides whether a thumbnail change actually increased clicks, every time a city planner argues that a new bus route reduced commute times -- a hypothesis test is the framework that separates "this looks like it worked" from "we have statistical evidence that it worked." The formal vocabulary and process aren't bureaucratic formality. They're the difference between decisions based on evidence and decisions based on gut feelings.

How to Use This Simulation

Select a preset scenario to see a real-world research claim that a hypothesis test could address.
Click each step in the flowchart to walk through the hypothesis testing process -- vocabulary, worked examples, and a decision task appear for each stage.
Complete the vocabulary tasks at Steps 1, 3, and 5 to test whether you can distinguish terms students commonly confuse.
Check the Explanation Panel below -- it updates at every stage to connect vocabulary to the full process.

Scenario:

Hypotheses

Click Step 1 to begin

Walk through the process to build the hypothesis pair.

Test Setup

Significance level, test type, and parameter.

Process Status

0 of 6 steps explored

Click each step in order to walk through the full process.

What's Happening

Quick Check

A researcher conducts a hypothesis test and fails to reject H₀. Which interpretation is correct?

Try This

A battery manufacturer claims their AA batteries last at least 500 hours on average. A consumer testing lab suspects the batteries last fewer than 500 hours and plans to test a sample.

(1) What is the population parameter being tested? (2) Write H₀ using correct notation. (3) Write Hₐ using correct notation. Switch to the "Battery Life" preset and click Step 1 to verify your answers against the simulation's hypothesis display.

Consider two claims about a customer service chatbot's average response time:

(a) "The average response time is less than 2 minutes."

(b) "The average response time has changed since the last update."

(1) Write H₀ and Hₐ for each claim. (2) Identify the test type for each (left-tailed, right-tailed, or two-tailed). (3) In one sentence, explain why the direction of the research claim determines which type of test is used.

A UX team redesigned their app's checkout flow. They expect the redesign to increase the completion rate, but a regression would also be important to detect. Should the test be one-tailed (Hₐ: μ_new > μ_old) or two-tailed (Hₐ: μ_new ≠ μ_old)?

(1) Write H₀ and Hₐ for both formulations. (2) Describe how each formulation would change the conclusions drawn from the same data. (3) Recommend a formulation with reasoning that addresses the team's actual research question.

Instructor Notes

Teaching Notes

This simulation is the gateway to the seven-simulation hypothesis testing arc (Sims 22-28). It establishes vocabulary and process before any computation. Let students click through the entire six-step flowchart with one scenario before discussing anything. The vocabulary tasks at Steps 1, 3, and 5 surface the exact confusions that cause errors in Sims 23-28: confusing H₀ with Hₐ, mixing up "parameter" and "test statistic," and using "accept H₀" instead of "fail to reject."

The preset scenarios are intentionally varied: left-tailed (allergy medication, battery life), two-tailed (commute time), and right-tailed (campus dining). Have students walk through at least two different presets to experience the "every test follows the same process" insight alongside the vocabulary work.

Common Student Errors

Assigning the research claim to H₀ instead of Hₐ. Students think "hypothesis" means "what I believe," so the null must be the researcher's belief. It's the opposite.
Saying "accept H₀" instead of "fail to reject H₀." The jury analogy (not guilty ≠ innocent) helps, but students need to hear it multiple times across Sims 22-28.
Treating α and p-value as the same thing. At this stage, emphasize that α is chosen before the test and p is computed after. The comparison of the two produces the decision.
Confusing "parameter" with "test statistic." Both are numbers associated with the hypothesis, but the parameter is the population value being claimed, and the test statistic is computed from the sample to evaluate that claim.

Discussion Questions

Why is it important to state hypotheses and choose α before collecting data? What could go wrong if a researcher chose α after seeing results?
A friend says "the study proved the drug works." Based on the process you walked through, what's wrong with this phrasing, and how would you correct it?
Two researchers study the same question but one uses a one-tailed test and the other uses a two-tailed test. Could they reach different conclusions from the same data? Why?

Exam Connection

Exam items on this topic typically present a research claim and ask students to identify H₀ and Hₐ, determine the test type (one-tailed vs two-tailed), or select the correct interpretation of a hypothesis test result. The three mini-interactions in the flowchart directly prepare students for these formats. The Quick Check targets the "fail to reject = proof" misconception, which appears frequently as a distractor on exams.