Understand statistics as inference from a random sample, and decide by simulation whether a model is consistent with data

Learning goal: Understand statistics as a process for making inferences about population parameters based on a random sample from that population, distinguish a parameter from a statistic, and decide whether a specified model is consistent with results from a data-generating process by simulating that process many times.

Parameters and statistics. A population is the whole group a question is about. A parameter is a number that describes the population: the population mean μ, the population proportion p, the population standard deviation σ. Parameters are usually unknown, because measuring everyone is too expensive or impossible. A sample is the part of the population actually measured, and a statistic is a number computed from the sample: the sample mean x̄, the sample proportion p̂, the sample standard deviation s. Hypothetical example: a school has 1,200 students, and the question is what proportion walk to school. The parameter is p, the proportion of all 1,200 who walk. A simple random sample of 50 students is drawn, using a random-number generator on the list of all 1,200 so that every group of 50 has the same chance of being chosen, and 18 of them walk. The statistic is p̂ = 18 ÷ 50 = 0.36.

Statistics as a process. Statistical inference is the process of using the statistic to say something about the parameter, together with an honest statement of how far off the statistic could be. The chain runs: state a question about a population parameter; select a random sample from that population; compute the statistic; use what is known about how statistics vary from sample to sample to make a claim about the parameter with a margin of uncertainty. The second step is the one that makes the rest possible. If the 50 students were the first 50 to arrive on a rainy morning, the sample would be tilted toward bus riders and the statistic would say nothing reliable about p. A random sample gives every member of the population a known chance of selection, removes the selector's preferences from the choice and, crucially, makes the behavior of the statistic predictable: another random sample of 50 would give a somewhat different p̂, perhaps 0.30 or 0.42, and the size of that sample-to-sample variation can be worked out or simulated. Sampling variability is not an error; it is the thing statistics measures. A later lesson turns it into a margin of error.

Models and data-generating processes. A model is a claim about the process that produces data. "This coin lands heads with probability 0.5 on each flip" is a model; so is "this spinner lands on red on 25% of spins" and "each of the 1,200 students is a walker with probability p, independently." The second skill in this lesson is to decide whether a model is consistent with results that were actually observed. The logic is simple: if the model were true, what results would the process produce, and how often? If the observed result is the kind the model produces routinely, the data are consistent with the model. If the model would produce a result like the observed one only rarely, the data give reason to question the model.

Simulation as the tool. Sometimes the probability can be calculated directly. For a fair coin, the probability of five tails in a row is (1/2)⁵ = 1/32 = 0.03125. Often it is easier to simulate: use a random-number generator to imitate the process under the model, repeat it many times, and count how often a result at least as extreme as the observed one appears. The simulation must copy the process faithfully: the same number of trials, the model's probability for each trial, and independent trials. The counts quoted in this lesson are illustrative: they show what one run of each simulation typically produces, checked against the exact probability where one exists, but they are not a recorded run that a reader can reproduce. A real run gives slightly different counts.

Model example: five tails in a row. A model says a spinning coin falls heads up with probability 0.5. Would five tails in a row make you question it? Simulation: 1,000 sets of five flips, each flip a random number below 0.5 counted as tails. The run produced 0 tails in 29 sets, 1 tail in 149, 2 in 298, 3 in 304, 4 in 185 and 5 in 35. Five tails appeared in 35 of 1,000 sets, 3.5%, close to the exact 3.125%. So five tails in a row is uncommon but not rare: a fair coin does it about one time in 32. Whether it is unusual enough to question the model depends on the rule stated before flipping: 1/32 ≈ 3.1% if only a run of tails was the concern, or 2/32 = 6.25% if all heads or all tails counts. With only five flips it should prompt more flips rather than a verdict. That is a judgment, and the number that informs it is the probability, 1 in 32, not a feeling that the run "looks" suspicious.

Model example: three reds in forty spins. A spinner is claimed to land on red 25% of the time, so 40 spins should give about 10 reds. A student gets only 3. Simulation under the model: 1,000 sets of 40 spins, each spin red with probability 0.25. The run produced between 1 and 19 reds per set, most often 9 or 10, and 3 or fewer reds in only 10 of the 1,000 sets, 1%. (The exact probability of 3 or fewer is 0.0047.) A result this extreme is something the model produces about once in a hundred tries or less. That is strong reason to question the model: the spinner may be unbalanced, or the spinning method may favor some sectors. The data do not prove the model false; a fair spinner could do this once in a while. They make the model hard to believe.

A consistent result. Twelve heads in 20 flips of a coin looks high. Simulating 1,000 sets of 20 fair flips, results at least as far from 10 as 12 (that is, 12 or more heads, or 8 or fewer) occurred in 525 sets, 52.5% (exactly 50.3%). The model produces results like this more than half the time, so 12 heads in 20 is entirely consistent with a fair coin. Consistency is not proof either: it means the data give no reason to reject the model.

How unlikely is unlikely? A common convention treats a probability below 5% as reason to question a model, but the cutoff is a convention, and the right response to a borderline result is to collect more data. Two errors to avoid: declaring a model false because a merely uncommon result occurred (the 1-in-32 coin), and declaring it true because the data are consistent with it, when the data would be consistent with many other models too.

Worked parameter and statistic: A hypothetical district wants the mean number of books its 9,000 students read last summer. It randomly selects 120 students and finds a sample mean of 3.4 books. Population: the 9,000 students. Parameter: μ, the mean for all 9,000, unknown. Sample: the 120. Statistic: x̄ = 3.4. Inference: μ is probably near 3.4, and the next lesson shows how to say how near. A second random sample would give a different x̄, perhaps 3.1 or 3.7; that variation is what the margin of error will describe.

Worked exact probability against simulation: Under the fair-coin model, P(five tails) = 0.5 × 0.5 × 0.5 × 0.5 × 0.5 = 1/32 = 0.03125. In 1,000 simulated sets we expect about 31 sets with five tails; one run gave 35. The simulation agrees with the calculation within the run-to-run variation one expects from 1,000 repetitions, which is how a simulation is checked when an exact answer exists.

Worked simulation design for the spinner: Model: each spin is red with probability 0.25, independently. One trial of the simulation: generate 40 random numbers between 0 and 1, count how many are below 0.25, record the count. Repeat 1,000 times. Compare: how many of the 1,000 counts are 3 or fewer? One run gave 10. Decision: 10 in 1,000 is 1%, so a result of 3 reds is one the model rarely produces; question the model. With random digits instead of a computer, read pairs of digits 00–99 and call 00–24 red, 40 pairs per trial.

Worked consistent result: Observed: 12 heads in 20 flips. Model: fair coin. Result at least as extreme: 12 or more heads or 8 or fewer, since a fair model is questioned by too many or too few. One run of the simulation gave 525 of 1,000 sets that extreme, so the observed result is ordinary under the model. Conclusion: the data are consistent with a fair coin; they do not establish that it is fair.

Represent the simulation: Flip a coin five times and record the number of tails; repeat 40 times as a class and make a dot plot of the 40 counts. Beside it write the 1,000-run frequencies 29, 149, 298, 304, 185, 35 as proportions and the exact probabilities 1/32, 5/32, 10/32, 10/32, 5/32, 1/32. Explain in two sentences why the three columns differ and why the differences shrink as the number of repetitions grows.

Independent performance: Choose a model you can test: a die that is claimed to be fair, a bag with a claimed proportion of one color, or a game piece claimed to land face up half the time. (1) State the model precisely, including the number of trials you will run. (2) Collect the real data. (3) Simulate at least 200 sets under the model with a spreadsheet random function or random digits, record the statistic for each set, and count how many are at least as extreme as your observed result. (4) Decide whether the data are consistent with the model and say what probability drove the decision. (5) Identify the population, the parameter and the statistic in your study. Success means the simulation copies the model's probability and trial count exactly, the extreme region is defined before the count is made, and the conclusion distinguishes "inconsistent with the model" from "the model is proved false".