Compute residuals, draw a residual plot and use it to assess informally how well a function fits the data

Learning goal: For data on two quantitative variables shown on a scatter plot, compute the residuals of a fitted function, draw a residual plot, and informally assess the fit of the function by discussing the residuals: their signs, their pattern and their size in context. Explain why a high correlation, a zero sum of residuals or a small residual at one point does not show that a function fits.

The data and a first function. On a hypothetical test track, a car is braked to a stop from seven speeds and the stopping distance is measured. The numbers are invented for teaching and chosen so that the arithmetic is clean. The pairs (speed in km/h, stopping distance in m) are (20, 5.8), (30, 11.1), (40, 15.8), (50, 21.9), (60, 30.0), (70, 39.3), (80, 47.6). On a scatter plot with speed on the horizontal axis, the variables are related positively and strongly: faster speeds go with longer stopping distances. The form looks almost straight at first glance, with a slight upward bend. Linear regression (least squares) returns d = 0.7v − 10.5, and the correlation coefficient is r = 0.992. Many people would stop here. Residuals are the tool for deciding whether they should.

What a residual is. For each data point, residual = observed value − predicted value, that is, y − f(x), where f is the fitted function. A residual is measured in the units of y. It is positive when the point lies above the graph (the function predicted too little), negative when the point lies below the graph (the function predicted too much) and zero when the graph passes through the point. There is one residual for every data point. For the line, the predicted distances at the seven speeds are 3.5, 10.5, 17.5, 24.5, 31.5, 38.5 and 45.5 m, so the residuals are 5.8 − 3.5 = +2.3, 11.1 − 10.5 = +0.6, 15.8 − 17.5 = −1.7, 21.9 − 24.5 = −2.6, 30.0 − 31.5 = −1.5, 39.3 − 38.5 = +0.8 and 47.6 − 45.5 = +2.1 m.

The residual plot. A residual plot graphs each residual against its x-value, with a horizontal line at zero. It takes the tilt of the fitted function out of the picture and magnifies what is left over, so patterns that are hard to see on the scatter plot become obvious. A function fits well when two things are true of its residuals. First, they show no pattern: positive and negative residuals are mixed along the whole x-axis, with no curve, no steady trend and no widening. Second, they are small in context: small compared with the y-values and with the accuracy the situation needs. For the line, the residuals read +, +, −, −, −, +, + from left to right: positive at both ends and negative in the middle, a U shape. That is a pattern, and a pattern means the function is missing something systematic about how the variables are related. Here the line predicts too little at low and high speeds and too much in between, which is what happens when a straight line is laid across data that curve upward.

Trying the function the context suggests. Physics gives a reason to expect a curve. While the driver reacts, the car covers a distance proportional to its speed; once the brakes are on, the braking distance under a steady deceleration is proportional to the square of the speed. A quadratic function is suggested, and quadratic regression returns d = 0.005v² + 0.2v (the constant term comes out as 0). Its predictions are 6.0, 10.5, 16.0, 22.5, 30.0, 38.5 and 48.0 m, so its residuals are −0.2, +0.6, −0.2, −0.6, 0.0, +0.8 and −0.4 m. The signs are mixed with no curve, and the largest residual is 0.8 m, compared with 2.6 m for the line. The sum of the squared residuals falls from 22.60 for the line to 1.60 for the quadratic function. The informal assessment: the quadratic function fits these data well, to within about a meter at every speed, and the line does not, because its residuals have a clear curved pattern.

The difference matters when the functions are used. At 50 km/h, inside the data, the line predicts 24.5 m and the quadratic function 22.5 m; the measurement was 21.9 m. At 100 km/h, an extrapolation for both, the line predicts 59.5 m and the quadratic function 70 m, a difference of 10.5 m in a prediction about braking. At 0 km/h the line predicts −10.5 m, and it predicts a stopping distance of zero at 15 km/h; the quadratic function predicts 0 m at 0 km/h.

Four cautions, each with its condition. (1) A sum of zero proves nothing. For a least-squares line, and for any least-squares polynomial fit that includes a constant term, the residuals always add to zero apart from rounding. Check the line: 2.3 + 0.6 − 1.7 − 2.6 − 1.5 + 0.8 + 2.1 = 0. A badly chosen form still has residuals that cancel, so the sum cannot be the test; the pattern and the sizes are. (The property needs both conditions: a least-squares fit made on y itself, and a family with an added constant term. A line drawn by eye, a given function, a least-squares line forced through the origin, or an exponential function a·bˣ, which has no added constant term, need not have residuals that add to zero.) (2) A small residual at one point proves nothing. At 30 km/h the line's residual is +0.6 m, exactly the same as the quadratic function's. Judged at that one speed the two functions look equally good. Any line laid across a curve is close to the curve near the places where they cross. Fit is a property of all the residuals together. (3) A high correlation does not show that a line is the right form. The correlation coefficient measures how close the points are to a straight line; here r = 0.992, and the residual plot still shows a curve. A value of r close to 1 or −1 says that a line captures most of the variation, not that nothing systematic is left. The reverse also holds: a residual plot with no pattern but large residuals means the form is reasonable while individual predictions are imprecise. (4) A family that contains another always comes at least as close. A least-squares quadratic function can never have a larger sum of squared residuals than the least-squares line for the same data, because the line is the special case a = 0. So a smaller sum of squares alone does not justify the quadratic function. Ask three things: did the pattern disappear, is the improvement large compared with the residuals themselves, and does the context support the form? Here all three answers are yes.

Other patterns and what they mean. Residuals that fan out, small on one side of the plot and large on the other, mean the function's predictions are less precise where the spread is wide, even if the form is right. A single residual much larger than the rest marks a possible outlier: check the record, and remember that one extreme point can pull a fitted function toward itself. Residuals that are all positive or all negative mean the graph sits wholly below or above the points, as with a line drawn by eye too low. Residuals that climb steadily from negative to positive mean the data rise faster, or fall more slowly, than a given or hand-drawn line: its slope, as a signed number, is too small.

What "informally" means. No formal test or cutoff is used. The assessment is a short argument in words: describe the signs and pattern of the residuals, state their typical and largest sizes in the units of y, compare those sizes with what the context needs, and say what follows for using the function.

Worked residuals for a good linear fit: The hypothetical candle from the lesson on fitting functions to data had heights 24.2, 21.2, 19.1, 16.3, 14.3, 11.6, 8.6 and 6.7 cm after 0 to 7 hours, and the least-squares line h = −2.5t + 24. The predicted heights are 24.0, 21.5, 19.0, 16.5, 14.0, 11.5, 9.0 and 6.5 cm, so the residuals are +0.2, −0.3, +0.1, −0.2, +0.3, +0.1, −0.4 and +0.2 cm. On a residual plot the signs alternate irregularly across the whole time axis: no curve, no trend, no widening. The largest residual is 0.4 cm on heights between 6.7 and 24.2 cm. Informal assessment: a linear function fits these data well, and predictions inside 0 to 7 hours should be good to about half a centimeter. The residuals add to 0, as they must for a least-squares line.

Worked comparison of a line and an exponential function: The hypothetical bouncing ball had heights 100, 78, 64, 51, 40, 33 and 26 cm on bounces 0 to 6. The least-squares line is h = −12n + 92, with r = −0.980. Its predictions are 92, 80, 68, 56, 44, 32 and 20 cm, so its residuals are +8, −2, −4, −5, −4, +1 and +6 cm: positive at both ends, negative in the middle, the same U shape as the stopping data, with a largest residual of 8 cm. The exponential function h = 99.1(0.800)ⁿ, fitted through logarithms, predicts 99.1, 79.3, 63.4, 50.7, 40.6, 32.5 and 26.0 cm, giving residuals +0.9, −1.3, +0.6, +0.3, −0.6, +0.5 and 0.0 cm: mixed signs, no curve, largest 1.3 cm. The exponential function fits and the line does not, although r = −0.980 sounds strong. Notice that the exponential function's residuals add to +0.4, not 0 (about +0.1 before the coefficients were rounded). Nothing is wrong. The zero-sum property belongs to least-squares fits on y for families with an added constant term, such as mx + b; the function a·bⁿ has no added constant term, and it was fitted on ln h besides, so the property does not apply to it. The consequences agree with the residuals: at bounce 8 the line predicts −12(8) + 92 = −4 cm, an impossible height.

Worked residuals from a line drawn by eye: A student draws a line through the candle scatter plot and reads off h = 23 − 2.5t. Its predictions are 23.0, 20.5, 18.0, 15.5, 13.0, 10.5, 8.0 and 5.5 cm, so the residuals are +1.2, +0.7, +1.1, +0.8, +1.3, +1.1, +0.6 and +1.2 cm. Every residual is positive and they add to 8.0, a mean of +1.0 cm: the line has the right slope but sits about 1 cm below the points. Raising the intercept by 1.0 gives h = 24 − 2.5t, the least-squares line. Residuals that are all of one sign are a pattern too, and they say which way to move the graph.

Worked fan pattern: A hypothetical delivery service is given the function T = 5 + 3d for the time in minutes of a delivery of d kilometers and checks it against eight deliveries: (2, 11.5), (4, 16), (6, 24.5), (8, 26), (10, 39), (12, 35), (14, 53), (16, 46). The predictions are 11, 17, 23, 29, 35, 41, 47 and 53 minutes, so the residuals are +0.5, −1, +1.5, −3, +4, −6, +6 and −7 minutes. The signs alternate, so there is no curve and a linear form is reasonable (the residuals add to −5 minutes, so the given function runs a little high on average). But the sizes grow steadily with distance, from about 1 minute for the shortest trips to 6 or 7 minutes for the longest: a fan. Informal assessment: the given function is a fair description of the average trend, its predictions are close for short deliveries, and for long deliveries it can be off by several minutes either way, so a promised arrival time for a 16 km delivery needs a wider margin than one for a 2 km delivery.

Represent and compare: For the seven stopping-distance pairs, make a table with five columns: speed, observed distance, the line's prediction, the line's residual, and the quadratic function's residual. Then draw two residual plots on the same scales, one for d = 0.7v − 10.5 and one for d = 0.005v² + 0.2v, each with a horizontal line at zero and the vertical axis running from −3 to 3 m. Under each plot write the sum of the residuals, the largest residual in size, and one sentence describing the pattern or the lack of one.

Independent performance: Use a data set of at least eight pairs of two quantitative variables, collected by you or supplied by your teacher, whose scatter plot shows a clear relationship. (1) Draw the scatter plot and describe how the variables are related. (2) Fit two functions from different families (for example a linear function and an exponential or quadratic function) with technology, and name the method. (3) Compute every residual for both functions, showing observed − predicted for at least three points, and draw both residual plots. (4) Write an informal assessment of each fit that discusses the residuals: their signs and pattern, their largest and typical sizes with units, and whether those sizes are acceptable in the context. (5) State which function you would use to predict, give one prediction, and explain why the correlation coefficient or one small residual could not have settled the choice. Success means every residual has the correct sign, both plots have a zero line and labeled axes, the assessment names a pattern or says there is none and supports that with the signs, and the final choice rests on the residuals and the context together.