Choose a level of accuracy appropriate to the limitations on measurement when reporting quantities

Learning goal: Choose a level of accuracy appropriate to limitations on measurement when reporting quantities, by reading the precision of a measuring instrument, tracking how uncertainty moves through a calculation, and rounding a reported result so that it claims no more than the measurements support.

Every measurement carries a limitation. A ruler marked in millimeters cannot tell you a length to the nearest hundredth of a millimeter. When a length is recorded as 12.4 cm on such a ruler, and we assume the ruler is accurate and was read correctly to the nearest millimeter, the honest meaning is that the true length is somewhere between 12.35 cm and 12.45 cm: the reading is correct to the nearest tenth of a centimeter, and nothing finer. The smallest step an instrument can show is its resolution, and it limits how finely a reading can honestly be stated, so a measurement should be written with exactly the digits that the instrument justifies. In this lesson, the precision a written value claims means the finest place value it states (12.4 cm claims tenths of a centimeter). In measurement science the word precision on its own means something different, how closely repeated measurements agree; the section on rounding below keeps the two apart. Writing 12.40 cm would silently claim a precision of a hundredth of a centimeter that the ruler does not have. Writing 12 cm would throw away information the ruler did give. The correct record is 12.4 cm, and its last digit is the uncertain one.

Significant figures as a bookkeeping device. The significant figures of a recorded measurement are the digits that carry information about its precision: all nonzero digits, zeros between them, and trailing zeros after a decimal point. 12.4 has three; 0.00450 has three (the leading zeros only place the decimal point, while the trailing zero says the measurement resolved hundred-thousandths); 1.200 × 10³ has four. A bare 1200 is ambiguous, which is one reason scientific notation exists: 1.2 × 10³ claims two figures and 1.200 × 10³ claims four. Counts are different from measurements. If 27 students are in a room, the 27 is exact, not a reading from an instrument, and it does not limit the precision of anything computed from it.

Uncertainty grows when measurements are combined. Suppose a rectangular card measures 12.4 cm by 7.8 cm, both on the millimeter ruler. The calculator gives an area of 12.4 × 7.8 = 96.72 cm². But the true length lies between 12.35 and 12.45, and the true width between 7.75 and 7.85. The smallest area consistent with the readings is 12.35 × 7.75 = 95.7125 cm², and the largest is 12.45 × 7.85 = 97.7325 cm². The area is therefore known only to be somewhere between about 95.7 and 97.7 cm². Reporting 96.72 cm² claims a precision of a hundredth of a square centimeter when the measurements support a range two square centimeters wide. The appropriate report is about 97 cm², two significant figures, matching the two significant figures of the less precise measurement, 7.8 cm. That is the reasoning behind the usual convention: a product or quotient should be reported with no more significant figures than the least precise factor, and a sum or difference should be reported to no more decimal places than the least precise addend. The conventions are shortcuts for the interval reasoning above; when the stakes are high, do the interval reasoning itself.

A worked model inside the lesson. A runner covers a course whose length was measured as 2.5 km (two significant figures) in a time of 11.0 minutes read from a stopwatch to the nearest tenth of a minute (three significant figures). The calculator quotient is 2.5 ÷ 11.0 = 0.22727… km per minute. Multiplying by 60 to get kilometers per hour gives 13.636… km/h. The factor 60 is an exact conversion, not a measurement, so it does not reduce the precision. The least precise measurement is 2.5 km with two significant figures, so the speed should be reported as about 14 km/h, or 0.23 km per minute. Reporting 13.64 km/h would claim four figures from a distance known only to two.

Rounding is not the same as low quality. Students sometimes feel that writing more decimals is "more accurate". It is the opposite: a result written with more digits than the data justify is less honest, because a reader will take those digits as information. Three words need to be kept apart. Accuracy is how close a measurement is to the true value. Precision is how closely repeated measurements of the same thing agree with one another. Resolution is the smallest step the instrument can show. A measuring process can be precise but inaccurate: a mis-calibrated scale that reads 2.437 kg, 2.436 kg and 2.438 kg on three weighings of a bag whose true mass is 2.6 kg agrees with itself to about a gram, yet every reading is about 0.16 kg too low. It can be accurate but imprecise: a bathroom scale that reads 2 kg, 4 kg and 3 kg for a 3 kg object scatters widely, although its readings center on the true value. A single reading shown to the gram tells you the scale's resolution, not its precision; only repeated readings show precision. Choosing a level of accuracy means matching the digits you report to what the instrument and the calculation actually support. Extra digits are not dishonest when the uncertainty is stated beside them: 96.7 ± 1.0 cm² is an honest report. What misleads a reader is extra digits with no stated uncertainty, and the significant-figure convention in this lesson is a way of reporting without writing the ± part.

Unit conversions do not add precision. A fence measured as 100 feet to the nearest foot converts to 100 × 0.3048 = 30.48 m, because 1 foot is exactly 0.3048 m. The conversion factor is exact, but the original reading was uncertain by half a foot, about 0.15 m. The significant-figure convention therefore reports 30.5 m, keeping the three significant figures of 100 feet measured to the nearest foot, not 30.48 m. The convention is only a rough guide here: 30.5 m suggests an uncertainty of about 0.05 m, but the interval, about 30.33 m to 30.63 m, shows that even the tenths digit is uncertain, so when it matters, state the uncertainty: 30.48 ± 0.15 m. The extra digits came from the conversion, not from the tape measure.

When to keep extra digits. Round only at the end of a calculation. If you round 0.22727 to 0.23 and then multiply by 60 you get 13.8, which drifts from the 13.6 the full value gives. Carry the calculator's digits through intermediate steps and apply the appropriate level of accuracy once, when the quantity is reported. Also keep in mind the purpose of the report: a carpenter cutting shelving wants the nearest millimeter; a person estimating paint for a room wants the nearest square meter. The limitation on measurement sets the maximum precision you may claim; the context may justify claiming even less.

The condition on all of this. These rules apply to measured quantities. Defined constants, counts and exact conversion factors have unlimited precision and never reduce the significant figures of a result. And the rules are conventions that approximate the honest interval; when two measurements have very different uncertainties, or when a result is a small difference of two large measured numbers, the interval reasoning is the reliable guide and the shortcut can mislead.

Worked circumference and area from one measured diameter: A circular plate has a diameter measured as 8.3 cm with a millimeter ruler (two significant figures; true diameter between 8.25 and 8.35 cm). Circumference: π × 8.3 = 26.075… cm. Report about 26 cm. Area: π × (4.15)² = 54.106… cm². Report about 54 cm². The constant π is known to as many digits as wanted and does not limit precision; the two-figure diameter does. Interval check for the circumference: π × 8.25 ≈ 25.92 and π × 8.35 ≈ 26.23, so the true value lies between 25.9 and 26.3, and 26 cm is the honest report.

Worked fuel range: A car's tank holds 13.2 gallons (measured at the pump, three significant figures) and the car's fuel economy was measured as 25.1 miles per gallon (three significant figures). The range is 13.2 × 25.1 = 331.32 miles. Both measurements have three significant figures, so report about 331 miles. The interval reasoning agrees: 13.15 × 25.05 ≈ 329.4 and 13.25 × 25.15 ≈ 333.2, so the true range is within a few miles of 331, and the hundredths in 331.32 are meaningless.

Worked average of masses: Three bags are weighed on a scale that reads to the nearest 0.1 kg: 2.3 kg, 2.5 kg and 2.4 kg. Their mean is (2.3 + 2.5 + 2.4) ÷ 3 = 7.2 ÷ 3 = 2.4 kg. The 3 is a count and exact. By the addition rule the sum 7.2 keeps one decimal place, matching the readings, so it has two significant figures. By the quotient rule an exact count does not limit the significant figures, so the mean keeps two: 2.4 kg, not 2.4000 kg. The calculator shows 2.4 here, but if the readings had been 2.3, 2.5 and 2.5 the calculator would show 2.4333…, and the appropriate report would still be 2.4 kg.

Worked conversion that seems to add digits: A garden is measured with a tape as 100 feet, to the nearest foot. Converting with the exact factor 0.3048 m per foot gives 30.48 m. Because the original reading is uncertain by about half a foot, which is 0.5 × 0.3048 ≈ 0.15 m, the converted length is uncertain by about 0.15 m, and reporting 30.48 m claims a precision of a centimeter that the tape measurement never had. The significant-figure convention reports 30.5 m, keeping the three significant figures of 100 feet measured to the nearest foot. That report still understates the uncertainty (30.5 m suggests about ±0.05 m, while the real uncertainty is about ±0.15 m), which is why 30.48 ± 0.15 m is the more informative report when the uncertainty matters.

Instrument-and-precision record: Make a table with columns instrument, smallest division, a sample reading, the interval the reading really means, and the number of significant figures. Fill it for a millimeter ruler, a kitchen scale reading to 1 gram, a stopwatch reading to 0.01 s, a measuring cup marked every 50 mL, and a car odometer reading to 0.1 mile. For the stopwatch row, add a note explaining why a hand-operated stopwatch reading of 12.83 s might justify only 12.8 s if the person pressing the button adds up to a couple of tenths of a second of uncertainty (treat that figure as a hypothetical to be tested, not a fact).

Independent performance: Measure a real rectangular object with a ruler and record its length and width with the precision the ruler supports. Compute its area and perimeter with a calculator, then do the interval reasoning: find the smallest and largest area consistent with your readings. Report the area and perimeter with an appropriate level of accuracy and explain, in two or three sentences each, why you chose that number of significant figures. Then convert the perimeter to a different unit using an exact conversion factor and show that the conversion does not increase the justified precision. Success criteria: the recorded readings show exactly the digits the ruler supports, the interval calculation brackets the calculator value, the reported figures claim no more precision than the least precise measurement, and the explanation distinguishes the measured quantities from the exact count and conversion factors.