Understanding Margin of Error
All data that are based on samples, such as the ACS and the census long-form samples include a range of uncertainty. Two broad types of error can occur: sampling error and non-sampling error. Non-sampling errors can result from mistakes in how the data are reported or coded, problems in the sampling frame or survey questionnaires, or problems related to non-response or interviewer bias. The Census Bureau tries to minimize non-sampling errors by using trained interviewers and by carefully reviewing the survey’s sampling methods, data processing techniques, and questionnaire design.
Sampling error occurs when data are based on a sample of a population rather than the full population. Sampling error is easier to measure than non-sampling error and can be used to assess the statistical reliability of survey data. For any given area, the larger the sample and the more months included in the data, the greater the confidence in the estimate. The Census Bureau reported the 90-percent confidence interval on all ACS estimates produced for 2005 and earlier. Since the release of the 2007 ACS data, margins of error have been provided for every ACS estimate. Ninety-percent confidence intervals define a range expected to contain the true value of an estimate with a level of confidence of 90 percent. Margins of error are easily converted into these confidence ranges.
For example, the 2007 ACS Data Profile for South Carolina, shown below, shows that 828,713 married-couple families resided in the state in 2007.
By adding and subtracting the margin of error from the point estimate, we can calculate the 90-percent confidence interval for that estimate:
828,713 – 11,898 = 816,815 = Lower-bound interval
828,713 + 11,898 = 840,611 = Upper-bound interval
Therefore, we can be 90 percent confident that the true number of married-couple families in South Carolina in 2007 falls somewhere between 816,815 and 840,611.
The margin of error around an estimate is important because it helps you draw conclusions about the data. Small differences between two estimates may not be statistically significant if the confidence intervals of those estimates overlap. However, the Census Bureau cautions data users not to rely on overlapping confidence intervals as a test for statistical significance, because this method will not always produce accurate results. Instead, the Census Bureau recommends conducting statistical significance tests.
In some cases, data users will need to construct custom ACS estimates by combining data across multiple geographic areas or population subgroups or it may be necessary to derive a new percentage, proportion, or ratio from published ACS data. In such cases, additional calculations are needed to produce confidence intervals and margins of error for the derived estimates. Note that these error measures do not tell us about the magnitude of non-sampling errors.