Large polling samples
cut the ubiquitous
error margin

Question: Polls on various subjects seem to appear frequently now. Can you please clarify just what is meant by "margin of error" and how the pollsters arrive at the number?

Answer: We asked Jim Dannemiller, president of SMS Research in Honolulu, to explain the process of deriving the margin of error -- in layman's terms.

SMS Research has conducted recent polls for the Star-Bulletin, in which the margin of error was plus or minus 3.7 percentage points.

There is a certain amount of error in every sample survey, simply because you're asking just some of the people. But, over the years, statisticians have figured out "how that error is generated and how it acts and how it affects the results of a survey," Dannemiller said.

In short, they've figured out the formulas to estimate the general amount of error in any given survey. "Mostly those formulas show us that the error depends on the actual sample size, regardless of the size of the population," he said.

The basic formula is (square root (.25/number of respondents)) X 1.96, or divide .25 by the number of respondents. Take the square root of that and multiply by 1.96.

Dannemiller said he often uses 196 instead of 1.96 so he doesn't have to move the decimal point to get percentage points. His firm also generally uses a "finite population correction" because the number of households in the state is known, he said.

That's done by "multiplying by 1 minus the sample fraction, before taking the square root" or 196 X (square root ((.25/n) X (1-(n/N))).

Dannemiller also explained there is an "industry standard" used in establishing a poll's accuracy, which is plus or minus 5 percentage points at the 95 percent confidence level (95 percent of the time, it is expected to be correct).

At that standard, a 55 percent-45 percent split between two candidates is "as good as dead even," he said. "So we have to go beyond plus or minus 5 before we can call it one way or the other."

Taking a larger sample size would bring the margin of error down. If it got down to plus or minus 3 percentage points, "That would mean at 55-45, the guy at 55 (percent) is going to win," Dannemiller said.

In general:

» The bigger the sample size, "it gets better."

» The bigger the percentage difference, the better. "So, at 50-50, my uncertainty is going to be high, but at 95-5, I'm not absolutely certain, but I know who's going to win. We don't have to refer to the formula," Dannemiller said.

» The actual size of the population being surveyed matters in determining the sample size.

If you're surveying registered voters and the number on Oahu is about 430,000, "I only need 400 cases to get my plus or minus," Dannemiller said. But if the survey is only about the 20,000 or so students at the University of Hawaii, "then obviously, you don't need 400."

It also depends on "how accurate you really want to be," he said. "You could raise the confidence level to the 99 percentile to be 99 percent sure, in which case the sample size will have to be much larger in order to get that confidence level."

|