With the pending USA election, the news is awash with poll results showing a candidate’s preference by voters. And when these results are presented, there is usually a caveat that “however, these results are within the margin of error” which actually makes the results a bit less conclusive. Why is that?
Without going through the plethora of maths that arrives at what follows, let me explain.
If we are trying to determine a parameter of a population (like the percentage of people that prefer a candidate), we need to ask everyone in a population in order to know the answer exactly. This is impossible in many situations, especially in the USA where all the people who will vote cannot be asked the question. So a sample of voters must be used. Now there are a lot of things to be considered to make sure that the sample used is truly random (that is, not biased), but let’s assume going forward, that the samples used are random.
First, without the math, for large samples, the distribution of the parameter being measured is approximately normal. This is fancy statistical wording that means the values one gets taking sample after sample will follow a bell curve:
This curve is adjusted so that the probability of the parameter of interest between two values is the area under the curve. This means that the area under the entire curve must be 100%. So the probability that the parameter is between a and b, based on the sample, is the shaded area below:
If this is 68% of the total area, then that is the probability that the parameter being measured is between a and b.
Now let’s get to the current scenario. Suppose 1000 people are surveyed and 52% prefer candidate A and 48% prefer candidate B. Let’s look at the associated bell curve for candidate A:
A lot of math is involved here and a lot of assumptions (though they are reasonable). Notice that the curve is centered at the sample result of 52% (0.52 is the decimal equivalent). The range 0.49 to 0. 55, which is 0.52 – 0.03 and 0.52 + 0.03, are the numbers that include 95% of the area (that is probability). Without going through a lot of theory here, this range of numbers is the 95% confidence interval for this sample. So a statistician can say “based on this sample, I am 95% confident that the true percentage of all voters who support candidate A is between 49% and 55%”. This means that based on this sample, the true preference for candidate A can be as low as 49%. The number 0.03 which is added and subtracted from the sample result is called the margin of error. This 95% confidence interval is the most common one used.
Now let’s look at the bell curve for candidate B and its associated confidence interval:
Notice that the 95% confidence interval for this result is 45% to 51%. That is , based on this sample, we can be 95% confident that the true percentage of all voters who support candidate B is between 45% and 51%. This means that the true preference for candidate B can be as high as 51%. And this is higher than the possible low preference of candidate A at 49%. That means that even though the sample shows that candidate A is preferred, the difference between the two values are not significant enough to make the statement that candidate A is truly preferred at the 95% confidence interval. In other words, the result is within the margin of error.
Now let’s say that the survey result was that candidate A was preferred at 55% and candidate B at 45%. The confidence interval of candidate A’s bell curve would be 52% to 58%. Candidate B’s confidence interval would be 42% to 48%. So based on this sample, the highest that the true preference of candidate B is 48% and the lowest preference of candidate A would be 52%. There is no overlap here so this would be significant enough to say that all voters do prefer candidate A. That is, the result is outside the margin of error. And when you see results with that statement, that is much better for candidate A.