7 Responses to “Gallup: Obama Up By 3”

1. ExiledIndependent Says:
   August 4th, 2008 at 11:46 am

   Given the radical inaccuracy of polls like this during the primaries, I’m a little surprised that they’re getting so much screen-time here on Donkle. My guess is that we’re going to get polling burnout sometime in mid-September.

2. kranky kritter Says:
   August 4th, 2008 at 1:43 pm

   Justin is tone deaf on this point, exiled.

   And on top of that. I guess he wants us to believe that these polls have margins of error that are less than 3 or something. For the uninformed, here’s how it’s supposed to work. If a poll reports A:52-B: 48 with a margin of error of +/- 3 points, that means that A has somewhere in the neighborhood of 49 (52 minus 3) to 55 (52 plus 3), and B has 45-51%.

   So A 52%, B 48% with +/-3 includes the possibility that B is ahead 51-49, as well as all the other iterations implied by the error margin. Which suggests that if its close, a single poll doesn’t tell you very much. However, since neither the pollsters nor the poll financiers nor the media has any motive to clarify this point, folks pretend that static back and forth along a regular baseline is trend, and not static. Sort of like the face of Jesus on a grilled cheese sandwich.

3. Justin Gardner Says:
   August 4th, 2008 at 1:49 pm

   Radical inaccuracy? Tone deaf? Uninformed?

   From Gallup:

   The general-election results are based on combined data from August 1-3, 2008. For results based on this sample of 2,659 registered voters, the maximum margin of sampling error is ±2 percentage points.

   Moreover, I do the Gallup and Rasmussen polls on a DAILY basis so I don’t just highlight them when the margins are bigger. This gives the readers of this blog a true sense of the ebbs and flows of these dailies, which have been remarkably stable over the past couple months.

   In any event, before you go accusing me on not doing my homework, being tone deaf, etc., check your facts. There’s nothing that makes me angrier than being accused of being misinformed when the exact opposite is true.

4. ExiledIndependent Says:
   August 4th, 2008 at 9:14 pm

   Sorry Justin, wasn’t directing this at your or the data that you’re reporting.

   I’m referring to the fact that during the primaries, the polling predictions were often wildly different from actual human behavior inside the voting station. What people say they’re going to do and what they actually do are two different things. So, while it’s nice for regular site content, I personally don’t put a huge amount of stock in the polls (again, not disparaging the data or the polls you’re using; they’re just sort of non-news from my point of view).

5. kranky kritter Says:
   August 5th, 2008 at 12:30 pm

   Two related points Justin, at least one of which you seem intent on missing…

   •More often than not, that which you characterize as “ebbs and flows” are better interpreted as static. If the underlying state is stable, but is measured with an instrument with an error rate, then the appearance is of ebbs and flows. Small ebbs and flows are static. No one who wants to understand data, its measurement, and the difference between signal and noise should ever dismiss this.

   Suppose you had 2 beakers of water that were at constant temperatures of 51 degrees and 49 degrees, and you measured them every day with a thermometer which was accurate only within +/- 2 degrees, for an error range of 4 degrees. Daily sampling using the crappy thermometers would suggest an ebb and flow, and sometimes the results would show the cooler water to be warmer by 51-49.

   • You need to get this. If the margin of error is +/-2 and you are ahead by 3 points, you might be behind by one. If the margin of error is +/-2, you need to be up by more than 4 to be outside the margin of error.

6. Justin Gardner Says:
   August 5th, 2008 at 1:28 pm

   Actually, we’re both wrong.

   First, from Harris Interactive…

   This number is actually a purely theoretical calculation of what the likely maximum error (at a 95% confidence level) would be if the survey had used a pure probability sample with a response rate of 100% and there were no other possible sources of error. In the real world of polling there are several other sources of error that may sometimes be larger than this theoretical calculation of sampling error, and there is no good way to calculate them. However, a new Harris Poll shows that most people do not understand this.

   Essentially, MOE is used to determine how many times out of 100 any given poll would show a variance from the numbers presented. So the number “2″ in this poll is not tied to the actual poll percentages themselves.

   Kevin Drum has some more about this…

   So in the poll quoted above, how probable is it that Kerry is really ahead? The MOE of the poll is 3% [...] Kerry’s lead is 2%, which means there’s a 75% probability that he’s genuinely ahead of Bush (i.e., that his lead in the poll isn’t just due to sampling error).

   Generally speaking, national polls use sample sizes of about 1,100, which translates to an MOE of 3%. State polls often use a sample of 600, which produces an MOE of 4%. Subsets of polls sometimes have MOEs of 5% or higher.

   So if Obama is ahead of McCain by 4, and there’s a +/-2 MOE, then, according to the table at Kevin Drum’s site, there would be AT LEAST a 92% chance that these numbers are right since that’s the % they’re using for a +/-3 MOE.

   Lesson learned, agreed?

7. kritter Says:
   August 5th, 2008 at 7:57 pm

   Right. The notion of MOE doesn’t really assign an even probability across the range, it expresses more confidence in numbers close to the stated one and less confidence in others. I counted that as too complicated to explain.

   Because confidence in results is expressed as a probability, there’s really no point where, mathematically speaking, the results are perceived as incontrovertibly accurate. So even if the poll had Obama ahead by 30 points, there’d be a tiny probability (again, mathematically speaking) that it was wrong.

   My point stands about margin of error. You’re on thin ice calling a 3 point lead “outside the margin of error” when the MOE is +/-2 points. If you want to talk about statistical significance, then be aware as Kevin states that the general minimum standard is that there’s less than a 1 in 20 chance that the results were due solely to chance. That corresponds to 95% confidence. As Kevin also notes, there’s no real magic to 95%, it was a number pulled out a rectum at some point.

   I’ll go ahead and endorse Drum’s notion of translating the poll results into the form of “XX% probability that candidate A is ahead by some undetermined nonzero amount.” But be aware that something like, say, a 75% probability isn’t that high. It corresponds to being wrong 1 time out of 4. So if 4 polls in a row expresseda 75% probability that Obama was ahead by some nonzero amount, odds are that one of them was wrong….

Leave a Reply

Related Posts:

• Gallup: Obama Leads By 3
• Gallup: Obama Up By 2
• Gallup: Obama Up By 3