I was listening to Larry Kudlow on the radio Saturday when he disclosed a nugget of information that usually indicates when Democrats are secretly manipulating public opinion. Larry was in the Office of Management and Budget while I was there — he as a leader in the OMB hierarchy and me as the senior mathematical statistician (a GS-15 working grunt).
Larry was brilliant in almost every area he touched, but like many economists, most political consultants and almost all judges, he was an amateur statistician. He loved to poke at the details of statistical data (an excellent instinct), but he did not recognize the red flags that the story in the numbers may be false.
Pollsters have to sell their services like any other consultant. They offer polling methodologies that are reliable but expensive, but their clients often opt for cheaper methods that “approximate” the results of more reliable procedures. The problem is that these cheaper methods can be easily manipulated, not necessarily by the takers of the opinion polls, but by the respondents themselves. The cheap alternatives use some variation of QUOTA sampling.
This how it works. The pollster sells the idea that his process has an excellent scheme for balancing the number of men/women, Democrats/Independents/Republicans, or other groups (hispanics, blacks, etc). This scheme involves “quotas.” Federal agencies are not permitted to collect data this way. For years the Democrats have known how to bias this opinion polling system.
When you get a polling call, you give your normal liberal opinions until it comes time to identify your affiliation — then you lie. This accomplishes two things: it gets your liberal views into the poll as an independent or a republican; and it uses up a slot in the quota for the other group and thus prevents an actual independent or republican from entering their true opinions in the poll. A small percentage of liars on the poll can have twice the effect on steering the results their way.
Democrats have been doing this to independents for years. Have you noticed how the left seems to “win over” the independents as the campaign moves along. FRAUD! Fraud generated by Dems posing as independents — It did not happen but Republicans panic and modify their game to win back independents.
Now the left is becoming more brazen. They are inventing a small body of “republicans for Hillary.” It doesn’t take many to pollute the Republican and/or Independent quota in the polls with clever liars and generate doubts about their level of support. Worse yet the apparent trends tend to spread panic among real voters who don’t realize the fraud being spread by the left. This how Cuccinnelli and company lost — the fraudulent polls led to criticism of the campaign and grassroots republicans were convinced they couldn’t win — and stayed home! All the while the Republicans were close or AHEAD.
So what was the nugget Larry saw but did not recognize. He mentioned that Hillary’s “bounce” seemed to come not from Democrats but from “Republicans” in the polls who showed less support of Trump! Sound familiar?
Don’t be fooled by crossover fraud. If a poll is using quotas (most newspapers and consultants), subtract 5 or 10 points from Hillary and add them to Trump as a first order bias correction. And tell your Republican officeholders before they buy into the Democrats’ statistical fraud, get a real statistician on the staff and pay for reliable sampling methods.
Jerry L Coffey, PhD Mathematical Statistics
29 comments
These polls are definitely skewed. Just ask President Romney.
This author is very correct poll construction and the methodologies that lie behind them today which the typical individual quotes are a black box to them and most others. The science of political candidate polling has unfortunately turned into the art of poll messaging it’s end not to inform but rather to generate that 60 second media message that is repeated as “fact” far and wide for some specific partisan end. But you don’t need to be a mathematician to flag the spurious and questionable polling output because most polls and especially media based and those associated with partisan think-tanks or PAC derived groups do little to cover their tracks.
The one question even the most novice poll reader should always ask on any poll result is what were the sample weightings used by the poll and did they publish those with the output or make them readily available for review. If no published poll weightings are included beware, if the D/R ratio is greater then 5-8 points beware (the largest in modern Democratic history the Obama presidential turnout generated only a plus 6-7 point weighting), if the independent response is very heavily skewed to one party or the other beware. There have been some ABC/NBC/CNN and Wall Street Journal sponsored polls this presidential cycle that have been very, very questionably constructed (I have seen poll democratic weightings over represented by as large as plus 20 points!). It’s a shame but appears to be becoming a more common occurrence in our hyper partisan politics. This isn’t about who will win or who will lose but just who is lying to your face with this garbage presented as some type of fact.
Good points. I just posted a comment on the lop-sided CBS News samples that Jeanine wrote about earlier. You have to look at the details behind the results. And the one fact they rarely admit is that perhaps it took over 2500 calls to get 1000 response. My academic friends tell me this is about the norm. The standard we imposed on most Federal agencies was a minimum of 70% response — below that you can’t get reliable variance calculations. Public Opinion polling rarely approaches this level (which means those +or- error claims are nonsense).
or maybe, just maybe, more people in this country support Hillary Clinton over Donald Trump right now.
I wouldn’t be surprised. Half our population doesn’t pay taxes, so they don’t care if they go up or down….the Clinton machine is working hard with lots of money to make this happen for her – at any cost.
Let me tell you why this is garbage (aside from the “it’s all a conspiracy!” angle being garbage)…
Pollsters (at least the ones who don’t cut corners) largely now have access to voter and commercial data. If Bob Smith, the 75 year old white male from Chesapeake with a GOP primary voting history and frequent purchases of sporting goods & catheter supplies on his credit card claims to be a 20 year old Chinese woman from Argentina, they know that they’ve either got the wrong person on the phone or the guy is full of shit. The reason they ask the matching questions isn’t to blindly trust the answers, but to verify that they have the correct person on the phone and that they’re being cooperative with the survey.
Remember that the whole purpose of the quota methodology is to cut costs — it always cuts corners. If you want to use secondary information the costs go up. This is the deadly circle — if you are going to spend gobs of time and money to “correct” the flaws in the quota scheme, you may as well spend the dollars on the more reliable methodology and avoid the risks of errors in “correction”.
Why are the candidates internal polls so accurate?
Because they are not about ratings and the cabbage.
Internal polls are more carefully done — more attention to what they seek to learn and better targeting of the sample of respondents. They sometimes fail to achieve results because the wrong issue or target sample is selected, but they are generally much better executed than the so-called “tracking polls.” The latter often use the same unreliable
methodology each time on the theory that the bias is the same from one iteration to the next — so the differences are “unbiased.” This is generally BS (Bad Science). The bias can and does change for many reasons — for example everyone “knows” there is supposed to be a bump or bounce in opinion surveys during and immediately after the party convention, and this expectation usually alters respondent behavior.
The response to the Democrat Convention was flat — as one
wag put it “Hillary made history by being the only nominee who has never gotten a bounce from a convention.” At that point the media and the Dem’s army of techies panicked. The oft cited Democrat “tech” advantage frequently manifests itself as the means for lying more effectively than the competition. The media drumbeat for Hillary went into overdrive, and with agonizing slowness, the needle began to move — not with more support for Hillary but with “Republicans” turning away from Trump. So what do you think — was this just Republican stupidity, or an information jihad by the radical Democrat
fringe? (Or possibly BOTH — )
Same thing happened to Romney all the polls said he would lose and then… Oh wait, this isn’t helping.
My doctoral,dissertation involved a questionnaire. To get advice, I visited Gallup and the census bureau for perspectives on questionnaire design. If you think asking a simple question is easy, I suggest you visit either one of them.
For example, I was shown samples of a supposedly simple questions and the associated results. Even simple sentences without adjectives but structured with different verbs could elicit wildly different answers. I remember one question that was about 8 words long that had something like seven pages of differing answers just because the 8 words were placed differently in the sentence.
Not only are statistics capable of being biased but the questions and the order in which they are asked can bias the results. My questionnaire was not random questions but structured from the start – input, thruput, output and anything additionally unique. Even structuring in this manner could create bias and I had to explain the nature of that bias in the dissertation.
So 35 years ago, an economist discovered a secret tactic used by the Democrats — that they orchestrate nationwide dishonesty from our neighbors when pollsters contact them.
And he didn’t tell the RNC.
Or he told the RNC, but those people, the kind we just saw manhandling the convention, equals to the Democrats in ruthlessness but opposites in political philosophy, opted not to expose OR imitate the fraud-committing Dems.
And in his following career as a journalist and commentator, he never unveiled this accusation with credibility.
Do I have that right?
The real problem is that everyone with an IQ above room temperature thinks he can design a survey and wont admit that his methods are vulnerable.
Your article wasn’t a commentary on survey methodology, it was a description of a Democratic party conspiracy to commit crossover fraud. But you don’t explain why Republicans don’t employ the same tactic, or Larry Kudlow was unable to convince others of the existence of systematic crossover fraud.
Shades of the grassy knoll.
Wishful thinking, Dr. Coffey. If my wide circle of Republican friends and activists, as well as independents, are any indication, Trump has been quite successful in convincing the public that he is not a good candidate for president. The only question remaining is whether Trump loses by single or double digits. And whether his poor showing kills our senate majority.
Dr. Coffey, interesting analysis. We would like to get you on our Virginia radio show to elaborate tomorrow. Can you email us your contact information to: John@JFradioshow.com. Thank you.
Sorry I did not get on to see your message earlier. I think I met you at a convention. I will send an email and perhaps we can do something later.
Are you saying that pollsters can actually identify who will lie to them, and build them into the polling data?
Actually, the pollsters know that a certain portion of the sample will lie and if they are any good at it they put in place compensation mechanisms. This, of course, requires an assumption that there is some semblance of a recognizable paradigm and not a paradigm shit which blows a complete hole in the validity of any poll. Sometimes, past performance is not a predictor of future results.
Oh Wow. There is a large complex literature on dealing with false information in surveys. A lot of it is designed to reduce the incentive to lie. One technique used in long surveys is to ask similar questions in different parts of the survey. The respondent often forgets what he said the first time and there is a clue about errors or lying.
Are you convinced that more democrats use this trick than Republicans? It makes logical sense that a poll could be gamed that way if quotas are used as you describe but I have not seen a concerted effort to do that by any group(I am clueless about popular D blogs)
Are these quotas divulged in the poll data so we can identify which polls should generate more skepticism?
What percentage of polls (local and national) would you guess use this quota technique?
It doesn’t take a large number. Most democrats and republicans lie to pollsters in small ways. All the major polling firms offer a quota system as a low cost option. Since they can’t sell it to government agencies, there must be a lot of private demand out there. How many Democrats are adherents of Sol Alinsky — how many Republicans?
Oh No! Don’t bust Miss Jeanine’s bubble.
I didn’t write the post.
I am fully aware of that, I was directing my comment to the author.
When you make me laugh at 4:00 a.m. you’ve done something Ruby.
Why would this bother me? I posted the article for Dr. Coffey!
You really don’t get it?
Good one!