Buyer Beware: There Are Polls, and Then There Are Polls
Posted at 12:52 p.m. on Nov. 7, 2013
Given the “success” (note sarcasm) of some polls in the Virginia gubernatorial race Tuesday, it might be worthwhile to note the very divergent surveys in the Texas gubernatorial race.
Public Policy Polling, which showed Terry McAuliffe with a 7-point lead among likely voters right before the election, found Republican Attorney General Greg Abbott leading Democratic state Sen. Wendy Davis by 15 points in its November 1-4 survey of 500 “Texas voters,” 50 percent to 35 percent. PPP, a Democratic firm based in North Carolina, used automated telephone interviews in conducting the survey.
But a University of Texas/Texas Tribune poll of 1,200 registered voters, conducted October 18-27, found the margin a much more narrow 6 points, 40 percent to 34 percent. The University of Texas/Texas Tribune poll is conducted by YouGov, which employs a controversial methodology. The survey, conducted over the Internet, relies on “a proprietary opt-in survey panel” that is not selected randomly.
Widely respected Langer Research Associates issued a “briefing paper” on opt-in online surveys in August of last year noting that the American Association for Public Opinion Research has been quite critical of such surveys — even arguing that reporting a margin of error for an opt-in sample is “misleading”— and that a number of major media organizations “have adopted policies ruling out the reporting of opt-in online surveys as news, given the unreliability of the method.”
Wisely, in my view, Roll Call does not report opt-in online surveys, but Roll Call’s parent company, The Economist, partners with YouGov to produce The Economist/YouGov Poll, which also uses the opt-in survey panel Internet methodology.