Response rates suggest the “death of telephone polling” is getting closer.
I honestly can’t recall the last time I took a call from a person I didn’t know or a number I didn’t recognize. So that means a pollster is never going to reach me. There are probably many other people like that. With a significant and increasing percentage of voters unreachable that way, polling is broken or at best barely functional.
Nate Cohn is a data journalist at the New York Times. He recently posted a letter from a reader who has increasing doubts about polling.
Given that pollsters are relying on calling people on the phone (per your methodology description at the bottom of the poll), how do you know where they are, and how do you account for the fact that so few people answer their phones at all anymore? I for one have moved twice since I got my current phone number, most recently to a different state, so my phone number has nothing to do with my actual location. Meanwhile, most of my calls are spam, so I almost never answer my phone unless I recognize the phone number — and I am someone who is old enough to have grown up with what is now called a landline. My teenage kids almost never answer their phones at all. The only people I know who still ever use a landline at all are my parents.
— Doug Berman, West Jordan, Utah
While Nate’s response was upbeat, his positive attitude was undercut by some of the observations in his reply.
How do you account for the fact that few people answer? Before I respond, I want to dwell on just how few people are answering. In the poll we have in the field right now, only 0.4 percent of dials have yielded a completed interview. If you were employed as one of our interviewers at a call center, you would have to dial numbers for two hours to get a single completed interview.
No, it wasn’t nearly this bad six, four or even two years ago. You can see for yourself that around 1.6 percent of dials yielded a completed interview in our 2018 polling.
[ ... ]
Call screening is definitely part of the problem, but if you screen your calls almost 100 percent of the time, it might be a little less of one than you might think. About one-fifth of our dials still contact a human. But once we do reach a person, we’ve got a number of challenges. Is this the right human? (We talk only to people named on the file, so that we can use their information.) If it is the right person, will he or she participate? Probably not, unfortunately.
This sounds rather dismal. Despite these major issues there are still a lot of people who are obsessive about polls – regardless of how inaccurate they increasingly turn out to be.
Last summer saw the abortion referendum in Kansas. The polling that was done for it suggested that it would be close. In fact, the vote to preserve abortion in Kansas won by about 18 points.
There were several elections this year to fill House vacancies. In almost every single one, Democrats did better than the polls suggested.
There are also occasions when polls underestimated the strength of the other side.
While Joe Biden won a clear majority of the popular vote, the margin was still smaller than what the polls predicted.
A day or two before the 2016 presidential election, FiveThirtyEight’s model (based to a significant degree on polls) gave Donald Trump just a 23% chance of winning the presidency. We know how that went.
Data journalists like Nate are actually quite good when they have useful data to work with. I usually go out of my way to read his articles. But phone polling has become increasingly problematic and no system to replace it has been entirely convincing.
Polls might still give a very general indication of the way things are heading, but campaign managers shouldn’t schedule victory celebrations based strictly on poll numbers.
As voters, we should turn out regardless of what the polls say. That’s more true than ever now. Polls don’t vote, people vote.
