Reached a point in Nate Silver's new book, On the Edge where he talks about what ChatGPT told him:
"More than forty years I've been doing innovation. I can't think of a single example of a large innovation that came from an expected player or a large player," Vinod Khosla told me. Taken literally, this is an exaggeration - a ChatGPT query turned up counterexamples of products like the Sony Walkman, the IBM PC, and the iPhone that were developed by well-established brands.
Nate. Come on man. Why are you asking ChatGPT this? Why is this a ChatGPT question? Why are you telling me that you're getting answers from ChatGPT? Why are you writing in a book that this is where you're getting your information from?
I don't think there's a problem with asking ChatGPT stuff per se. It's like a worse, more expensive Wikipedia that lies to you sometimes. But you then have to go actually think about the answers and whether they fit the question, and do research to see whether those answers actually comport with reality, and be ready to say "well, the machine was full of shit again", which is often the case.
I guess I'm just baffled by referencing ChatGPT as a method of investigation, like its an admission that Nate Silver would rather go to the machine than sit and think to himself about the issue for five minutes, or go to a contrary source to get their quote. I don't necessarily think that he should have lied about where he got that list from, but ... I don't know. It felt really lazy. I think if you're writing a pop science book, I want more from it than "I asked ChatGPT and here's what it told me".
(The book is full of interview snippets and anecdotes and halfway through, this is the first time that ChatGPT has shown up, but it does raise my skepticism levels of everything that's come before.)
In Canto 20 of Inferno, Dante confronts a pit where the sinners have had their heads twisted around backwards; they trudge, naked and weeping, through puddles of cooling tears. Virgil informs him that these are the fortunetellers, who tried to look forwards in life and now must look backwards forever.
In a completely unrelated subject, how about those election pollsters, huh?
Writing for The American Prospect, historian Rick Perlstein takes a hard look at characteristic failure modes of election polling and ponders their meaning:
Apart from the pre-election polling chaos we're living through today, Perlstein's main inspiration is W Joseph Campbell 2024 University of California Press book, Lost in a Gallup: Polling Failure in US Presidential Elections:
In Campbell's telling, US election polling follows a century-old pattern: pollsters discover a new technique that works spookily well..for a while. While the new polling technique works, the pollster is hailed a supernaturally insightful fortune-teller.
In 1932, the Raleigh News and Observer was so impressed with polling by The Literary Digest that they proposed replacing elections with Digest's poll. The Digest's innovation was sending out 20,000,000 postcards advertising subscriptions and asking about presidential preferences. This worked perfectly for three elections – 1924, 1928, and 1932. But in 1936, the Digest blew it, calling the election for Alf Landon over FDR.
The Digest was dethroned, and new soothsayers were appointed: George Gallup, Elmo Roper and Archibald Crossler, who replaced the Digest's high-volume polling with a new kind of poll, one that sought out a representative slice of the population (as Perlstein says, this seems "so obvious in retrospect, you wonder how nobody thought of it before").
Representative polling worked so well that, three elections later, the pollsters declared that they could predict the election so well from early on that there was no reason to keep polling voters. They'd just declare the winner after the early polls were in and take the rest of the election off.
That was in 1948 – you know, 1948, the "Dewey Defeats Truman" election?
If this sounds familiar, perhaps you – like Perlstein – are reminded of the 2016 election, where Fivethirtyeight and Nate Silver called the election for Hillary Clinton, and we took them at their word because they'd developed a new, incredibly accurate polling technique that had aced the previous two elections.
Silver's innovation? Aggregating state polls, weighting them by accuracy, and then producing a kind of meta-poll that combined their conclusions.
When Silver's prophecy failed in 2016, he offered the same excuse that Gallup gave in 1948: when voters are truly undecided, you can't predict how they'll vote, because they don't know how they'll vote.
Which, you know, okay, sure, that's right. But if you know that the election can't be called, if you know that undecided voters are feeding noise into the system whenever you poll them, then why report the polls at all? If all the polling fluctuation is undecided voters flopping around, not making up their mind, then the fact that candidate X is up 5 points with undecided means nothing.
As the finance industry disclaimer has it, "past performance is no guarantee of future results." But, as Perlstein says, "past performance is all a pollster has to go on." When Nate Silver weights his model in favor of a given poll, it's based on that poll's historical accuracy, not its future accuracy, because its future accuracy can't be determined until it's in the past. Like Dante's fortune-tellers, pollsters have to look backwards even as they march forwards.
Of course, it doesn't help that in some cases, Silver was just bad at assessing polls for accuracy, like when he put polls from the far-right "shock pollster" Trafalgar Group into the highly reliable bucket. Since 2016, Trafalgar has specialized in releasing garbage polls that announce that MAGA weirdos are way ahead, and because they always say that, they were far more accurate than the Clinton-predicting competition in 2016 when they proclaimed that Trump had it in the bag. For Silver, this warranted an "A-" on reliability, and that is partially to blame for how bad Silver's 2020 predictions were, when Republicans got pasted, but Trafalgar continued to predict a Democratic wipeout. Silver's methodology has a huge flaw: because Trafalgar's prediction history began in 2016, that single data-point made them look pretty darned reliable, even though their method was to just keep saying the same thing, over and over:
Pollsters who get lucky with a temporarily reliable methodology inevitably get cocky and start cutting corners. After all, polling is expensive, so discontinuing the polls once you think you have an answer is a way to increase the enterprise's profitability. But, of course, pollsters can only make money so long as they're somewhat reliable, which leads to a whole subindustry of excuse-making when this cost-cutting bites them in the ass. In 1948, George Gallup blamed his failures on the audience, who failed to grasp the "difference between forecasting an election and picking the winner of a horse race." In 2016, Silver declared that he'd been right because he'd given Trump at 28.6% chance of winning.
This isn't an entirely worthless excuse. If you predict that Clinton's victory is 71.4% in the bag, you are saying that Trump might win. But pollsters want to eat their cake and have it, too: when they're right, they trumpet their predictive accuracy, without any of the caveats they are so insistent upon when they blow it:
https://www.youtube.com/watch?v=1jDlo7YfUxc
There's always some excuse when it comes to the polls: in 1952, George Gallup called the election a tossup, but it went for Eisenhower in a landslide. He took out a full-page NYT ad, trumpeting that he was right, actually, because he wasn't accounting for undecided voters.
Polling is ultimately a form of empiricism-washing. The pollster may be counting up poll responses, but that doesn't make the prediction any less qualitative. Sure, the pollster counts responses, but who they ask, and what they do with those responses, is purely subjective. They're making guesses (or wishes) about which people are likely to vote, and what it means when someone tells you they're undecided. This is at least as much an ideological project as it is a scientific one:
But for all that polling is ideological, it's a very thin ideology. When it comes to serious political deliberation, questions like "who is likely to vote" and "what does 'undecided' mean" are a lot less important than, "what are the candidates promising to do?" and "what are the candidates likely to do?"
But – as Perlstein writes – the only kind of election journalism that is consistently, adequately funded is poll coverage. As a 1949 critic put it, this isn't the "pulse of democracy," it's "its baby talk."
Today, Tor Books publishes VIGILANT, a new, free LITTLE BROTHER story about creepy surveillance in distance education. It follows SPILL, another new, free LITTLE BROTHER novella about oil pipelines and indigenous landback.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
Nate Silver’s website suffered because of Trump and changes in political news coverage.
Opinion | Perry Bacon, Jr. | March 7, 2025
FiveThirtyEight became famous for its “forecasts” from founder Nate Silver. But the website (where I worked from 2017 to 2021) was trying to do much more than predict presidential election results. FiveThirtyEight was an attempt to improve and reimagine journalism.
I think it succeeded — even though the website is now defunct.
ABC News, which owned FiveThirtyEight, this week laid off the site’s 15 remaining staffers. The network had already made drastic cutbacks two years ago, with Silver himself departing back then.
We are in the midst of staff reductions throughout the journalism industry. That said, ABC News is not a newspaper in a declining city in the Midwest. If the network wanted to keep the site going, it could have. This decision probably wasn’t just about money.
[...]
Political journalism has changed in ways that have made FiveThirtyEight less essential. Silver started the website during the 2008 presidential campaign. (There are 538 votes in the electoral college.) He correctly saw a flaw in American political coverage. Journalism professors and many within the news industry had for years argued that political news was too focused on the “horse race” (who was going to win the next election) instead of policy issues. What Silver argued was that horse-race coverage, while extensive, was often quite bad. It was overly fixated on a single poll or arguing that a candidate appeared to be surging after delivering a strong speech, without any other evidence.
Averaging polls, scrutinizing demographics and voting histories of states — that all seems obvious now. It wasn’t 17 years ago.
[emphasis added]
I will miss FiveThirtyEight. It was always a reliable source of aggregate polling data. It also provided a lot of background information about the potential bias and reliability of individual polls.
R.I.P.
FiveThirtyEight
March 7, 2008 - March 5, 2025
_________________
Collage sources (before edits, starting in center, then moving top left to right clockwise, ending bottom left): 01, 02, 03, 04, 05, 06, 07
[See more excerpts from the column under the cut]
In 2010, the New York Times hired Silver and starting hosting FiveThirtyEight on its website. A few years later, ESPN hired him to create a FiveThirtyEight that would cover not only politics but also sports, science and other topics with statisticians and more traditional journalists working in a combined newsroom.
The site grew in size and influence. And other news organizations started borrowing its methods, averaging polls and producing statistical models to analyze elections.
[...]
The site often had political scientists and scholars write pieces. Fact-checking was extensive, adding to the site’s reliability and reputation.
But I knew FiveThirtyEight was in trouble when I saw not only stories similar to ours published in the Times and The Washington Post but also those larger organizations poaching our staffers.
Another factor that made the website less relevant was Trump. He made politics more about tweets, firings and other drama that the data can’t really capture.
[...]
But for me, FiveThirtyEight staffers and its devoted fans, the site was about much more than election predictions and even Silver. It was an alternative, higher form of journalism.
It was also a lovable community of nerds, wonks and junkies. Our readers were Democratic-leaning, but they weren’t people watching MSNBC just to hear how terrible Republicans are. They wanted us to tell them if a Democratic politician was going to lose. They loved that every article seemed to involve the writer examining election results down to the county level and producing three charts to support their thesis.
Silver now has one of the most popular political Substack newsletters; former managing editor Micah Cohen is now politics editor for Apple News; reporter Anna Maria Barry-Jester has moved on to cover public health for ProPublica. But from my vantage point, FiveThirtyEight is everywhere in more subtle ways. The amount of charts and data in stories about politics in particular is much larger than it was two decades ago. The chief political analyst at the New York Times is a data whiz named Nate (Cohn) who joined the paper essentially as Silver’s replacement. If you tell someone about a poll, they will often ask whether other surveys show the same result.
There is still too much horse-race coverage. I hate when I see polls of the 2028 Democratic primary. Can we wait a minute? But FiveThirtyEight made that coverage smarter and more rigorous — creating a legacy that will endure.
Trump's state-sponsored terrorists don't want people observing them when assaulting and killing US citizens despite the fact that they are already masked and not displaying identification.
Greg Bovino, head of the Border Patrol, was riding around Minneapolis and behaving like "an Afghan war lord". He did get out of his government vehicle to throw a gas canister at demonstrators and other citizens documenting the militarization of a US city by the Trump GesTACO.
Like other terrorists, Trump's agents are abducting little kids to use them as "bait".
ICE detains 5-year old Minnesota boy; lawyer says agents used him as ‘bait’
Trump's marquee issue is immigration. But according to poll aggregator Nate Silver, Trump is now 10.7% under water – with that margin widening as the administration continues to terrorize citizens.
As of Thursday, there are 285 days until the 2026 Midterm Elections. If you are doing something to help defeat Republicans at any level of government then you are an American hero. 🦸🏼♀️🇺🇸🦸🏻♂️
Silver Bulletin will begin publishing some new features soon — including Trump approval ratings. But the media just lost an extremely valuab
Nate Silver at Silver Bulletin:
Last night, as President Trump delivered his State of the Union address1, the Wall Street Journal reported that ABC News would lay off the remaining staff at 538 as part of broader cuts within corporate parent Disney. Having been through several rounds of this before, including two years ago when the staff was cut by more than half and my tenure expired too2, I know it’s a brutal process for everyone involved. It’s also tough being in a business while having a constant anvil over your head, as we had in pretty much every odd-numbered (non-election) year from 2017 onward at 538/FiveThirtyEight.3 I don’t know all of the staffers from the most recent iteration of the site, but the ones I have met or who I overlapped with are all extremely conscientious and hard-working people and were often forced to work double-duty as jobs were cut but frequently not replaced. My heart goes out to them, and I’m happy to provide recommendations for people I worked with there.
Beyond that, I wasn’t inclined to say too much more, but it felt weirder not to say anything at all. And it’s easier to say something here than filter it through a reporter or something.
For more extended thoughts on the environment at Disney — plus plenty of self-reflection/self-criticism — you can see the item at the bottom of SBSQ #12.4 But the basic issue is that Disney was never particularly interested in running FiveThirtyEight as a business, even though I think it could have been a good business. Although they were generous in maintaining the site for so long and almost never interfered in our editorial process, the sort of muscle memory a media property builds early in its tenure tends to stick. We had an incredibly talented editorial staff, but we never had enough “product” people or strategy people to help the business grow and sustain itself. It’s always an uphill battle under those conditions, particularly when it comes to recruiting and retaining staff, who were constantly being poached by outlets like the New York Times and the Washington Post.
“Data journalism” has a bad name but a bright future
It also doesn’t quite feel like the end, exactly. “Data journalism” may have been a dumb name for what we were doing — that one’s on me — and Fivey Fox aside, the FiveThirtyEight brand was never warm and cuddly. But it always found a huge audience, and coverage of polls and political data is now much smarter. Compare the extremely analytical polling deep dives that Nate Cohn is doing at the New York Times, for instance, to the vibes-based coverage of the Boys on the Bus era. That trend may get even more entrenched as former 538ers form a diaspora that filters out to the rest of the media.
Also, I’d like to think we’re carrying a piece of the FiveThirtyEight torch here at Silver Bulletin. So, just two more quick beats. One about our near-term plans here — in slightly awkward timing, we’re launching our Trump approval ratings dashboard tomorrow. But first, a shout-out to one of the extremely valuable functions that FiveThirtyEight provided.
Collecting and maintaining a database of public polls is a lot of work, requiring diligence, meticulousness, and dealing with constant complaints about edge cases from readers and pollsters. But it’s also a public service. Polling has its challenges, but I believe it’s vital in a democracy. People only get to vote every two to four years — if they’re lucky enough to live in a state where their vote even matters. While being too “poll-driven” has pitfalls, the alternative isn’t necessarily enlightened governance. Rather, left to their own devices, elected officials are often inclined to follow some combination of (i) narrow self-interest, (ii) the loudest voices in the room, and (iii) elite opinion, which often doesn’t match broader public opinion.
FiveThirtyEight had long made its data publicly available through APIs and other tools. At Silver Bulletin, I hope we’re upholding that tradition too — although admittedly with a twist. For instance, not only are our pollster ratings (which apply the methodology I originally developed for FiveThirtyEight5) publicly available, but so is the underlying database of more than 12,000 polls that populates them. And for our presidential election forecast last year, our polling averages were free for everyone — though, here’s the twist, of course — the probabilities that the model spat out were paywalled. Even so, there were literally dozens of data visualizations and downloadable files just beyond that paywall — we’re not giving away the cow (the model code), but you’re getting basically everything else.
Nate Silver, the creator behind FiveThirtyEight, touches on ABC News’s shutting down of the venerable data site on the same night as 47’s address to the nation. Seeing a poll aggregator like 538 going away is sad in many ways (just like HuffPost eliminating Pollster).
