Epic Systems makes the dominant electronic health record (EHR) system in America; if you're a doctor, chances are you are required to use it, and for every hour a doctor spends with a patient, they have to spend two hours doing clinically useless bureaucratic data-entry on an Epic EHR.
How could a product so manifestly unfit for purpose be the absolute market leader? Simple: as Robert Kuttner describes in an excellent feature in The American Prospect, Epic may be a clinical disaster, but it's a profit-generating miracle:
At the core of Epic's value proposition is "upcoding," a form of billing fraud that is beloved of hospital administrators, including the "nonprofit" hospitals that generate vast fortunes that are somehow not characterized as profits. Here's a particularly egregious form of upcoding: back in 2020, the Poudre Valley Hospital in Ft Collins, CO locked all its doors except the ER entrance. Every patient entering the hospital, including those receiving absolutely routine care, was therefore processed as an "emergency."
In April 2020, Caitlin Wells Salerno – a pregnant biologist – drove to Poudre Valley with normal labor pains. She walked herself up to obstetrics, declining the offer of a wheelchair, stopping only to snap a cheeky selfie. Nevertheless, the hospital recorded her normal, uncomplicated birth as a Level 5 emergency – comparable to a major heart-attack – and whacked her with a $2755 bill for emergency care:
Upcoding has its origins in the Reagan revolution, when the market-worshipping cultists he'd put in charge of health care created the "Prospective Payment System," which paid a lump sum for care. The idea was to incentivize hospitals to provide efficient care, since they could keep the difference between whatever they spent getting you better and the set PPS amount that Medicare would reimburse them. Hospitals responded by inventing upcoding: a patient with controlled, long-term coronary disease who showed up with a broken leg would get coded for the coronary condition and the cast, and the hospital would pocket both lump sums:
The reason hospital administrators love Epic, and pay gigantic sums for systemwide software licenses, is directly connected to the two hours that doctors spent filling in Epic forms for every hour they spend treating patients. Epic collects all that extra information in order to identify potential sources of plausible upcodes, which allows hospitals to bill patients, insurers, and Medicare through the nose for routine care. Epic can automatically recode "diabetes with no complications" from a Hierarchical Condition Category code 19 (worth $894.40) as "diabetes with kidney failure," code 18 and 136, which gooses the reimbursement to $1273.60.
Epic snitches on doctors to their bosses, giving them a dashboard to track doctors' compliance with upcoding suggestions. One of Kuttner's doctor sources says her supervisor contacts her with questions like, "That appointment was a 2. Don’t you think it might be a 3?"
Robert Kuttner is the perfect journalist to unravel the Epic scam. As a journalist who wrote for The New England Journal of Medicine, he's got an insider's knowledge of the health industry, and plenty of sources among health professionals. As he tells it, Epic is a cultlike, insular company that employs 12.500 people in its hometown of Verona, WI.
The EHR industry's origins start with a GW Bush-era law called the HITECH Act, which was later folded into Obama's Recovery Act in 2009. Obama provided $27b to hospitals that installed EHR systems. These systems had to more than track patient outcomes – they also provided the data for pay-for-performance incentives. EHRs were already trying to do something very complicated – track health outcomes – but now they were also meant to underpin a cockamamie "incentives" program that was supposed to provide a carrot to the health industry so it would stop killing people and ripping off Medicare. EHRs devolved into obscenely complex spaghetti systems that doctors and nurses loathed on sight.
But there was one group that loved EHRs: hospital administrators and the private companies offering Medicare Advantage plans (which also benefited from upcoding patients in order to soak Uncle Sucker):
The spread of EHRs neatly tracks with a spike in upcharging: "from 2014 through 2019, the number of hospital stays billed at the highest severity level increased almost 20 percent…the number of stays billed at each of the other severity levels decreased":
The purpose of a system is what it does. Epic's industry-dominating EHR is great at price-gouging, but it sucks as a clinical tool – it takes 18 keystrokes just to enter a prescription:
Doctors need to see patients, but their bosses demand that they satisfy Epic's endless red tape. Doctors now routinely stay late after work and show up hours early, just to do paperwork. It's not enough. According to another one of Kuttner's sources, doctors routinely copy-and-paste earlier entries into the current one, a practice that generates rampant errors. Some just make up random numbers to fulfill Epic's nonsensical requirements: the same source told Kuttner that when prompted to enter a pain score for his TB patients, he just enters "zero."
Don't worry, Epic has a solution: AI. They've rolled out an "ambient listening" tool that attempts to transcribe everything the doctor and patient say during an exam and then bash it into a visit report. Not only is this prone to the customary mistakes that make AI unsuited to high-stakes, error-sensitive applications, it also represents a profound misunderstanding of the purpose of clinical notes.
The very exercise of organizing your thoughts and reflections about an event – such as a medical exam – into a coherent report makes you apply rigor and perspective to events that otherwise arrive as a series of fleeting impressions and reactions. That's why blogging is such an effective practice:
The answer to doctors not having time to reflect and organize good notes is to give them more time – not more AI. As another doctor told Kuttner: "Ambient listening is a solution to a self-created problem of requiring too much data entry by clinicians."
EHRs are one of those especially hellish public-private partnerships. Health care doctrine from Reagan to Obama insisted that the system just needed to be exposed to market forces and incentives. EHRs are designed to allow hospitals to win as many of these incentives as possible. Epic's clinical care modules do this by bombarding doctors with low-quality diagnostic suggestions with "little to do with a patient’s actual condition and risks," leading to "alert fatigue," so doctors miss the important alerts in the storm of nonsense elbow-jostling:
Clinicians who actually want to improve the quality of care in their facilities end up recording data manually and keying it into spreadsheets, because they can't get Epic to give them the data they need. Meanwhile, an army of high-priced consultants stand ready to give clinicians advise on getting Epic to do what they need, but can't seem to deliver.
Ironically, one of the benefits that Epic touts is its interoperability: hospitals that buy Epic systems can interconnect those with other Epic systems, and there's a large ecosystem of aftermarket add-ons that work with Epic. But Epic is a product, not a protocol, so its much-touted interop exists entirely on its terms, and at its sufferance. If Epic chooses, a doctor using its products can send files to a doctor using a rival product. But Epic can also veto that activity – and its veto extends to deciding whether a hospital can export their patient records to a competing service and get off Epic altogether.
One major selling point for Epic is its capacity to export "anonymized" data for medical research. Very large patient data-sets like Epic's are reasonably believed to contain many potential medical insights, so medical researchers are very excited at the prospect of interrogating that data.
But Epic's approach – anonymizing files containing the most sensitive information imaginable, about millions of people, and then releasing them to third parties – is a nightmare. "De-identified" data-sets are notoriously vulnerable to "re-identification" and the threat of re-identification only increases every time there's another release or breach, which can used to reveal the identities of people in anonymized records. For example, if you have a database of all the prescribing at a given hospital – a numeric identifier representing the patient, and the time and date when they saw a doctor and got a scrip. At any time in the future, a big location-data breach – say, from Uber or a transit system – can show you which people went back and forth to the hospital at the times that line up with those doctor's appointments, unmasking the person who got abortion meds, cancer meds, psychiatric meds or other sensitive prescriptions.
The fact that anonymized data can – will! – be re-identified doesn't mean we have to give up on the prospect of gleaning insight from medical records. In the UK, the eminent doctor Ben Goldacre and colleagues built an incredible effective, privacy-preserving "trusted research environment" (TRE) to operate on millions of NHS records across a decentralized system of hospitals and trusts without ever moving the data off their own servers:
The TRE is an open source, transparent server that accepts complex research questions in the form of database queries. These queries are posted to a public server for peer-review and revision, and when they're ready, the TRE sends them to each of the databases where the records are held. Those databases transmit responses to the TRE, which then publishes them. This has been unimaginably successful: the prototype of the TRE launched during the lockdown generated sixty papers in Nature in a matter of months.
Monopolies are inefficient, and Epic's outmoded and dangerous approach to research, along with the roadblocks it puts in the way of clinical excellence, epitomizes the problems with monopoly. America's health care industry is a dumpster fire from top to bottom – from Medicare Advantage to hospital cartels – and allowing Epic to dominate the EHR market has somehow, incredibly, made that system even worse.
Naturally, Kuttner finishes out his article with some antitrust analysis, sketching out how the Sherman Act could be brought to bear on Epic. Something has to be done. Epic's software is one of the many reasons that MDs are leaving the medical profession in droves.
Epic epitomizes the long-standing class war between doctors who want to take care of their patients and hospital executives who want to make a buck off of those patients.
Tor Books as just published two new, free LITTLE BROTHER stories: VIGILANT, about creepy surveillance in distance education; and SPILL, about oil pipelines and indigenous landback.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
Tonight (November 27), I'm appearing at the Toronto Metro Reference Library with Facebook whistleblower Frances Haugen.
On November 29, I'm at NYC's Strand Books with my novel The Lost Cause, a solarpunk tale of hope and danger that Rebecca Solnit called "completely delightful."
Last week's spectacular OpenAI soap-opera hijacked the attention of millions of normal, productive people and nonsensually crammed them full of the fine details of the debate between "Effective Altruism" (doomers) and "Effective Accelerationism" (AKA e/acc), a genuinely absurd debate that was allegedly at the center of the drama.
Very broadly speaking: the Effective Altruists are doomers, who believe that Large Language Models (AKA "spicy autocomplete") will someday become so advanced that it could wake up and annihilate or enslave the human race. To prevent this, we need to employ "AI Safety" – measures that will turn superintelligence into a servant or a partner, nor an adversary.
Contrast this with the Effective Accelerationists, who also believe that LLMs will someday become superintelligences with the potential to annihilate or enslave humanity – but they nevertheless advocate for faster AI development, with fewer "safety" measures, in order to produce an "upward spiral" in the "techno-capital machine."
Once-and-future OpenAI CEO Altman is said to be an accelerationists who was forced out of the company by the Altruists, who were subsequently bested, ousted, and replaced by Larry fucking Summers. This, we're told, is the ideological battle over AI: should cautiously progress our LLMs into superintelligences with safety in mind, or go full speed ahead and trust to market forces to tame and harness the superintelligences to come?
This "AI debate" is pretty stupid, proceeding as it does from the foregone conclusion that adding compute power and data to the next-word-predictor program will eventually create a conscious being, which will then inevitably become a superbeing. This is a proposition akin to the idea that if we keep breeding faster and faster horses, we'll get a locomotive:
As Molly White writes, this isn't much of a debate. The "two sides" of this debate are as similar as Tweedledee and Tweedledum. Yes, they're arrayed against each other in battle, so furious with each other that they're tearing their hair out. But for people who don't take any of this mystical nonsense about spontaneous consciousness arising from applied statistics seriously, these two sides are nearly indistinguishable, sharing as they do this extremely weird belief. The fact that they've split into warring factions on its particulars is less important than their unified belief in the certain coming of the paperclip-maximizing apocalypse:
White points out that there's another, much more distinct side in this AI debate – as different and distant from Dee and Dum as a Beamish Boy and a Jabberwork. This is the side of AI Ethics – the side that worries about "today’s issues of ghost labor, algorithmic bias, and erosion of the rights of artists and others." As White says, shifting the debate to existential risk from a future, hypothetical superintelligence "is incredibly convenient for the powerful individuals and companies who stand to profit from AI."
After all, both sides plan to make money selling AI tools to corporations, whose track record in deploying algorithmic "decision support" systems and other AI-based automation is pretty poor – like the claims-evaluation engine that Cigna uses to deny insurance claims:
On a graph that plots the various positions on AI, the two groups of weirdos who disagree about how to create the inevitable superintelligence are effectively standing on the same spot, and the people who worry about the actual way that AI harms actual people right now are about a million miles away from that spot.
There's that old programmer joke, "There are 10 kinds of people, those who understand binary and those who don't." But of course, that joke could just as well be, "There are 10 kinds of people, those who understand ternary, those who understand binary, and those who don't understand either":
What's more, the joke could be, "there are 10 kinds of people, those who understand hexadecenary, those who understand pentadecenary, those who understand tetradecenary [und so weiter] those who understand ternary, those who understand binary, and those who don't." That is to say, a "polarized" debate often has people who hold positions so far from the ones everyone is talking about that those belligerents' concerns are basically indistinguishable from one another.
The act of identifying these distant positions is a radical opening up of possibilities. Take the indigenous philosopher chief Red Jacket's response to the Christian missionaries who sought permission to proselytize to Red Jacket's people:
https://historymatters.gmu.edu/d/5790/
Red Jacket's whole rebuttal is a superb dunk, but it gets especially interesting where he points to the sectarian differences among Christians as evidence against the missionary's claim to having a single true faith, and in favor of the idea that his own people's traditional faith could be co-equal among Christian doctrines.
The split that White identifies isn't a split about whether AI tools can be useful. Plenty of us AI skeptics are happy to stipulate that there are good uses for AI. For example, I'm 100% in favor of the Human Rights Data Analysis Group using an LLM to classify and extract information from the Innocence Project New Orleans' wrongful conviction case files:
Automating "extracting officer information from documents – specifically, the officer's name and the role the officer played in the wrongful conviction" was a key step to freeing innocent people from prison, and an LLM allowed HRDAG – a tiny, cash-strapped, excellent nonprofit – to make a giant leap forward in a vital project. I'm a donor to HRDAG and you should donate to them too:
https://hrdag.networkforgood.com/
Good data-analysis is key to addressing many of our thorniest, most pressing problems. As Ben Goldacre recounts in his inaugural Oxford lecture, it is both possible and desirable to build ethical, privacy-preserving systems for analyzing the most sensitive personal data (NHS patient records) that yield scores of solid, ground-breaking medical and scientific insights:
https://www.youtube.com/watch?v=_-eaV8SWdjQ
The difference between this kind of work – HRDAG's exoneration work and Goldacre's medical research – and the approach that OpenAI and its competitors take boils down to how they treat humans. The former treats all humans as worthy of respect and consideration. The latter treats humans as instruments – for profit in the short term, and for creating a hypothetical superintelligence in the (very) long term.
As Terry Pratchett's Granny Weatherwax reminds us, this is the root of all sin: "sin is when you treat people like things":
So much of the criticism of AI misses this distinction – instead, this criticism starts by accepting the self-serving marketing claim of the "AI safety" crowd – that their software is on the verge of becoming self-aware, and is thus valuable, a good investment, and a good product to purchase. This is Lee Vinsel's "Criti-Hype": "taking press releases from startups and covering them with hellscapes":
Criti-hype and AI were made for each other. Emily M Bender is a tireless cataloger of criti-hypeists, like the newspaper reporters who breathlessly repeat " completely unsubstantiated claims (marketing)…sourced to Altman":
Bender, like White, is at pains to point out that the real debate isn't doomers vs accelerationists. That's just "billionaires throwing money at the hope of bringing about the speculative fiction stories they grew up reading – and philosophers and others feeling important by dressing these same silly ideas up in fancy words":
All of this is just a distraction from real and important scientific questions about how (and whether) to make automation tools that steer clear of Granny Weatherwax's sin of "treating people like things." Bender – a computational linguist – isn't a reactionary who hates automation for its own sake. On Mystery AI Hype Theater 3000 – the excellent podcast she co-hosts with Alex Hanna – there is a machine-generated transcript:
https://www.buzzsprout.com/2126417
There is a serious, meaty debate to be had about the costs and possibilities of different forms of automation. But the superintelligence true-believers and their criti-hyping critics keep dragging us away from these important questions and into fanciful and pointless discussions of whether and how to appease the godlike computers we will create when we disassemble the solar system and turn it into computronium.
The question of machine intelligence isn't intrinsically unserious. As a materialist, I believe that whatever makes me "me" is the result of the physics and chemistry of processes inside and around my body. My disbelief in the existence of a soul means that I'm prepared to think that it might be possible for something made by humans to replicate something like whatever process makes me "me."
Ironically, the AI doomers and accelerationists claim that they, too, are materialists – and that's why they're so consumed with the idea of machine superintelligence. But it's precisely because I'm a materialist that I understand these hypotheticals about self-aware software are less important and less urgent than the material lives of people today.
It's because I'm a materialist that my primary concerns about AI are things like the climate impact of AI data-centers and the human impact of biased, opaque, incompetent and unfit algorithmic systems – not science fiction-inspired, self-induced panics over the human race being enslaved by our robot overlords.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
