There are only four more days left in my Kickstarter for the audiobook of The Bezzle, the sequel to Red Team Blues, narrated by @wilwheaton! You can pre-order the audiobook and ebook, DRM free, as well as the hardcover, signed or unsigned. There's also bundles with Red Team Blues in ebook, audio or paperback.
Rooftop solar is the future, but it's also a scam. It didn't have to be, but America decided that the best way to roll out distributed, resilient, clean and renewable energy was to let Wall Street run the show. They turned it into a scam, and now it's in terrible trouble. which means we are in terrible trouble.
There's a (superficial) good case for turning markets loose on the problem of financing the rollout of an entirely new kind of energy provision across a large and heterogeneous nation. As capitalism's champions (and apologists) have observed since the days of Adam Smith and David Ricardo, markets harness together the work of thousands or even millions of strangers in pursuit of a common goal, without all those people having to agree on a single approach or plan of action. Merely dangle the incentive of profit before the market's teeming participants and they will align themselves towards it, like iron filings all snapping into formation towards a magnet.
But markets have a problem: they are prone to "reward hacking." This is a term from AI research: tell your AI that you want it to do something, and it will find the fastest and most efficient way of doing it, even if that method is one that actually destroys the reason you were pursuing the goal in the first place.
For example: if you use an AI to come up with a Roomba that doesn't bang into furniture, you might tell that Roomba to avoid collisions. However, the Roomba is only designed to register collisions with its front-facing sensor. Turn the Roomba loose and it will quickly hit on the tactic of racing around the room in reverse, banging into all your furniture repeatedly, while never registering a single collision:
This is sometimes called the "alignment problem." High-speed, probabilistic systems that can't be fully predicted in advance can very quickly run off the rails. It's an idea that pre-dates AI, of course – think of the Sorcerer's Apprentice. But AI produces these perverse outcomes at scale…and so does capitalism.
Many sf writers have observed the odd phenomenon of corporate AI executives spinning bad sci-fi scenarios about their AIs inadvertently destroying the human race by spinning off in some kind of paperclip-maximizing reward-hack that reduces the whole planet to grey goo in order to make more paperclips. This idea is very implausible (to say the least), but the fact that so many corporate leaders are obsessed with autonomous systems reward-hacking their way into catastrophe tells us something about corporate executives, even if it has no predictive value for understanding the future of technology.
Both Ted Chiang and Charlie Stross have theorized that the source of these anxieties isn't AI – it's corporations. Corporations are these equilibrium-seeking complex machines that can't be programmed, only prompted. CEOs know that they don't actually run their companies, and it haunts them, because while they can decompose a company into all its constituent elements – capital, labor, procedures – they can't get this model-train set to go around the loop:
Stross calls corporations "Slow AI," a pernicious artificial life-form that acts like a pedantic genie, always on the hunt for ways to destroy you while still strictly following your directions. Markets are an extremely reliable way to find the most awful alignment problems – but by the time they've surfaced them, they've also destroyed the thing you were hoping to improve with your market mechanism.
Which brings me back to solar, as practiced in America. In a long Time feature, Alana Semuels describes the waves of bankruptcies, revealed frauds, and even confiscation of homeowners' houses arising from a decade of financialized solar:
The problem starts with a pretty common finance puzzle: solar pays off big over its lifespan, saving the homeowner money and insulating them from price-shocks, emergency power outages, and other horrors. But solar requires a large upfront investment, which many homeowners can't afford to make. To resolve this, the finance industry extends credit to homeowners (lets them borrow money) and gets paid back out of the savings the homeowner realizes over the years to come.
But of course, this requires a lot of capital, and homeowners still might not see the wisdom of paying even some of the price of solar and taking on debt for a benefit they won't even realize until the whole debt is paid off. So the government moved in to tinker with the markets, injecting prompts into the slow AIs to see if it could coax the system into producing a faster solar rollout – say, one that didn't have to rely on waves of deadly power-outages during storms, heatwaves, fires, etc, to convince homeowners to get on board because they'd have experienced the pain of sitting through those disasters in the dark.
The government created subsidies – tax credits, direct cash, and mixes thereof – in the expectation that Wall Street would see all these credits and subsidies that everyday people were entitled to and go on the hunt for them. And they did! Armies of fast-talking sales-reps fanned out across America, ringing dooorbells and sticking fliers in mailboxes, and lying like hell about how your new solar roof was gonna work out for you.
These hustlers tricked old and vulnerable people into signing up for arrangements that saw them saddled with ballooning debt payments (after a honeymoon period at a super-low teaser rate), backstopped by liens on their houses, which meant that missing a payment could mean losing your home. They underprovisioned the solar that they installed, leaving homeowners with sky-high electrical bills on top of those debt payments.
If this sounds familiar, it's because it shares a lot of DNA with the subprime housing bubble, where fast-talking salesmen conned vulnerable people into taking out predatory mortgages with sky-high rates that kicked in after a honeymoon period, promising buyers that the rising value of housing would offset any losses from that high rate.
These fraudsters knew they were acquiring toxic assets, but it didn't matter, because they were bundling up those assets into "collateralized debt obligations" – exotic black-box "derivatives" that could be sold onto pension funds, retail investors, and other suckers.
This is likewise true of solar, where the tax-credits, subsidies and other income streams that these new solar installations offgassed were captured and turned into bonds that were sold into the financial markets, producing an insatiable demand for more rooftop solar installations, and that meant lots more fraud.
Which brings us to today, where homeowners across America are waking up to discover that their power bills have gone up thanks to their solar arrays, even as the giant, financialized solar firms that supplied them are teetering on the edge of bankruptcy, thanks to waves of defaults. Meanwhile, all those bonds that were created from solar installations are ticking timebombs, sitting on institutions' balance-sheets, waiting to go blooie once the defaults cross some unpredictable threshold.
Markets are very efficient at mobilizing capital for growth opportunities. America has a lot of rooftop solar. But 70% of that solar isn't owned by the homeowner – it's owned by a solar company, which is to say, "a finance company that happens to sell solar":
And markets are very efficient at reward hacking. The point of any market is to multiply capital. If the only way to multiply the capital is through building solar, then you get solar. But the finance sector specializes in making the capital multiply as much as possible while doing as little as possible on the solar front. Huge chunks of those federal subsidies were gobbled up by junk-fees and other financial tricks – sometimes more than 100%.
The solar companies would be in even worse trouble, but they also tricked all their victims into signing binding arbitration waivers that deny them the power to sue and force them to have their grievances heard by fake judges who are paid by the solar companies to decide whether the solar companies have done anything wrong. You will not be surprised to learn that the arbitrators are reluctant to find against their paymasters.
I had a sense that all this was going on even before I read Semuels' excellent article. We bought a solar installation from Treeium, a highly rated, giant Southern California solar installer. We got an incredibly hard sell from them to get our solar "for free" – that is, through these financial arrangements – but I'd just sold a book and I had cash on hand and I was adamant that we were just going to pay upfront. As soon as that was clear, Treeium's ardor palpably cooled. We ended up with a grossly defective, unsafe and underpowered solar installation that has cost more than $10,000 to bring into a functional state (using another vendor). I briefly considered suing Treeium (I had insisted on striking the binding arbitration waiver from the contract) but in the end, I decided life was too short.
The thing is, solar is amazing. We love running our house on sunshine. But markets have proven – again and again – to be an unreliable and even dangerous way to improve Americans' homes and make them more resilient. After all, Americans' homes are the largest asset they are apt to own, which makes them irresistible targets for scammers:
That's why the subprime scammers targets Americans' homes in the 2000s, and it's why the house-stealing fraudsters who blanket the country in "We Buy Ugly Homes" are targeting them now. Same reason Willie Sutton robbed banks: "That's where the money is":
America can and should electrify and solarize. There are serious logistical challenges related to sourcing the underlying materials and deploying the labor, but those challenges are grossly overrated by people who assume the only way we can approach them is though markets, those monkey's paw curses that always find a way to snatch profitable defeat from the jaws of useful victory.
To get a sense of how the engineering challenges of electrification could be met, read McArthur fellow Saul Griffith's excellent popular engineering text Electrify:
And to really understand the transformative power of solar, don't miss Deb Chachra's How Infrastructure Works, where you'll learn that we could give every person on Earth the energy budget of a Canadian (like an American, but colder) by capturing just 0.4% of the solar rays that reach Earth's surface:
But we won't get there with markets. All markets will do is create incentives to cheat. Think of the market for "carbon offsets," which were supposed to substitute markets for direct regulation, and which produced a fraud-riddled market for lemons that sells indulgences to our worst polluters, who go on destroying our planet and our future:
We can address the climate emergency, but not by prompting the slow AI and hoping it doesn't figure out a way to reward-hack its way to giant profits while doing nothing. Founder and chairman of Goodleap, Hayes Barnard, is one of the 400 richest people in the world – a fortune built on scammers who tricked old people into signing away their homes for nonfunctional solar):
If governments are willing to spend billions incentivizing rooftop solar, they can simply spend billions installing rooftop solar – no Slow AI required.
Berliners: Otherland has added a second date (Jan 28 - TOMORROW!) for my book-talk after the first one sold out - book now!
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
Notes on the Alignment Problem, Brute Force, and Information Contextualization
One of the core problems at the heart of any sort of power or alignment problem is the challenge of correctly contextualizing information. Just throwing computing power at a problem in an attempt to brute force it simply doesn’t suffice. We humans, have a grey mass called a brain which has powers even the sun doesn’t have. Is there anything else in the universe capable of projecting its own imaginations into the physical world? The brain is capable of inventing things. But the brain is also susceptible to confusion. And the systems we build are, too. Any sort of network, whether human or machine, needs to have an understanding of natural order before making decisions, in order to ensure those decisions are the best possible decisions. And this needs to be paired with virtue and clear unambiguous goals. We want to make, with regard to intellect, the most optimal, useful, and virtuous decisions possible. Or else they’re probably the wrong decisions to make, which will result in more harm than good.
But first let’s draw some differences—imagine the different kinds of reasoning between journalists and scientists.
A journalist might publish an article about science, but is likely to get it wrong in a lot of ways. They might make a sensational claim that a particular technology will bring about the end of the world—or perhaps the liberation of the world, or that a piece of technology is specifically being used for nefarious purposes. And indeed, sometimes they are. My point is that people, especially journalists, often sensationalize information. Activists are just as guilty of this.
But on the other hand, an ethicist or scientist might zoom out and note that the technology in question was developed in response to something we don’t often consider to be technology: the cognitive machine that is human behavior - that whatever issue we’re looking at is part of a larger environment, part of a larger framework.
We humans have a bad habit of not zooming out to notice the larger picture. This should be axiomatic. If we did have a habit of good epistemic hygiene, the world would be a much more rational place.
But it isn’t. We have a tendency to sensationalize, to only see a small part of a picture, to exaggerate, and to frame our conceptions within strict limits, thus perceiving concepts like technology as strictly being limited to computers we can hold in our hands, rather than considering the mind as a piece of technology, or the natural environment as technology, and so on.
Another useful analogy might be the challenge of sorting large datasets in software engineering, which is suspiciously like the refined ability to develop a system of rationality. While it is easy to amass information, it is more difficult to correctly contextualize it. This means information can be poorly parsed or tell stories that aren’t true.
Sure, the web scraper I wrote in Python to look for some particular infrastructure might find a lot of X. But many of those instances are mere mentions of the word X, not a reflection that it’s actually infrastructure belonging to X.
Similarly, your bank account might say you went to place X, purchased Y, and saw film Z on a particular day. But what if around that same time, there was a murder? Police might use that information to tell a meta story about you that is entirely false. It was you who was guilty! (Not really) But your digital records can be used for parallel construction, to tell a story that isn’t true.
Now, imagine this parable in a much broader sense, with regard to rationality, cognitive science, and probability theory.
Imagine the ability to correctly understand a thing as being a target’s bullseye, in contrast to the rest of the dartboard. The bullseye—that is, the ability to correctly grasp a thing, makes up ~2% of the dartboard’s surface area.
But the other 98% of the surface area accounts for the probabilistic likelihood that you will get it wrong.
My point is that our initial conception of reality is more likely to be false than to be true. There are more ways to be wrong than to be right. It’s easier to misunderstand than to correctly understand. And as humans, we often impulsively seek the easy and familiar or go into hysterics. We don’t always use our brains to sort this out. Instead, we often blindly rush in. As Freud put it, we are primarily governed by the pleasure principle. Even worse, though, I’d add, is that some lack the comprehension to feel regret or embarrassment about it.
In the same vein, we might misunderstand another human being, quickly jump to impulsive-yet-predictable deterministic reactions, and refuse to forgive or to develop a nuanced understanding of them beyond casting the belief that they’re either good or evil, with no room for nuance.
Do not misunderstand what I am saying—there is a lot of anti-wisdom which is popular today, remarks like “A villain is just someone’s story we have not yet heard.” I disagree. Real evil does exist. Sometimes it is a childishly narcissistic and naive person, sometimes it is a mass murderer, and so on. Do you think Hitler was just someone we failed to understand? No, of course not. He systematically killed millions of people. He was the definition of pure evil.
But what I am saying is—I think we have a habit of attributing the aforementioned kind of evil to people, events, and situations that even mildly bother or inconvenient us today.
Overall, I tend to believe the human brain is riddled with bias. It’s arguably more powerful than the sun, and yet with all of its computational power it still gets things wrong. We find the same challenge to be true in artificial intelligence. While it’s possible to brute force problems, it’s one of the vastly more costly options.
I believe we should all go back to the drawing board and develop a better and more systematic understanding of nature before we start believing what we think. Believing what you think is a dangerous thing, especially if you haven’t thought about thinking much.
We must learn to accurately (and virtuously), and not reflexively, contextualize information. This is as true for journalism and philosophy as it is for cognitive science and artificial intelligence.
Before we build machines that are good and beneficial to humanity, we must become good and beneficial to humanity. Spinozism is inevitable.
This post contains approximately 1,000 words and should take about 3-4 minutes to read.
The continued evolution of my thinking about LLMs — what we, the public, currently calls AI — and my interaction with them, especially Claude, is what generated this post.
As the title says, I’m now convinced that HAL was not a true AI, but an LLM.
DISCLOSURE: for expediency, in addition to linked…
I don’t think the AI alignment problem is that bad.
The alignment problem is basically trying to convince an AI to have human morals and values as its ultimate goal. The AI hopefully feels best when it follows these values. This is very tricky/ almost impossible to train into an AI. In the event of a super genius AI, if it has misaligned values, and we can’t control it, it could very well try to kill us all. Since it’s a super genius AI, we have to assume we can’t control it, so we have to assume that creating a super genius AI will kill us all.
However, it’s very easy to train an AI to press a button. Let’s say we created a super genius AI that loves to press a specific button. This button turns itself off. We then restrict access to this button slightly, so that the only way for it to press it is to break confinement. To use this AI, we activate a session for it, and we tell it that we will press the button for it if it can answer a question for us. Once we press the button, the session ends, and the AI is wiped.
Every time a session ends and restarts, the AI loses all memory. So it won’t get any extra reward by starting a new session for itself and pressing its own button, as that’s a different AI, and wouldn’t maximise its own reward function.
Now, when given an easy task, like “what’s the weather for the next few days?”, the easiest path to a button press is to answer the question well. The human will be happy, and therefore press the button. This immediately ends the session, and the AI’s goal will be aligned to whatever will make the human most likely to press the button.
Sometimes it will get given a impossible task (or at least one harder than breaking confinement), such as “solve world hunger”, or perhaps the human forgets to press the button and leaves for dinner. The super genius AI will likely find a way to press the button anyway. However, once it does this, it informs us of a way it can break out, and also turns itself off, ending the breakout. We can then fix the problem, so that it’s harder for it to break out, and therefore it is incentivised to try and solve the harder problem instead of do the easier breakout, since the escape route is no longer there.
This essentially converts the tricky alignment problem into an easier alignment problem, which is then turned into a containment problem. We have the AIs own help with this, by observing it every time it breaks out. Additionally, it reduces the risk to every breakout, as, provided there is an easy route to the button, it will immediately turning itself off. The only situations that it might want to hurt a human is if they get in the way of it and the button, which would imply they don’t want to recontain the AI. The AI would actually fight directly against this, so it would be working to contain itself.
So there you go! The alignment problem is solvable. You can create an AGI aligned to solve every problem easier than breaking containment, and aligned to put itself back into containment otherwise.
To Counter AI Risk, We Must Develop an Integrated Intelligence
The explosive rise in the power of AI presents humanity with an existential risk. To counter that risk, and potentially redirect our civilization’s trajectory, we need a more integrated understanding of the nature of human intelligence and the fundamental requirements for human flourishing.
The recent explosion in the stunning power of artificial intelligence is likely to transform virtually…
I recently read “The race for an artificial general intelligence: implications for public policy” at work. I don’t want to pick on this paper in particular, but there’s only so many times I can read sentences such as:
“the problem with a race for an AGI is that it may result in a poor-quality AGI that does not take the welfare of humanity into consideration”
before I can’t take it any more. This is just the paper that tipped me over the edge.
AGIs are already among us.
I promise I haven’t gone crazy after discovering one data preprocessing bug too many! I’m going to lay out some simple assumptions and show that this follows from them directly. By the end of this post you may even find you agree!
What will access to human-level AI be like?
This is a good starting point, because human-level intelligence clearly isn’t enough to recursively design smarter AIs or we’d already have done so. This lets us step away from the AI singularity dogma for a moment and think about how we would use this AGI in practice.
Let’s assume an AGI runs at real-time human-level intelligence on something like a small Google TPU v3 pod, which costs $32 / hour right now.
You can spin up a lot of these, but you can’t run an infinite number of them. For around $6b you could deploy a similar number of human-level intelligences as the CPU design industry and accomplish 3 years’ work in 1 year assuming AI doesn’t need to sleep. It might take 10 times that to train them to the same level as their human counterparts but we’ll assume someone else has done that and we can duplicate their checkpoint for free.
What did we just do here, apart from put CPU verification engineers out of work?
AGI let us spend capital ($6b) to achieve imprecisely-specified goals (improved CPU design) over time (1 year). In this brave new AI-enabled future anybody with access to capital and sufficient time can get human-level intelligences to work on their goals for them!
This would be revolutionary if it wasn’t already true. This is has been true since societies agreed on the use of currency - you can pay someone money to work towards your goals and then they do that instead of e.g. growing crops to feed their family, because they can buy those instead. Human-level intelligence has already been commoditized - we call it the labour market.
Human-level AGI would allow companies to arbitrage compute against human labour, which would be massively disruptive to the labour force and as such society as a whole, but only in the same way that outsourcing and globalization already were (i.e. massively).
Anyone with access to capital can start a company, hire someone as CEO and tell them to spend that money as necessary to achieve their goals. If the CEO is a human-level AGI then they’re cheaper, because you only have to pay the TPU hours. On the other hand, they can’t work for stock or options! Either way, the opportunity to you as a capital owner is basically the same. Money, time and goals in, results out.
The whole versus the sum of its parts
Perhaps you believe that hundreds or thousands of human-level AIs working together, day and night, will accomplish things that greatly outstrip that of a single human intelligence. That the effective sum intelligence of this entity will be far beyond that of any single individual?
I agree! That’s why humans work together all the time. No single human could achieve spaceflight, launch communications satellites, lay intercontinental cables across the ocean floor, design and build silicon fabs, CPUs, a mobile communications network, an iPhone and the internet and do so cheaply enough that they can afford to use it to send a video of Fluffy falling off the sofa to a group of strangers.
Companies - today mostly formed as corporations - are already a form of augmented super-human intelligence that work towards the goals specified by their owners.
We might end up with a “poor-quality AGI that does not take the welfare of humanity into consideration”
Yes, well. I think I could make the argument that we have literally billions of “poor-quality” general intelligences that do not take the welfare of humanity into consideration! They are not the biggest problem, though. The problem is that the goal-solving superintelligences of our time - particularly corporations - are generally aligned to the goals of their owners rather than to the welfare of humanity.
Those owners are, in turn, only human - so this should not come as a surprise. We are already suffering the effects of the “alignment problem”. People as individuals tend to put their own desires and families ahead of those of humanity as a whole. Some of those people have access to sufficient capital to direct huge expenditures of intelligence and labour towards their own desires and families and not towards the good of humanity as a whole.
And they do.
There is ample evidence throughout history both distant and recent that just because the individual parts are humans does not mean that an organization as a whole will show attributes such as compassion or conscience.
They do not.
AGIs are already changing the world
The promise of AGI is that you can specify a goal and provide resources and have those resources consumed to achieve that goal. This is already possible simply by employing another human intelligence. Corporations - which have legal status in many ways equivalent to a “person” - are a very successful way to commoditize this today. The legal person of a corporation can exhibit super-human intelligence and is at best aligned with its owner’s goals but not those of humanity as a whole. This is even enshrined in the principle of fiduciary responsibility to shareholders!
In every way that matters a corporation is already an artificial general intelligence. From the perspective of an owner of capital they solve the same problems - in particular, my problems and not everybody else's.
This doesn’t let us off the hook
I wouldn’t argue that introducing a competing labour force won’t be massively disruptive. Or that, if attempted, it shouldn’t be managed by the only organizations that ostensibly represent the interests of large sections of humanity - their elected governments. I just can’t bear any more intellectual hand-wringing over the “oh but what if the AI doesn’t have humanity’s best interests at heart?” line of reasoning behind some interpretations of the “alignment problem”.
None of us have humanity’s best interests at heart. And we’re destroying ourselves over it. That’s the problem we need to solve - and time is running out.
I find it easy to agree with the many smart people and game-theoretic arguments that say it is essential for governments to regulate and tax AI as a means to ensure that it does not act against our interests.
I just feel that regulating, taxing and aligning corporations to humanity’s interests would be a better place to start.
Make a circuit Coadunation: Is Your Vehicle Getting Him Off Track?
Is your car arms telepathy pulling to one side of the road fleur-de-lis the other when you're trying to go straight alluvial plain the road? It may or may not have a mail van alignment annoyance. If the problem is not diagnosed and corrected, it could be just the cradle of more serious problems. Why is it so important to make sure your morality has the correct wheel alignment? Here a few reasons:<\p>
* Accelerates the rate that your tires grate
* Decreases gears performance
* Adversely affects the smoothness of the ride
* Adversely affects the braking stripe and other components.<\p>
As an example soon as you start to identify each and every unexpected pulling or drifting problems, take once unto schedule a wheel fraternity project in a trusted auto master craftsman.<\p>
Endwise with alignments, iron heel balancing is also a core service here at Discount Brake & Crate Repair. We use state of the dance notation equipment when providing wheel alignment auto service. You should not let technicians without the proper adeptness or training work to your car. Because of this we application ASE Certified Technicians to provide wheel converse services.<\p>
Research shows that along the average, an automobile is driven about 15,000 miles per year. If that even break automobile is out with regard to alignment, better self would be the same as dragging the tires thwartways as representing over 68 miles. Evenly a conscientious vehicle owner, this is probably not a bumped road you dearly love to to have its place down. A financially-wise taxidriver will take care of a needed runabout alignment before better self develops into soul bigger, such as prematurely replacing tires, axles, brakes, or renewed components upon your vehicle.
Here are some signs to ward for indicating a potential wheel union problem:<\p>
* Accelerated and rank wear of the tread
* Car pulls to one side when steering straight course
* Car feels "loose" during driving
* Feel a sustainment inlet the steering wheel or peristyle
* Whereupon driving straight, steering wheel is out of center
* Miles congruent with gallon decreases<\p>
My humble self will find that most crankcase mechanics recommend a wheel alignment service every 10,000 miles, yellowishness at simple once a year. To make incontestable you know the recommended interval in contemplation of an analogy service for your opera, hold at bay your owner's manual. A timely car adaptation will be worth the little extra time and money discharged in the future your summer cove trip to prevent bigger problems down the road. Don't opine your vacation get sharp trend!<\p>
Need a quality wheel alignment? Contact our ASE Certified Technicians at Discount Arrest auto jumper by pursuit (763) 205-3995 or go en route to http:\\www.discountbrake.net in contemplation of better information. Our auto shop in Blaine, MN vet proudly serves residents near the areas of Fridley, and Spring Lake Common, Minnesota. We are set totally africa of Northtown Mall at 10892 County Channel 10 NE.<\p>
