#newcomb's problem

[bonvivant-divisionbell]

"nothing's changed but you." [x]

a stunning commission from @an-artistic-failure. thank you for wading through my vibes-based descriptions of cosmic horror. hank was terrific to work with and brought to life my very self-indulgent request to celebrate the longest fic I've ever written for a bunch of turtles and one (1) highly traumatised child soldier.

[mitchfynde]

Deep down I know that all my worst enemies are two-boxers.

[jbeshir]

Confusion Around Newcomb’s Problem: Theories of Counterfactuals

So the big reason I’ve been sitting this discourse out is that my take on the discourse has been that it seemed to be to be mostly people with different theories of counterfactuals talking past each other, with no one having read any of the papers that explain what a theory of counterfactuals is.

My main suggestion on all of this is to read one or more of the papers discussing various decision theories, what differs in them, and the different questions they ask to make their decisions. Towards Idealised Decision Theory is a relatively short (13 pages without the citations) one which I think deconfuses things considerably. But I’m going to give explaining a go! I might make errors here, since I’ve not checked this off anyone else.

Very roughly: The disagreement is over how you resolve that as a deterministic/by-hypothesis-perfectly-predictable entity, reasoning conditional on yourself performing any action other than the one you are, in fact, going to do, suffers from problems with contradiction. You need a theory of counterfactuals- a method for constructing hypothetical worlds where you take each action in order to choose between them- to decide where you introduce that potential contradiction at.

Causal Decision Theory’s theory of counterfactuals works by blackboxing (in a somewhat underdefined way) the physical system that is you, and for every action, holding everything not causally downstream of that physical system constant while updating everything casually downstream of it.

It, more or less, answers the question of “If a hand of God reached into the universe at this point and changed my behaviour in a way that ignored causality, what behaviour would I like them to give me?”, and accordingly is big on two-boxing, accepting awful deals in (one-shot) Ultimatum Games against a counterparty who could perfectly predict what they’d accept, paying out in the Counterfactual Blackmail Problem, and defecting on perfect copies of itself in one-shot prisoners’ dilemmas.

Evidential Decision Theory’s theory of counterfactuals works by, for every action, supposing that you observe yourself to perform each action and holding nothing constant.

It more or less answers the question of “If I could see myself take any action, which one would make my expectation for the world highest?”. This sounds a lot like “I choose the action that leads me to expect to win/get the million dollars”, but it’s also big on paying out in XOR blackmail (”If you receive this message, I the superhonest and great predictor predict EITHER that all your investments lose all their value before you can sell them XOR that you will send me $1,000 in the next 24 hours″) and does other weird things.

Updateless Decision Theory’s proposed theory of counterfactuals works by blackboxing (in a somewhat underdefined way) the abstract computation that is your decision making process given all its current parameters, and for every action, holding everything not causally downstream of that computation constant while updating everything causally downstream of it.

It more or less answers the question of “If a hand of God reached into the universe and changed the output of the decision-computation that I’m doing now everywhere it’s instantiated, what would I like them to change it to?”. This is much like CDT, except that it treats the behaviour of all perfect copies, simulations, and sufficiently good predictors (in its “somewhat undefined way”) as also causally downstream of your decision. There’s proof-based (suppose that UDT(observation) = action, what I can prove from this is downstream) and graphical (more CDT looking) conceptions of this.

(There’s also Functional Decision Theory, which I’ve not read about yet, and Timeless Decision Theory, which as far as I can tell is in effect just an earlier name for graphical UDT.)

Which of these appeals to you most probably depends on your position on other things. Of these I lean to UDT personally; CDT and EDT are both reflectively inconsistent, in that if you had them you’d wish you could immediately self modify/make binding precommitments to behave differently under a wide range of circumstances, and there’s circumstances that seemingly adaptive human intuitions differ from both. I would like to one-box and get my million, but also not pay XOR blackmail. UDT as I read it is very heavy on determinism; the rebuttal in that mindset to “you can’t change the past” is essentially “you can’t change the future either”, it views you as not changing but determining the behaviour of anything that is, ever has, or ever will evaluate you with sufficient fidelity, which might either put you off or feel like a way to resolve decision-making with your views on determinism, depending. But there’s a lot of handwaving you need to take on faith that there’s a tidy version in there somewhere.

Mostly though in terms of talking about what choice is better than another in Newcomb’s, it comes down to different ideas of what action selection is doing under determinism/in the presence of extremely good predictors, and how their counterfactuals are being constructed. If you’re answering “which action leads me to expect a world where I get money” and someone else is asking “which action would I like a causality-breaking agent to make me do” then that’s going to go around in circles until that’s properly noticed and addressed.

[fatpinocchio]

Bad Luck Brian

One-boxes Box B is empty

[squareallworthy]

So if you were planning on two-boxing but you’re worried you might get burned, I'm willing to buy the contents of your box B, sight unseen, for $500,000. But only on the condition that you change your mind and one-box.

Of course I'm good for it. I just one-boxed and I've got my million right here.

[sulfurdragon-blog]

Newcomb’s problem w/ evil god

What if there was a god which smites anyone who: under the circumstance that the god does NOT exist, would one-box? Then the decision theory that would result in both

A - the circumstance where the god exists: not get smited and B - the circumstance where the god doesn't exist: one box

is impossible.

Even if your function is

f(reality) = {one box, if reality = god not exists

two box, if reality = god exists,

then the god would smite you if the god exists because it doesn’t care about your f(god exists), and instead would look at f(god not exists) and see that you would one box so ur screwed. 

This implies that it is possible for reality to punish you for being rational.

That said, I’m still in camp one box because I don’t think the punished for being rational thing applies to the regular newcomb’s problem cuz math probability.

random notes: So I think it’s perfectly obvious to one box because I then get to walk away with a million dollars so it’s the rational decision. Then there’s the two boxers who whine that they’re “being punished for being rational” and at first I was like lolwut? Then I consider, “what if they have a point?” and came up with this modified thought experiment to the Newcomb’s problem.

[mitchfynde]

Newcomb's problem

one-boxer

two-boxer

Voting ended onMay 7

There are 2 boxes in front of you labeled A and B. You can either take both (two-boxer), or only box B (one-boxer).

Box A always contains 1000 dollars.

Box B's contents have been chosen by the predictor.

If the predictor has predicted you will take both boxes, box B is empty. If the predictor has predicted you will only take box B, then box B contains 1 million dollars.

You aren't aware of what was predicted or the contents of box B. You are aware of the contents of box A, which is 1000 dollars.

The predictor is able to predict choices with near-certainty.

What's your choice? 1 box or 2?

[seedless-grapes-and-despair]

I recently watched this youtube video by Veritasium about Newcomb's problem, which is (definition from wikipedia because I'd probably butcher it):

two boxes are designated A and B. The player is given a choice between taking only box B or taking both boxes A and B. The player knows the following:

Box A is transparent, or open, and always contains a visible $1,000.

Box B is opaque, or closed, and its content has already been set by the predictor, who predicts with near-100% certainty:

If the predictor has predicted that the player will take both boxes A and B, then box B contains nothing.

If the predictor has predicted that the player will take only box B, then box B contains $1,000,000.

The player does not know what the predictor predicted or what box B contains while making the choice.

And upon watching, I gotta admit I feel like there's a pretty clear answer. But I'm curious, what do you guys think?

which box(es) are you taking?

Only Box B

Both Box A and Box B

Voting ended onMay 10