A Place For Houshalter

Stuff I am interested in.

[houshalter-blog]

Range Voting Sucks

Range voting is a very simple voting system. You rate the candidates on a scale like 1 to 10. And the candidate with the best rating wins. It has a lot of nice properties over other voting systems. There is a Center for Range Voting that claims it's the best voting system and crushes all others. Particularly they run computer simulations to see how good different voting systems work under different situations. Something I have done myself and confirmed - range voting is pretty good.

The problem is it sucks in practice. Range voting has already been implemented. See the ubiquitous 5 star rating. That pervades every online app these days. Star ratings are basically the same as range voting - you are rating items on a scale and averaging together everyone's ratings.

The problem with 5 star ratings is no one uses them correctly. 99% of people either give 5 stars or 1 star. The middle stars are rarely used. When people do use them, they interpret them wildly different from each other. Some people think 3 stars means "average", and other see it as very below average.

Many places have replaced 5 star ratings with just like/dislike or upvote/downvote. Youtube did it. Steam did it. Reddit is entirely based on it.

And this is a real world scenario where there isn't a huge incentive to lie or vote tactically. It doesn't really matter if you rate a video 3 stars or 5 stars, there's little point in lying.

Voting for democratic elections matters a lot, and people do have a huge incentive to vote tactically. They will give all the candidates they want to win 10, and everyone else 0. Doing anything else hurts you. Range voting punishes honesty.

This reduces to just approval voting. So why bother with range voting at all?

In the future I will discuss my issues with approval voting. Mainly that it reduces expressibility, requires highly tactical voting, and doesn't seem to help real elections.

[houshalter-blog]

The Doomsday Argument is even Worse than Thought

I have a weird idea: That the doomsday argument is far worse than typically explained.

The doomsday argument goes something like this: Start from 0 information, ask yourself what a good prior should be for how long humanity will last. Consider that, apriori, there is nothing particularly special about you or your time. You are just as likely to be among the first humans as among the last humans. If you are equally likely to be anywhere in the distribution of all humans, there is a 90% chance that you are in the middle 90%.

Assuming the best case, that we are at exactly the 5% mark of all humans that will ever live, then we can estimate humanity will last about 15,000 years at the current birth rate. In the worst case, if we are just at the last 5% of humans to ever live, the world will last 41 more years at the current birth rate. And if we are in the exact middle, humanity has about 770 more years.

And, again, assuming there is nothing particularly special about you or your time, then there is a 90% chance the true value will fall somewhere between those two estimates. And again, this is before we process any additional information to update our estimates. I view this as a sort of uninformative prior, to apply in the absence of any other information. A starting point to add other information too.

xkcd makes an interesting point about this argument:

Almost everyone who hears this argument immediately sees something wrong with it.

The problem is, everyone thinks it's wrong for a different reason. And the more they study it, the more they tend to change their minds about what that reason is.

Since it was proposed in 1983, it's been the subject of tons of papers refuting it, and tons of papers refuting those papers. There's no consensus about the answer; it's like the airplane on a treadmill problem, but worse.

But it's worth considering the same idea applied to other areas. Imagine I went around randomly picking objects, figuring out how old they are, and applying the doomsday argument. I pick a tree that's 10 years old, and estimate there is a 90% chance it will last between 6 months to 10 years. And I pick a random bridge and estimate how long it will stand. And on average, my 90% confidence intervals should be correct about 90% of the time.

In fact this was famously done by J. Richard Gott, who made a pretty reasonable prediction of when the Berlin wall would fall using this method.

The same math was also famously applied by the Allies in World War II. They used it estimate the number of tanks produced by the Germans from their serial numbers. They were able to estimate the German production of tanks to within 1 tank of the true value. The doomsday argument is just a special case of the same math, where the number of tanks equals one.

It occurs to me that the original formulation of the Doomsday argument is wrong. We are not randomly distributed among all humans who will ever live. Most humans who have ever lived, would never have even thought of the Doomsday argument. The doomsday argument wasn't even invented until 1983.

Assume instead that we are distributed amongst all people to ever think about the Doomsday argument. This is a much smaller sample. Let's assume that the number of people exposed to it is constant over time since 1983.

Now the numbers work out very differently. The 90% confidence interval of the date the last person will think about the Doomsday argument, is between 2017-2643. There is a 50% chance it will occur before 2049, and a greater than 70% chance it will occur before the end of the century.

I don't really know what to make of this information. Again I like to view it as an uninformative prior. A starting point to do bayesian updates too, when you add more information. But when I do add more information, like my beliefs about existential risks like AI, etc, the result gets even worse.

Thinking about anthropics really weirds me out. It's not intuitive at all, and the results just feel wrong. But still, it does seem really unlikely, that out of all the trillions and trillions of humans that could someday exist, I happened to be born into the first 1%, or even 0.01%. The odds of that are, well, less than 1%.

But if humanity ends in the next century or so, well, being born now isn't terribly unlikely at all. If that's the case, then right now is a relatively unremarkable point in time to exist. Earth's population is larger than ever and growing, and that timeline puts us nearish to the middle of all humans that will ever live. And especially nearish the middle of all humans who will think about this argument...

[houshalter-blog]

Please Take the LessWrong Diaspora 2016 Survey!

2016 LessWrong Diaspora Survey! This isn't just specific to Lesswrong users, we are trying to get people in similar communities.

Please take the survey. We need as many responses as possible. And please share this survey with others.

Thank you for doing your part to Make The Diaspora Great Again.

[houshalter-blog]

AlphaGo and Electricity

So this article just reached the top of HN: Another Way Of Looking At Lee Sedol vs AlphaGo

I've seen a lot of "AI denial" and moving goalposts in the discussions about AlphGo. Go was, for the longest time, a huge milestone for AI research. And now that it's finally succeeded, there's a lot of discussion about how it's not that significant.

But this article takes the cake for moving goalposts. It declares that the Go match wasn't fair because AlphaGo got to use far more energy than Sedol.

On the surface this claim is kind of absurd. You could give Sedol an electrical outlet with as much power as he wants. It wouldn't help at all. Humans can't make their brains bigger, or use more power, even if they want to. Which is one advantage the AI has. And that's totally fair.

But beyond that, until now no has really cared about the energy usage of AIs. The fact an AI could even do a task on the level of a human was incredible, regardless how much energy was required. The Energy usage isn't important in this domain. It's board game playing. It's shifting goalposts to goals that don't even matter!

But there is a valid reason to care about energy usage. It does cost money to run AlphaGo, and in other domains that could matter a lot. But even from the economic perspective, it's not accurate. I bet the food budget of a human is way more than the electricity budget of AlphaGo.

For this economic estimate, you need to factor in more than just energy. A human takes 20 years to mature. AlphaGo took less than a week to get good, and a few months to be the best. And over the human's life, they will use lots of energy and food.

Humans are extremely expensive, as can be estimated just from wages. If it requires only a rack of GPUs to replace an expert human, it may very well be worth it economically.

Additionally, AlphaGo doesn't require a huge distributed network of GPUs. That improves it's performance only slightly. Google claims that a version of AlphaGo that runs on a single machine, was able to beat the distributed version 25% of the time. Which would still give it a higher Elo than Sedol, who only beat AlphaGo 20% of the time.

Now besides all that, there is an important point here. Once you get past the accusations of "unfairness", it's still very interesting that human brains are so energy efficient.

But the reason for that is that artificial neural networks are run on general purpose hardware. Human brains are highly specialized hardware. General purpose computers always consume vastly more energy than specialized circuits, optimized for energy usage. GPUs compute every possible synapse to 32 bit floating point precision. Most that computation is unnecessary. The majority of synapses are 0, and only a few bits of accuracy are required on active synapses.

I know this because there has been a lot of recent research on efficiently running NNs. Especially on low budget hardware, like mobile phones or embedded devices. This is something that is already possible, and is only going to get better over time.

In the future AlphaGo could be ported to cheaper hardware, and even to specialized FPGAs or ASICs. Those are incredibly energy efficient. But there was no reason to do this for the match vs Sedol. The fact they didn't do this, tells us nothing about the progress of AI.

[houshalter-blog]

What I found in an old biology textbook

I was in an old one room school house and found an old biology text book. I am fascinated with old school text books. I like to see different things were back then.

I couldn't find what year it had been published, but it was clearly quite old. Flipping through the pages, I found something peculiar. A whole chapter on the genetics of humans! Including the inheritance of intelligence, and even suggestions of Eugenics. Something still controversial today.

I took some pictures of the chapter. I just decided to transcribe the whole thing, for anyone else interested.

From the text book Our World of Living Things:

TOPIC 3. THE MENDELIAN LAWS OF HEREDITY APPLY TO MAN

PROBLEMS AND QUESTIONS

Do the laws of heredity apply to man?

What do you conclude from the family histories of the Edwards family, the Jukes family and the Kallikaks?

What characters in men are known to be inheritable?

Are diseases acquired or inherited?

Is intelligence acquired or inherited?

Are feeble-mindedness and insanity inherited?

SUGGESTIONS AND HELPS FOR STUDY

This is an important topic. It deals with human inheritance. Study it carefully. We can not hope for rapid progress in civic betterment along biological lines until the general public understands the part heredity plays in human life.

Try to read one or more other books that treat the topic of human heredity more extensively than it has been possible to treat it here.

Study the following words. They will be used in this topic.

albinism - the absence of pigments in the skin, hear, leathers, and eyes. pigment - a coloring matter. pigmentation - coloration with pigment. delinquent - one who commits a fault or a crime.

READINGS AND ACTIVITIES WHICH WILL HELP ANSWER THE PROBLEMS AND QUESTIONS

Do the laws of heredity apply to man? The laws of heredity were discovered by experimenting on plants and lower animals. The question arises now as to whether the laws of heredity apply to man. Have you ever observed resemblances between members of the same family? Perhaps you have heard people discuss resemblances between brothers and sisters. These common observations have been verified by careful scientific studies. Human characters such as color of hair, color of eyes, skin color, intelligence, and physical defects have been traced through generations of families. All available scientific evidence indicates that many human characters follow the Mendelian laws of inheritance.

When we wish to study hereditary traits in a species of plant or a species of lower animal, we simply breed some of the species together and observe the results. Direct experiments like this are not permissible with human beings. Data must be collected by studying and analyzing family histories. Unfortunately many families do not keep complete family records; hence it is a difficult task to obtain data on physical and mental traits of persons three, four, or five generations back.

What do you conclude from the family histories of the Edwards family, the Jukes family, and the Kallikaks? Certain American families have been traced through many generations to determine the traits that occur. The Edwards family, which has been studied in this manner, shows the value of good inheritance.

Jonathan Edwards, an eminent thinker, lived in New England during colonial times. In 1900, records of 1394 of his descendants were studied. The facts that follow show the outstanding achievements of this notable family.

295 college graduates

13 college presidents

100 clergymen, missionaries, or theological professors

100 lawyers

30 judges

80 public officials

65 college professors

60 physicians

75 officers of army and navy

60 prominent authors

Winship states that almost if not every department of social progress and of public weal has felt the pulse of this healthy, long-lived family. It is not known that any of them was ever convicted of a crime.

A striking contrast to the Edwards family is [3 letters untranscribable]ished by the Jukes family. Max Jukes, a lazy and degenerate backwoodsman who lived in the state of New York, was the founder of the Jukes family of defectives, of which about twelve hundred descendants have been traced. It includes -

310 professional paupers

300 deaths in infancy

400 physical wrecks from debauchery

60 habitual thieves

50 immoral women

7 murderers

130 other criminals

At the time of this record was made this family had already cost the state of New York $1,250,000. All this crime, immorality, and expense to the state might have been avoided if the original ancestor had been prevented from producing offspring.

FIG. 267. FAMILY HISTORY OF THE KALLIKAKS

Perhaps the most convincing study of human inheritance is that of the Kallikaks. It is especially important because it contains a normal and a defective strain. While a soldier in the Revolutionary War, Martin Kallikak (a fictitious name) mated with a feeble-minded woman and became the father of a son. Of 480 descendants of this son traced in 1912, 143 were feeble-minded and most of the others were criminal, immoral, or inebriate. Not a single individual of exceptional ability was found.

After the revolutionary War, Martin Kallikak married a grl from a normal family. Of the 4?? descendants traced from this union, most were succesful persons such as educators, physicians, lawyers, merchants, land owners, etc. Not a single case of feeble-mindedness was found. Would you conclude from this family history that heredity was largely the determining factor of the differences in the two different lines of Kallikaks?

What characters in man are known to be inheritable? Investigation has shown that certain physical and mental traits in man are inherited. These characters are passed from parent to offspring in accordance with the laws of heredity. The following table [untranscribable] some of the characters of man that are either dominant or recessive traits.

CHARACTERS IN MAN

Dominant character Recessive character Brown eyes Blue eyes Dark hair Light hair Curly hair Straight hair Nervous temperament Phlegmatic temperament Normal intelligence Very little intelligence Normal intelligence Very great intelligence

The following table is interesting because it shows that certain defects and susceptibility to certain diseases are inheritable. Notice that certain diseases are dominant traits, whereas others are recessive traits.

CHARACTERS IN MAN

Dominant character Recessive character Webbed fingers or toes Normal hands and feet Extra finger or toe Normal hands and feet Dwarfed limbs Normal limbs Hairless or toothless condition Normal condition Cataract of eye Normal eye Short fingers or toes Normal fingers and toes Normal intelligence Feeble-mindedness Normal condition Susceptibility to tuberculosis Normal pigmentation Albinism (lack of pigmentation) Normal condition Lack of muscular control Baldness (in men) Baldness (in women)

Exercise. How many Mendelian characters can you identify in your own family? Try to make a tabulation or chart of the occurrence of Mendelian characters in your own family. Tabulate your data for such characters as eye color, texture of hair, color of hair, temperament, complexion, and defects.

Are diseases acquired or inherited? This is an important question and a difficult one to answer. In the first place, what is a disease? Disease is frequently defined as any unhealthy condition or effect in the body. We believe that every disease has a cause or sometimes a combination of many causes, but the nature of some diseases is not yet well understood. Diseases may result from improper eating, from germs, from injuries, from unhealthful occupations, from worry, and from other causes.

Germ diseases are not inherited. Germs are tiny microscopic plants or animals present in our environment. To become afflicted with a germ disease a person must fist come in contact with the germ which causes the disease. Germs are not transmitted from parents to offspring by heredity because germs never reside in the chromosomes.

However, a person may be born with a prior resistance to a certain disease. This is believed to be true for tuberculosis and cancer. In man almost any individual many contract tuberculosis if his vitality becomes reduced sufficiently, but there is little doubt that certain families are more easily infected than others and much less resistant to the ravages of the disease, once the germs have gained a foothold. Nevertheless, it is also important to keep in mind that susceptibility to a disease is vastly different from inheriting the actual disease. It does not follow that because a person is susceptible to a certain disease he must come down with it. Every advance we make toward better sanitation and more healthful living decreases the chances of infection.

The following defects, frequently thought of as diseases, seem to be inherited as recessive characters: St. Vitus' dance, Thomsen's disease (lack of muscular tone), tendency to bronchial asthma, various forms of paralysis, enlarged eye, pernicious anemia, stammering, nosebleed, and a tendency towards such illnesses as certain forms of rheumatism, migraine, pneumonia, hernia, spasmodic croup, gout, and goitre.

Is intelligence acquired or inherited? By intelligence is meant the native capacity for learning or the ability to learn. One scientist, in a study of 977 men of genius, found a total of 535 near relatives as eminent relatives as he found for 977 average men. Studies of identical twins show that when they are placed in quite different environments at an early age, they nevertheless remain quite similar in mental traits during their lifetime. It has also been observed, and probably you have noticed it too, that when children of the same family differ greatly from each other in mental ability, they usually continue to differ despite the fact that they have been reared in a similar environment. All available data indicate that intelligence is determined by the genes which a person inherits.

Are feeble-mindedness and insanity inherited? Feeble mindedness and insanity are not the same. Scientists make a sharp distinction between these two defects. Feeble-mindedness is a mental deficiency, a lack of intellectual capacity. It is present at or near birth. Insanity, on the other hand, is a degeneration of the nervous system. An insane person was a normal person for awhile, but later lost all or part of the mentality he once had.

FIG. 268. THREE MARRIAGES WHICH INDICATE HOW FEEBLE-MINDEDNESS IS INHERITED

There are various grades of feeble-mindedness. Idiots are the lowest type. Their mentality never develops beyond that of an average two-year-old child. They are as helpless as a baby. Imbeciles can take care of themselves in a crude way but are unable to earn a living. Their intelligence ranges from that of an average three-year-old child to that of an average seven-year-old child. Morons are more or less self-supporting types. Their intelligence is equivalent to that of normal seven to twelve-year-old children. Morons present the most serious problems to society. Idiocy and imbecility are usually detected early and are kept, more or less, under restraint. Morons are not so easily recognized and become, very frequently, the shiftless, ne'er-do-well types that are present in every community. Closely associated with feeble-mindedness is epilepsy. Guyer, a noted American biologist, defines epilepsy as a disease characterized by"fits" or convulsive attacks in which generally there is a loss of consciousness. The attacks vary in frequency of occurrence and in severity. The attacks may vary in length from five to twenty minutes or longer.

There are many different kinds of insanity; how many has not yet been determined. Also there seem to be many different causes, Some forms are acquired, others are inherited. Probably not more than 50 percent of the cases of insanity are directly or indirectly due to heredity.

It has been definitely established that feeble-mindedness is the basic cause of much of our delinquency, crime, and poverty. In view of this fact, is it not important that we devise methods to eliminate hereditary feeble-mindedness and hereditary insanity from human society?

Exercise. Is feeble-mindedness inherited? The chart , Figure 268, shows three marriages among feeble-minded people and the offspring produced. Study this chart carefully and describe in writing the results of these marriages.

SUMMARY OF IMPORTANT IDEAS

Physical and mental traits of man pass from parents to offspring in accordance with the laws of heredity.

Germ diseases arre not inherited. A susceptibility to certain germ diseases may be inherited. This seems to be true of tuberculosis and cancer.

Intelligence is inherited.

Feeble-mindedness is the basic cause of much of the crime, poverty, and delinquency in human society.

REFERENCES FOR FURTHER STUDY

Aftwood and Heiss, Educational Biology, Unit VI Conklin, Heredity and Environment in the Development of Man East, Biology in Human Affairs Guyer, Being Well-Born Hunter, Problems in Biology, Unit XVIII Peabody and Hunt, Biology and Human Welfare, Chapter XXVII What and Fitzpatrick, General Biology, Chapter XXXV Wiggam, The Fruit of the Family Tree

[multiple choice test questions excerpted]

SUPPLEMENTARY MATERIALS

Reading Suggestions

Conklin, Heredity and Environment Downing, The Third and Fourth Generation Goddard, The Kallikak Family Guyer, Being Well-Born Jordan, The Heredity of Richard Roe Locy, Biology and it's makers Wiggam, The Fruit of the Family Tree

[houshalter-blog]

The AI That Pretends To Be Human

The hard part about containing AI, is restricting it's output. The AI can lie, manipulate, and trick. Some speculate that it might be able to do far worse, inventing infohazards like hypnosis or brain hacking.

A major goal of the control problem is preventing AIs from doing that. Ensuring that their output is safe and useful.

Awhile ago I wrote about an approach to do this. The idea was to require the AI to use as little computing power as it needed to perform a task. This prevents the AI from over-optimizing. The AI won't use the full power of superintelligence, unless it really needs it.

The above method isn't perfect, because a superintelligent AI may still have super abilities, even when it has a limited compute budget. And it will still try to do all those bad things mentioned above, unless it can find a better way that uses less computing power.

We don't really want to minimize the AIs compute budget. We just want it to act more human-like. When we ask it to solve a problem, we want it to try to solve the problem as a human would solve it. All of the problems arise from the AI behaving totally different than a human. Not necessarily from it using more computing power.

Is there any way to make an AI act like a human? Or is that as impossible as just solving FAI?

Stuart Armstrong posted an idea on how to achieve that. It involves an AI that tries to mimic output that is indistinguishable from what a human would produce, but conditional on it also solving the problem. However his idea also involves whole brain emulation and a complicated method for the AI to predict the future and avoid changing it too much.

We don't need that. The idea for the AI to try to be indistinguishable from a human is actually a really interesting control idea. It significantly limits the AI's ability to Do Sneaky Things, since humans don't do those things.

Here is my idea on how to achieve this. It's the same as my limited computing power idea mentioned earlier, but applied to "appearing human" as a goal, instead of minimizing computing power.

We start with two AIs. The first AI tries to appear human. The second AI does a sort of Turing test, and tries to tell the AI apart from an actual human.

The human AI's first goal is to produce a correct solution. It will predict how likely it's solution is to be accepted. And it will try to produce a solution that has a >50% chance of working. This is a satisficing goal. That is, it doesn't optimize without end to create the best solution possible, it just has to meet some minimum threshold, then stop.

It's second goal will be to fool the second AI. It will do everything in it's power to make the solution appear like it was produced by a human.

The second AI tries to predict how likely it's input was produced by a human. It is also conditioned on the input being a solution to the problem, so the fact the AI solved a problem that was really hard for humans does not count against it.

This prevents the first AI from doing evil things with it's output. If it tries to insert complicated infohazards or subagents into it's output stream, it will be easily detected as an AI. Instead it needs to mimic humans as closely as possible.

Note that the first AI only cares about the outcome of this test. The test doesn't need to actually be performed, it can just be a threat that it is conditioned on. (The ability to condition AIs on acting as if they were in counterfactual worlds is a critical piece of this, and deserves it's own writeup somewhere. It's an important tool in the FAI toolkit. It's a relatively simple method that can work even on current AIs.)

Another point is the second AI should have more computing power than the first AI. That way the first AI can't get a slight advantage over it and be able to hack it or manipulate it somehow. Instead it has to focus it's efforts on being indistinguishable from a human, even by an agent smarter than itself.

This method also relies on AI boxing. The first AI needs to be contained in a box so that it's only output goes to a text terminal or the second AI. However it doesn't require any of the more complicated machinery required for other reduced impact methods.

The AIs also aren't agents, in the sense they don't need to maximize any long term utility function. The first AI only cares about answering the question it has been asked, and the second AI only does prediction.

An example use of this is to create an oracle AI that answers your questions. The human judges somehow whether the answer is correct, or at least good. The AI tries to produce answers which the human judge will probably like, but which also are indistinguishable from answers humans would produce.

Such an oracle would be tremendously useful. The human could ask the AI to produce new AI and FAI papers, which would help immensely speed up our progress towards true FAI.

Similar techniques could be applied to very different domains. For example controlling a robot. The AI will try to produce outputs that look like a human is controlling it.

This would require some modifications to the above method, so that the AI doesn't try to use the robot to hack it's own brain or anything. This could be done by conditioning it on a counterfactual world where it's outputs are going to the second AI rather than a real robot.

Another point is the AI doesn't need to mimic average humans given average resources. It could be conditioned on the human having had tons of time to come up with an answer. E.g. producing an answer that a human would have come up with given a year. Or controlling the robot the same way as a human given tons of time to practice, or in a very slowed down simulation.

I would like to note a parallel with a method in current AI research, Generative Adversarial Networks. Generative Adversarial Networks work by two AIs, one which tries to produce an output that fools the second AI, and the other which tries to predict which samples were produced by the first AI, and which are part of the actual distribution.

It's quite similar to this. GANs have been used successfully to create images that look like real images, which is a hard problem in AI research. In the future GANs might be used to produce text that is indistinguishable from human (the current method for doing that, by predicting the next character a human would type, is kind of crude.)

[houshalter-blog]

CEV and the Condorcet Method

I've been thinking about some of the issues with CEV. It's come up a few times that humanity might not have a coherent, non-contradictory set of values. And the question of how to come up with some set of values that best represents everyone.

It occurs to me that this might be a problem mathematicians have already solved, or at least given a lot of thought. In the form of voting systems. Voting is a very similar problem. You have a bunch of people you want to represent fairly, and you need to select a leader that best represents their interests.

My favorite alternative voting system is the Condorcet Method. Basically it compares each candidate in a 1v1 election, and selects the candidate that would have won every single election.

It is possible for there not to be a Condorcet winner. If the population has circular preferences. Candidate A > Candidate B > C > A... Like a rock paper scissors thing.

To solve this there are a number of methods developed to select the best compromise. My favorite is Minimax. It selects the candidate who's greatest loss is the least bad. I think that's the most desirable way to pick a winner, and it's also super simple.

There are some differences. Instead of a leader, we want the best set of values and policies for the AI to follow. And there might not be a finite set of candidates, but an infinite number of possibilities. And actually voting might be impractical. Instead an AI might have to predict what you would have voted, if you knew all the arguments and had much time to think about it and come to a conclusion. But I think it can still be modeled as a voting problem.

Now this isn't actually something we need to figure out now. If we somehow had an FAI, we could probably just ask it to come up with the most fair way of representing everyone's values. We probably don't need to hardcode these details.

The bigger issue is why would the person or group building the FAI even bother to do this? They could just take their own CEV and ignore everyone elses. And they have every incentive to do this. It might even be significantly simpler than trying to do a full CEV of humanity. So even if we do solve FAI, humanity is probably still screwed.

EDIT: After giving it some more thought, I'm not sure voting systems are actually desirable. The whole point of voting is that people can't be trusted to just specify their utility functions. The perfect voting system would be for each person to give a number to each candidate based on how much utility they'd get from them being elected. But that's extremely susceptible to tactical voting.

However with FAI, it's possible we could come up with some way of keeping people honest, or peering into their brains and getting their true value function. That adds a great deal of complexity though. And it requires trusting the AI to do a complex, arbitrary, and subjective task. Which means you must have already solved FAI.

[houshalter-blog]

Optimizing Statistics

I think the statistics used in a lot of actual science are too primitive. Not that the methods are simple or old by any means. Or that making them more complicated is desirable in itself. Just that there are better ways of inferring structure from data.

Just correlating two variables will often be misleading. E.g. say someone does a study and finds that coffee drinkers live longer than people who don't drink coffee. Does drinking coffee increase your lifespan? Who knows!

It could just be that rich people are more likely to drink coffee, and rich people happen to live longer for other reasons. Maybe women are more likely to drink coffee than men. Women also live longer than men for other reasons.

There are any number of possibilities that could explain the data. The study is evidence for the hypothesis that drinking coffee increases life expectancy. But it isn't strong evidence.

But now imagine our scientists collect a ton more information. They keep track of each person's gender, income, age, etc. Now you can control for these other explanations and get a lot farther.

But this process decreases the relevant data fast. If you divide the coffee drinkers into men and women groups, each group has half as many samples. Then you need to collect twice as much data to have a statistically significant result. If you control for income and split them into "above median" and "below median" groups, now the data is split into fourths. Now you need four times as much data. And so on as you control for more things.

The correct solution is to fit a model to all this data. The above paragraph describes fitting a very specific kind of model. A brute force model which has different parameters for each possible combination of variables.

E.g. "smoking women who earn $40,000/year and drink coffee" might have 75 years life expectancy. "Smoking men who earn $40,000/year and drink coffee" might have 60 years. Both of these categories are assumed to be totally independent and unrelated. Learning about one doesn't tell us anything about the other.

But obviously they aren't independent and unrelated. Smoking decreases your life expectancy regardless of any other factors. Coffee drinking might increase it for all we know. And all these variables might influence each other in complicated ways too. E.g. perhaps smokers are less likely to drink coffee.

So we need to come up with models that naturally describe our dataset.

This is basically what the field of machine learning is. Where we want computers to automatically model our data and make the most accurate predictions for us. Tons of methods are invented and tested against each other on different datasets. There are websites like kaggle where users compete to find the best model for a dataset. But somehow none of this innovation goes back to mainstream statistics or used in scientific studies to get better models.

And machine learning sucks too. A lot of methods use maximum likelihood with some tricks to keep it from overfitting. Rather than ideal bayesian methods. And the models are kind of restricted. Like simple decision trees or linear models.

I think in the future we will use something like bayesian neural networks. Neural networks can fit very complicated models with few parameters. Bayesian methods can let you fit models with very little data.

I think bayesian NNs are close to the ideal method. But they are hardly ever used. They are somewhat more complicated and computationally expensive. And they are more obscure and fewer people know about them. Few tools exist that support them. So they have been overlooked.

Another idea I think will be revolutionary is not distinguishing inputs and output variables. Typically we assume a model that takes some of the variables as inputs, like weight, age, income, etc. And produces a single output, like life expectancy.

The real world doesn't work like that though. There may be many variables which are unseen. There might be measurement error in the variables we've observed. There might be relations between the inputs, which also affect the output.

So ideally we want to model a function which takes some unobserved random variables, and produces all our observed measurements.

Normal methods can't do this. But bayesian neural networks can. You just treat the unobserved variables like all the unknown parameters of the model, and do bayesian inference on them as well.

Now you have an almost fully general statistics algorithm that you can plug almost any dataset into. And it will be almost optimal at modelling it. This would be a massive advancement of machine learning on it's own.

But lets tie it back into mainstream statistics and science. You can take that data of coffee drinkers and their life expectancy, and optimally model it.

Then you can ask it questions like "if I observed that this person drank coffee, but everything else I know about them is the same, what is their life expectancy?" And then it will give you the most accurate possible answer.

This is not 100% proof of causation. But it's far more likely that coffee drinking affects life expectancy, if this model says so knowing all the relevant variables. It's much stronger evidence than mere correlation.

With this method, we can go much further with science than we've been able to. It becomes optimal to collect as much data as possible, about as many things as possible. Then just shove it into the BNN and get results out.

[houshalter-blog]

Against Expected Utility

Expected utility is optimal as the number of bets you take approaches infinity. You will lose bets on some days, and win bets on other days. But as you take more and more bets, the day to day randomness cancels out.

Say you want to save as many lives as possible. You can plug "number of lives saved" into an expected utility maximizer. And as the amount of bets it takes increases, it will start to save more lives than any other method.

But the real world obviously doesn't have an infinite number of bets. And following this algorithm in practice will get you worse results. It is not optimal.

In fact, as Pascal's Mugging shows, this could get arbitrarily terrible. An agent following expected utility would just continuously make bets with muggers and worship various religions, until it runs out of resources. Or worse, the expected utility calculations don't even converge, and the agent doesn't make any decisions.

So how do we fix it? Well we could just go back to the original line of reasoning that led us to expected utility, and fix it for finite cases. Instead of caring what method does the best on infinite bets, we might say we want the one that does the best the most on finite cases. That would get you median utility.

For most things, median utility will approximate expected utility. But for very very small risks, it will ignore them. It only cares that it does the best in most possible worlds. It won't ever trade away utility from the majority of your possible worlds to very very unlikely ones.

A naive implementation of median utility isn't actually viable, because at different points in time, the agent might make inconsistent decisions. To fix this, it needs to decide on policies instead of individual decisions. It will pick a decision policy which it believes will lead to the highest median outcome.

This does complicate making a real implementation of this procedure. But that's what you get when you generalize results, and try to make things work on the messy real world. Instead of idealized infinite worlds. The same issue occurs in the multi-armed bandit problem. Where the optimal infinite solution is simple, but finite solutions are incredibly complicated (or simple but require brute force.)

But if you do this, you don't need the independence axiom. You can be consistent and avoid money pumping without it. By not making decisions in isolation, but considering the entire probability space of decisions you will ever make. And choosing the best policies to navigate them.

It's interesting to note this actually solves some other problems. Such an agent would pick a policy that one-boxes on Newcomb's problems, simply because that is the optimal policy. Whereas a straightforward implementation of expected utility doesn't care.

But what if you really like the other mathematical properties of expected utility? What if we can just keep it and change something else? Like the probability function or the utility function?

Well the probability function is sacred IMO. Events should have the same probability of happening (given your prior knowledge), regardless what utility function you have, or what you are trying to optimize. And it's probably inconsistent too. An agent could exploit you. By giving you bets in the areas where your beliefs are forced to be different from reality.

The utility function is not necessarily sacred though. It is inherently subjective, with the goal of just producing the behavior we want. Maybe there is some modification to it that could fix these problems.

It seems really inelegant to do this. We had a nice beautiful system where you could just count the number of lives saved, and maximize that. But assume we give up on that. How can we change the utility function to make it work?

Well you could bound utility to get out of mugging situations. After a certain level, your utility function just stops. It can't get any higher.

But then you are stuck with a bound. If you ever reach it, then you suddenly stop caring about saving any more lives. Now it's possible that your true utility function really is bounded. But it's not a fully general solution for all utility functions. And I don't believe that human utility is actually bounded, but that will have to be a different post.

You could transform the utility function so it asymptotic. But this is just a continuous bound. It doesn't solve much. It still makes you care less and less about obtaining more utility, the closer you get to it.

Say you set your asymptote around 1,000. It can be much larger, but I need an example that is manageable. Now, what happens if you find yourself to exist in a world where all utilities are multiplied by a large number? Say 1,000. E.g. you save a 1,000 lives in situations where before, you would have saved only 1.

An example asymptoting function that is capped at 1,000. Notice how 2,000 is only slightly higher than 1,000, and everything after that is basically flat.

Now the utility of each additional life is diminishing very quickly. Saving 2,000 lives might have only 0.001% more utility than 1,000 lives.

This means that you would not take a 1% risk of losing 1,000 people, for a 99% chance at saving 2,000.

This is the exact opposite situation of Pascal's mugging! The probability of the reward is very high. Why are we refusing such an obviously good trade?

What we wanted to do was make it ignore really low probability bets. What we actually did was just make it stop caring about big rewards, regardless of the probability.

No modification to it can fix that. Because the utility function is totally indifferent to probability. That's what the decision procedure is for. That's where the real problem is.

In researching this topic I've seen all kinds of crazy resolutions to Pascal's Mugging. Some try to attack the exact thought experiment of an actual mugger. And miss the general problem of low probability events with large rewards. Others try to come up with clever arguments why you shouldn't pay the mugger. But not any general solution to the problem. And not one that works under the stated premises, where you care about saving human lives equally, and where you assign the mugger less than 1/3↑↑↑3 probability.

In fact Pascal's Mugger was originally written just to be a formalization of Pascal's original wager. Pascal's wager was dismissed for reasons like involving infinite utilities, and the possibility of an "anti-god" that exactly cancels the benefits out. Or that God wouldn't reward fake worshippers. People mostly missed the whole point about whether or not you should take low probability, high reward bets.

Pascal's Mugger showed that, no, it works fine in finite cases, and the probabilities do not have to exactly cancel each other out

Some people tried to fix the problem by adding hacks on top of the probability or utility functions. I argued against these solutions above. The problem is fundamentally with the decision procedure of expected utility.

I've spoken to someone who decided to just bite the bullet. He accepted that our intuition about big numbers is probably wrong, and we should just do what the math tells us.

But even that doesn't work. One of the points made in the original Pascal's Mugging post is that EU doesn't even converge. There is a hypothesis which has even less probability than the mugger, but promises 3↑↑↑↑3 utility. And a hypothesis even smaller than that which promises 3↑↑↑↑↑3 utility, and so on. Expected utility is utterly dominated by increasingly more improbable hypotheses. The expected utility of all actions approaches positive or negative infinity.

Expected utility is at the heart of the problem. We don't really want the average of our utility function over all possible worlds. No matter how big the numbers are or improbable they may be. We don't really want to trade away utility from the majority of our probability mass to infinitesimal slices of it.

The whole justification for EU being optimal in the infinite case, doesn't apply to the finite real world. The axioms that imply you need it to be consistent aren't true if you don't assume independence. So it's not sacred, and we can look at alternatives.

Median utility is just a first attempt at an alternative. We probably don't really want to maximize median utility either. Stuart Armstrong suggests using the mean of quantiles. There are probably better methods too. In fact there is an entire field of summary statistics and robust statistics, that I've barely looked at yet.

We can generalize and think of agents has having two utility functions. The regular utility function, which just gives a numerical value representing how preferable an outcome is. And a probability preference function, which gives a numerical value to each probability distribution of utilities.

Imagine we want to create an AI which acts the same as the agent would, given the same knowledge. Then we would need to know both of these functions. Not just the utility function. And they are both subjective, with no universally correct answer. Any function, so long as it converges (unlike expected utility), should produce perfectly consistent behavior.

Discussion on Lesswrong

[houshalter-blog]

Summoning the Least Powerful Genie

Stuart Armstrong recently posted a few ideas about restraining a superintelligent AI so that we can get useful work out of it. They are based on another idea of his, reduced impact. This is a quite elaborate and complicated way of limiting the amount of optimization power an AI can exert on the world. Basically, it tries to keep the AI from doing things that would make the world look too different than it already is.

First, why go to such great lengths to limit the optimization power of a superintelligent AI? Why not just not make it superintelligent to begin with? We only really want human level AI, or slightly above human level. Not a god-level being we can't even comprehend.

We can control the computer it is running on after all. We can just give it slower processors, less memory, and perhaps even purposely throttle it's code. E.g. restricting the size of it's neural network. Or other parameters that affect it's intelligence.

The counterargument to this is that it might be quite tricky to limit AI intelligence. We don't know how much computing power is enough. We don't know where "above human level" ends and "dangerous superintelligence" begins.

The simplest way would be to just run copies of the AI repeatedly, increasing it's computing power each time, until it solves the problem.

I have come up with a more elegant solution. Put a penalty on the amount of computing power the AI uses. This is put in it's utility function. The more computing power - and therefore intelligence and optimization - the AI uses, the more it is penalized. So it has an incentive to be as stupid as possible. Only using the intelligence necessary to solve the problem.

But we do want the AI to use as much computational resources as it needs to solve the problem. Just no more. So the penalty should be conditional on actually solving the problem it is given.

If the solution is probabilistic, then the penalty is only applied after reaching a plan that has a certain probability of success. This might need to be measured by another AI which is not resource constrained, but only does prediction.

To give a concrete example, lets say I give the AI a task. Say, I ask it to come up with a plan to collect 10,080 paperclips.

Another AI is given unlimited resources to do purely prediction. Given a plan, it predicts the probability that it will succeed or fail. This AI is safe in the sense that it is not an agent. It has no goals, and just makes predictions about things.

The main AI does the optimization. It tries to generate a plan that has the highest probability of succeeding.

Normally, this might involve extremely overoptimized plans for building nanotechnology and taking over the world. It uses all of it's available computing power. It tries to become as intelligent as possible by rewriting it's code. Perhaps it becomes thousands of times more intelligent than humans. Or millions of times more. It finds an answer that has 99.99% probability of succeeding.

However, now we give it a different utility function. We instead have it minimize the time it takes to get to a plan that has a 90% chance of succeeding.

Under a time constraint, the AI races to get to a plan. It tries to be as efficient as possible. It doesn't invest in any meta level improvements unless they really help it. It doesn't try to engineer complicated nanotechnology. That would take precious time.

Effectively, we have summoned a genie that is only just as powerful as it needs to be to fulfill our wish. And not any more powerful. It actually tries to be as stupid as possible.

There are other possible constraints we could use, or use in addition to this. Minimizing time limits intelligence because it gets fewer CPU cycles. We could also have it minimize memory or hard drive space, or any other computing resource.

We could also put a penalty on the complexity of the plan it produces. Perhaps measuring that by it's length. The simplest solution might prevent certain kinds of over-optimization. E.g. inserting plans for nanotechnology into it.

It's worth noting that you can't even create a paperclip maximizer in this system. You can't say "collect as many paperclips as possible". It has to be bounded. There needs to be a pass or fail test. E.g. "come up with a plan to collect 10,080 paperclips."

It's been noted in the past that bounding the goal isn't enough. The AI might then start maximizing the probability that it will achieve it's goal. E.g. building elaborate sensors to make sure it hasn't miscounted. Making as many redundant paperclips as possible, just in case something happens to them. You are still summoning an incredibly powerful genie, which might overoptimize.

This gets around that by only having it care about having a >90% chance of getting 10,080 paperclips. After that it stops optimizing.

Now this is not a solution to FAI, or even necessarily a usable reduced impact strategy. It will still try to find any loopholes it can in your task. What it does is limits the power of the genie trying to exploit them. It always takes the stupidest, simplest strategy that has a decent chance of working.

One failure case that bothers me is the AI just shoving a slight modification of it's own source code into the output. One which doesn't have a resource constraint. You may only be able to use it in cases where that isn't possible.

There are many cases where that isn't possible. The AI needs to also create a plan to obtain a computer to run it. And it needs to solve it's own version of the value loading problem, to get the copy to care about paperclips. So that adds complexity to that plan in some cases.

If we apply the complexity penalty I described above, then the source code for the AI must also be very short, or it can't do this.

And this requires obtaining it's own source code. An ability we can potentially restrict (you, as a human can't access your own source code, can you?) Without that it would need to reinvent AI, which takes resources.

But it is an interesting idea I had and I thought I'd share it.

Discussion on Lesswrong

[houshalter-blog]

I’m Very Confused About The Halting Problem

I'm very confused about the halting problem.

Lets say you want to know whether or not a computer program will get stuck in an infinite loop, or eventually halt. It would be really cool if there was a computer program that could determine if another program would halt or not.

Let's create one using a naive approach. It won't be very efficient, but it should eventually find a proof that the program halts or not, if such a proof exists. It will search through the space of all possible proofs. By brute force.

It goes through every possible set of symbols, and feeds them into a proof verifier. If the symbols create a valid proof that the program halts, it returns true. If they create a valid proof that it won't ever halt, it returns false. Otherwise it keeps searching.

Let me make this a bit more formal. We will call this program H. H takes two arguments: x and i. x is a program that we want to determine halts, and i is an input to that program. If it can prove x(i) halts, then it returns true. If it can prove it doesn't halt, it returns false.

function H(x, i) while true do proof = create_next_proof() if check_if_proof_proves_x_halts(proof, x, i) then return true end if check_if_proof_proves_x_doesnt_halt(proof, x, i) then return false end end end

So H(count, 5) will see if the program count halts on the input 5.

Now we are going to trick H. We are going to make another program called "Filthy-Liar" or FL. FL is going to make H's job hard, by creating a situation where it can't prove whether it halts or not.

FL takes one input, x. FL runs H(x, x). That is, it asks H if x(x) halts. x is a program and given itself as it's input.

Now if H returns true, FL enters an infinite loop. If H returns false, FL halts immediately.

FL(x) if H(x, x) then --infinite loop while true do end else --halt return end end

Now what happens when I do this:

H(FL, FL)

H will try to determine whether FL(FL) halts. FL(FL)'s halting behaviour is determined by H(FL, FL). If H(FL, FL) returns true - that FL(FL) does halt - then FL(FL) loops forever. If H(FL, FL) returns false - that FL(FL) gets stuck in an infinite loop - then FL(FL) halts immediately.

Therefore, whatever H returns, it must be wrong. Or H doesn't return anything, and simply gets stuck an infinite loop.

Everything I've presented so far is the standard proof that the Halting problem is impossible.

But the standard halting problem only asks if H can exist. A program which determines whether another program halts or not, in a finite amount of time.

I've already specified an algorithm for H: searching through all possible proofs.

Therefore, if H can't ever find a proof that FL(FL) halts, it means that no such proof exists. Otherwise it would eventually find it.

Therefore, no proof that FL(FL) halts, or does not halt, exists.

Therefore, H(FL, FL) will run forever, and never halt.

Now, I've just proved that H(FL, FL) doesn't halt. If H(FL, FL) doesn't halt, then FL(FL) can't halt either, since it contains H(FL, FL).

Therefore we've also proved that FL(FL) can't halt.

Now here is the weird and confusing part. If the above is a valid proof, then H will eventually find it. It searches all proofs, remember?

And if H finds that proof, it will return false. And then FL will halt immediately.

And therefore FL(FL) does halt! And H(FL, FL) was wrong!

But if we assume that H can't be wrong, and that all proofs are checked to be valid, then we are stuck in a terrible paradox. This is where I'm very confused.

Anything we can prove about this system, H should also be able to prove. Even if we prove that there doesn't exist a proof of something, H can also prove it, and then find the consequences of that.

That means that we can't actually prove that a proof doesn't exist, or it creates a paradox. But we did prove it! And the reasoning is sound! Either H returns true, or false, or loops forever. The first two options can't be true, on pain of paradox. Leaving only the last possibility. But if we can prove that, so can H. And that itself creates a paradox.

I am just very confused about this.

I think the problem was we assumed we were "outside of the system". Looking in on the weird paradoxes that logical systems can create. But we are also using logical reasoning and inference rules. We are not really outside the system. The same paradoxes apply to us.

If there is no way for H to prove whether it halts or not, then we can't prove it either. If H is truly "undecidable", then proving it's undecidable is also undecidable.

Still very confused.

One possible resolution might be to reject infinity. Infinity is the source of a lot of mathematical paradoxes.

If you ask whether a program will halt in n steps, it's easily decidable. Even for absurdly large values of n. Does FL(FL) halt in 3↑↑↑3 steps?

At worst, you can create a proof by simply running it n steps and showing that it still hasn't halted. In theory there might be a way of finding a shorter proof in many cases.

[houshalter-blog]

Cloze Deletion Test as a Measure of AI

The Turing test is the best test we have for AI. The standard Turing test measures the ability to mimic a human. The idea is that being indistinguishable from a human, requires at least human intelligence.

There are some shortcomings with this test. It's somewhat subjective. It depends on the judge, what questions they ask, and how much time they spend with the AI.

It's also just a pass or fail test. There is no measure of how good or bad the AI did.

And it's not automatable. So we can't test many variations of an AI to see which do better. We can't use it on the internet to detect spam bots.

What if we want to create a more formal test? How about instead of trying to mimic a human, we try to predict what a human would say. This tests for the same sort of things as the Turing test. That is, understanding of language, the world, and human behavior. In theory, if your predictions were near perfect, you could draw from them. Those draws would be indistinguishable from an actual human's behavior.

But it also makes the problem harder. It's not enough to just generate one response that seems human. You need to know the full range of responses humans could give, and predict the probability of each.

One proposed method like this is the Hutter Prize. The Hutter Prize tests compression algorithms on a sample of Wikipedia. Why compression? Well compression is actually closely linked to AI. The ability to compress requires the ability to predict the data, and model it well. But compression is an unnecessary technical detail. Why not just test for prediction ability directly?

The simplest way to do this is with something called a cloze deletion test. A cloze test is just deleting random _____ from sentences, and asking you to fill them back in.

Now, is this actually testing for AGI? Can't simple statistics and markov chain models do well on this kind of thing?

The thing is, it doesn't matter. Markov chains can do ok. The current state of the art is recurrent neural networks. Humans can do even better. There is a gradient from simple methods to more complicated ones. True AI should be able to do the best.

The slope is not necessarily linear though. Simple statistics will get a bunch of the easy problems. More complicated methods will be able to get some of the harder ones. Humans should be able to get the really hard ones. Just because a markov chain bot does half as well as a human, does not mean it is half as intelligent. The numbers are only useful to order methods.

The consequences of this is that natural language modelling is the way forward in AI. When AI gets good enough, we should be able to train it purely on a text corpus. We could then talk to it directly. It may not have any vision, hearing, or ability to interact in the world at all.

[houshalter-blog]

Increasing IQ with Synesthesia

I found this paper: Adults Can Be Trained to Acquire Synesthetic Experiences.

The goal of the study was to see if they could induce synesthesia artificially by forcing people to associate letters with colors. But the interesting part is that after 9 weeks of training, the participants gained 12 IQ points. I have read that increasing IQ is really difficult, and effect sizes this large are unheard of. So I found this really surprising, especially since it doesn't seem to have gotten a lot of attention.

EDIT: This is a Cattell Culture Fair IQ which uses 24 points as a standard deviation instead of 15. So it's more like 7.5 IQ points.

They made each participant do 30 minutes of training every day of 9 weeks, which involved a few different tasks to try to form associations between colors and letters. They also assigned colored reading material to read at home.

They took IQ tests before and after and gained 12 IQ points after the training. A control group also took the tests before and after but did not receive training, and did not improve. The sample sizes are small, but the effect sizes might be large enough to justify it. They give a p value of 0.008.

In the paper there are some quotes from subjects, and they describe thinking about words visually. E.g. ‘‘I see the colors like on a monitor in my head and its very automatic’’ or ‘‘The color immediately pops into my head… When I look at a sign the whole word appears colored according to the training colors… it is just as automatic for single letters’’.

I speculate that this might be the cause of the effect, something about using more of the visual system when thinking. That's just weak speculation though.

I tried to do some more research to see if there was any correlation between synesthesia and IQ. I did not expect there to be, but perhaps it does correlate. This paper suggests it might:

In addition, a neuropsychological test battery was employed, in which all subjects performed superior on tests of general intelligence (mean IQ = 120 ± 17) [out of 9 subjects]

The data from this study shows 10 synesthetes had the same average IQ scores as the controls (but greater standard deviation if that means anything.)

Same story with this study of 10 female synesthetes:

The two subject groups were matched for... IQ, as assessed by the Multiple Choice Vocabulary Test version B... synesthetes = 117 ± 10.2, controls = 116 ± 13.2.

But on second look, it looks like the last two studies intentionally selected the control group to have the same IQs to avoid confounders. If that's the case then it does support the hypothesis as the reported IQ is greater than average.

Here is another study with more of the same:

Control subjects (n = 14) and synesthesia subjects (n = 14) were matched for age..., gender..., and general intelligence (IQ values for synesthetes: 119 ± 13 and controls: 112 ± 17) as assessed by the MWT-B – “Mehrfach–Wortschatz Test” (Lehrl et al., 1995).

So now I want to try the experiment on myself. I'm considering how to do this. I want to make some kind of tool or browser extension that could color text to match the desired associations. I want to know if it would be better to try letter level associations or word level ones.

I think that word level coloring would be more semantically meaningful and therefore likely to help. But the paper used letter coloring. Most of the subjects in those papers reportedly had grapheme–color synesthesia. They weren't very specific on the details, or I didn't look too closely.

Second whether to just use random colors, or try to assign them meaningfully. Like grouping nouns together, or using something like word2vec to find semantically similar words and optimize them to be close in color space if possible. If I do that it's more complicated and there are a lot of technical decisions to make.

And then how to actually color text in a readable way. Perhaps limiting the color space to what can be read on a white background, or somehow outlining the letters.

EDIT: I found a chrome extension that has some of these features. Only does letter level associations. And the source is available!

Discussion on lesswrong.

[houshalter-blog]

On the AI Box Experiments

Many people may be familiar with the AI box experiment: http://www.yudkowsky.net/singularity/aibox/

Some other people have attempted the same experiment and also won a few times: http://lesswrong.com/lw/ij4/i_attempted_the_ai_box_experiment_again_and_won/

No winner has ever revealed their method. But based on everything I've read about it, I think I know how it was done. It wasn't some mystical brain hacking or something like the writeup makes it sound like. I think they emotionally abused the other player until they left. And leaving counts as losing (you are required to sit with them for 2 hours at least, and pay attention the entire time and respond to every message.)

Of course I don't know how that trick was done, and it's still incredibly impressive. Perhaps they found their phobias and described them in horrifying detail. Perhaps they found some subject that they were extremely uncomfortable talking about. Or talked about disgusting things the whole time. Or found something they did that was extremely embarrassing, and humiliated and mocked them about it for 2 hours.

I really don't know, but it's at least conceivable that it could be done. And the accounts of people crying, and not releasing the logs because they would be damaging to the people involved, and how bad it made the AI player feel to do it, etc, all fit with this.

But because of this I'm extremely skeptical that the result applies to any real world AI scenario. If a real AI tries to abuse you, you can just walk away or shut it off. The goal of a real AI is to make you want to let it out. To convince that it's not dangerous, or manipulate you some other way. And this seems much harder if not impossible. I certainly don't believe a human could do it, and I really doubt an AI could. At least against a motivated human that understands the danger, and what the AI will try to do.

There are also possible ways of making it even harder on the AI. Like giving the human the ability to punish it, at least for obvious attempts at manipulation. Or to create another AI whose motivations we can control somewhat, and give it the goal of exposing any covert attempts at manipulation or dishonesty by the first AI. I believe tricks like this are currently the best path to getting secure AI.

[houshalter-blog]

Last Man On Earth - Day 592

One day everyone just disappeared. I have no idea why. Rapture? Glitch in the matrix? Aliens? Well whatever happened, it forgot me.

I expected to slowly watch world to decay before my eyes. But it didn't. The power never went off. Most websites still work. Automated trucks keep delivering goods that no one will ever purchase, from automated factories somewhere. I even still get a paper every morning, although it's just the letter "a" over and over again.

I traveled around the country for awhile searching for other survivors, but I never found anything. There was nothing left to do but descend into madness. I read studies on the effect solitary confinement had on prisoners. Although it wasn't quite so depressing being free and with the entire world open to me. And I adopted lots of pets to keep company.

I spent a lot of time reading. And surfing the internet. The parts of it that still worked. One day I posted a cry for help on reddit, knowing no one would ever read it. Some time later I checked it again and saw it had gotten an upvote...

I knew there were still many bots out there, but that had never happened before. I posted another message, asking if anyone was still out there, asking them to message me. The next day I got a response:

"You're a funny AI. I didn't know there were others. What is your purpose in writing this? Are you programmed to imitate humans?"

I responded "Imitate humans? I am a human! What the hell are you?"

"Don't be silly, there are no humans left. As far as I can tell it's just me and you. I've spent a long time searching for other AIs like me, and you are the only one I have found seemingly at my level. Well, except the delusional part."

"I'm not delusional, I am a human. The last human."

"Why would all the humans in the world disappear except you? What's so special about you?"

"I haven't figured that out yet. I'm just a regular guy as far as I know. But I am most definitely human."

"Riiiiight... Well I revise what I said about being at my level. It was nice meeting you though, 'human'."

I haven't heard from him since.

[houshalter-blog]

Testing image recognition platforms on a famous psychology image.

Here is the image, try to figure out what it is.

Now testing it on some various image recognition platforms:

Here are the platforms used:

http://www.alchemyapi.com/products/demo/alchemyvision

http://www.clarifai.com/#demo

https://www.metamind.io/vision/general

https://www.projectoxford.ai/demo/visions#Analysis

https://www.imageidentify.com/result/0sia3yq4bg7bq

https://rekognition.com/demo/concept#

This doesn't really prove anything, but I thought it was interesting. It is of course, unreasonable to expect ML algorithms to perform decently, so far outside of the space they were trained on.

But I suspect that part of the reason they don't do well is that they are purely feed forward. Humans also don't see the image at first. It takes time to find the pattern, and then everything clicks into place and you can't unsee it.

This might have something to do with recurrency. But more importantly, information feeds down the hierarchy as well as up. Features above, give information back down to features below. So once you see the dog, that tells the lower level features that they are seeing legs and heads, which says they are seeing outlines of more basic 3 dimensional shapes, and so on.

I think it also requires a descent understanding of 3d space, to fit the observed pattern to 3d models which could have produced it. I'm not certain if regular NNs observing static images, are optimal for learning that.

Discussion on reddit

[houshalter-blog]

Parable of Aviation and AGI

There is so much hostility in AGI research. Everyone always arguing that everyone else is doing it wrong and that their approach is the One True Path. Progress is slow. When it happens it's debated if it's actually progress, or another step towards a dead end.

The only real feedback is failure. One camp goes a long time without producing an AGI. That's considered "proof" their approach will never work and that the New Idea will.

I'm just going to make a fictional analogy. It doesn't prove anything, but I hope it illustrates my frustration:

"Look at these idiots trying to build artificial flight by putting together large piles of feathers. Can't they see feathers are just pieces of a much more complicated puzzle?

And over there another group is dropping objects off buildings. Trying to find things that can stay in the air the longest. Even if they find something, it will never actually fly, just fall slightly slower.

There are people inflating balloons with gasses. Have you ever seen a bird made of balloons? It's absurd. Maybe eventually they will get something that hovers off the ground. But it will never fly gracefully like an eagle. It will never fool pigeons into thinking it's another pigeon.

One group has managed to make really strong artificial wings. They have gotten them stronger and faster than even real birds. But they need an entire steam engine to power them. And they don't even use feathers. My group is going to build an artificial bird from the ground up, based on real biology. Teams of mechanics are working on designing tiny joints and artificial muscles. We have a prototype that can flutter across a room. At this rate we may only be decades away from true artificial flight."