Watching the “you will excel at what you measure” trap devour basic moral practice in real time is fascinating in a terrible kind of way
If you spend any significant amount of time studying any social science or people-related policy, you’ll quickly run into the old adage “you will excel at what you measure”. This adage is a warning.
In order to mark progress in any area, we need a way to measure it. So we develop systems to measure complex social systems and behave accordingly. If you want to measure how effectively children are being educated, you can, for example, decide on what they should know by a given age, test them on that knowledge, and grade them in accordance to how well they do on the tests. A higher grade means a more successful student, a better teacher, a better school. Then you can tinker with what you’re testing as necessary, and with teaching methods and soforth to see how it affects scores on the tests.
Except, if you do this, then you’ve defined successful education as the ability to get high grades. You invite cheating (on the student, teacher and even school level), you invite teaching to the test rather than for general comprehension and ability, you invite boiling down the experience of education to test scores. And, of course, you invite massively increasing the inaccuracies caused by some people simply being better at taking tests than others. Someone with low to moderate comprehension who’s good at tests might get a higher grade than someone who understands the material but has anxiety or is unable to properly intuit the meaning of vague test questions. Grades can go up and up and up, while education consistency and quality falls.
This is, as anyone who’s worked in a school or sends their children to school knows, a known problem. ‘Grading systems cause huge problems in education” is NOT by any means a revolutionary and controversial statement. Over time, grading systems have been changed to favour testing comprehension and skill demonstrations, Individual Learning Plans and testing accommodations have become very popular to give a more accurate idea of people’s abilities, and soforth. A good half of my teaching degree was about compensating for the problems in this system. But you can’t patch up all the holes, and the pressure from people taking letter grades way too seriously – parents, school boards, funding systems, those looking to hire teachers – are always going to cause problems, make teaching to the test a matter of survival. We measure grades, so that is what we excel at.
The same problem exists in economics. Most countries measure their health via Gross Domestic Product (GDP). This is basically a measure of how much money is swilling around in there and it’s an AWFUL yardstick. A country full of sick, desperate people going into massive medical debt has a higher GDP than an identical country not facing a health crisis, for instance. But it is the dominant model, so it’s what investors look at, it’s what other countries look at, it’s what voters look at. It’s what you must excel at, to be considered to have a ‘good’ economy. Other models exist, and are often proposed as a better alternative, but if one of those were dominant, new problems would exist – we’d excel at what they measure, and drop in what the GDP measures, and cause new economic issues. If you boil a system down to measurements, you will excel at making those measurements go up.
You should never, ever let yourself fall into the trap of believing that they tell you anything useful about how the system is doing.
Morality and justice are social technology. They’re a bunch of rules and instincts that both evolution and cultural education have given us to allow us to operate in societies. They’re integral to societies in the same way that math is; you need math complex enough to measure the grain, you need morality complex enough to measure the social harmony. People pretend they’re more than that, but they aren’t. “Good” and “bad” are concepts as real as “millionaire” and “straight-A student”, and nothing more.
In the vast, vast majority of societies out there, the end goal is essentially the same – to minimise harm to the populace. They want everyone to have as much safety and comfort as possible. Most disagreements are about the relative value of different individuals (is one race, religion or culture more important than another? Is one sex more important than another? Is a king more important than a slave?), or about methodology (is it better for everyone to have to follow strict social norms, or for everyone to be free to express themselves how they choose; which creates more safety and harmony? What social norms are best? How much control should one have over one’s property, or one’s animals, or one’s children? When somebody transgresses, what is the appropriate system for judging and metering out discipline? What is the appropriate sort of discipline?). People disagree radically on both relative individual value and on methodology, but the general goal is the same. Morality and justice are social technology, tools to be used. Law and social consequence is how their power is enacted.
People often forget this. And that is very, very dangerous.
People will decide on what is ‘good’ and ‘bad’ behaviour, isolate it from the system, and proceed to excel at what they measure. They’ll decide that ‘good people’ use certain language and have certain values and ‘bad people’ use other language and do bad things, they’ll look at harmful power dynamics and decide that the world is full of ‘oppressors’ (can be ignored) and ‘oppressed’ (must be supported), ‘abusers’ (should be mocked and attacked) and ‘abused’ (should be believed and coddled), and stumble blindly forward like my robovac with a dirty sensor bumping into every wall in their way. They’ll see a complex social situation and instead of going ‘what’s the best way to reduce harm?’, immediately try to decide who involved is more oppressed and get their answer from that. They’ll see people use language they don’t like and decide that person must have nothing of value to add to a conversation, because they’re a bad person.
Today, I saw someone muse that the fact that American football causes huge amounts of brain damage that compounds over many years might contribute to why USA footballers seem to keep doing random unhinged things. Somebody else immediately attacked them because rape and domestic abuse is common among footballers (footballers being the attackers), so by suggesting a physical reason for unstable behaviour, this person was making excuses for rape. You might notice that this response has absolutely nothing to do with protecting people from rape or domestic abuse, and absolutely everything to do with making sure nobody might accidentally sympathise with a ‘bad person’ by suggesting that brain changes change behaviour. A focus on minimising harm would want to explore this, because removing risk factors for causing rapists means less rapists. Less rape is the goal. ‘Rape is evil’ is the tool used to achieve it. But this person got distracted by the tool of measurement, making sure that the buck stops there.
Yesterday, I saw a post about police violence, pointing out ‘police shouldn’t kill guilty people either’. This was a response to how people often protest police killing innocent people, which is definitely bad, but the point is that the police shouldn’t be killing anyone outside of strict self defense. The justice system is what meters out punishment, not the personal discretion of a state-sponsored gang with too many combat toys. The role of the police to to prevent violence and capture wrongdoers, not deal out extrajudicial executions. I’m sure I don’t have to explain in detail why this is so fucking important, but one set of tags on the posts made the distinction “except for pedophiles and rapists”. I have never seen anybody miss the point of a post so badly. Clearly, this person had once again gotten distracted by the system of measurement – pedophiles and rapists are evil people who do evil things, therefore they should be eliminated as expediently as possible – without considering the effect on the system. No, police randomly shooting rapists does not make a better society. If you support the death penalty for rape, that’s a whole arse different question.
These kneejerk reactions don’t just happen with pedophiles and rapists (although they are very effective for it, which is why dangerous and unsavoury elements like to call the groups they hate pedophiles). I’ve also seen people get upset at historical demonstrations of queer unity and support because the people in them called each other words they don’t like and get all distracted by minutae on who’s ‘allowed’ to ‘reclaim’ what words, preferring to condemn gay men calling lesbians ‘muffdivers’ despite the massive personal risk and great benefit of the demonstration. I’ve seen people quibble over what groups of disabled people experience more ableism than others, and which queer subcommunities are more oppressed, in order to determine who the good guy in a complex situation is or who deserves their support more. I’ve seen people slip all the minorities they belong to into an argument like they’re laying out the cards to summon Exodia (because most oppressed person is most deserving of support person and therefore most correct person), I’ve seen people distract from arguments they’re having in order to try to trap the other person into saying something that can be interpreted as sexist or racist so they can show that their opponent is the Bad Person (and therefore they’re the good person and therefore correct in the argument), I’ve seen people look at two people with conflicting needs (such as an autistic person who verbally stims and one who reacts badly to too much sound) and stop to decide which one is oppressing the other one to determine which one is being ableist.
This is all fucking bullshit. It’s meaningless nonsense. The only reason any of this matters is in how it relates to causing actual real world harm. I’d rather be called a tranny bitch by someone who votes in support of my healthcare than the most polite and up-to-date language by someone who votes against it. I’d rather know about risk factors that make someone more likely to be an abuser or rapist than shy away from such things because I don’t want to risk thinking of them as anything other than an Unknowable Evil. I don’t fucking care what Problematic ™ views someone holds about a cartoon and I don’t care who’s the Most Pure or the Most Oppressed or who used to say slurs online when they were fifteen if they’re behaving appropriately now. None of that fucking matters, and it’s not justification for harassing or hurting people.
Your sense of justice and morality are social tools. Sharpen them, clean them, look after them. And use them to build with purpose, rather than blindly hacking at whatever’s in front of you. Or you’ll just make a mess.
(Source)
This phenomenon is super prevalent in the Tech sector, partially because we haven’t figured out how to measure programmer productivity well. I’m sure it has nothing to do with how most good programmers want to solve a problem as efficiently as possible, which often means optimizing for whatever metric is being measured at the expense of everything else.
Thanks @derinthescarletpescatarian for connecting that microcosm to what is happening in the broader world.
