On one problem with meritocracy: measuring merit
From "The Tyranny of the Meritocracy: Democratizing Higher Education in America", by Lani Guinier became the first woman of color appointed to a tenured professorship at Harvard Law School.
In 2004, economist Jesse Rothstein published an independent study that found only a meager 2.7 percent of grade variance in the first year of college can be effectively predicted by the SAT. The LSAT has a similarly weak correlation to actual achievement in law school. Jane Balin, Michelle Fine, and I did a study at the University of Pennsylvania Law School, where we looked at the first-year law school grades of 981 students over several years and then looked at their LSAT scores. It turned out that there was a modest relationship between their test scores and their grades. The LSAT predicted 14 percent of the variance between the first-year grades. And it did a little better the second year: 15 percent. Which means that 85 percent of the time it was wrong. I remember being at a meeting with a person who at the time worked for the Law School Admission Council, which constructs the LSAT. When I brought these numbers up to her she actually seemed surprised they were that high. “Well,” she said, “nationwide the test is nine percent better than random.” Nine percent better than random. That’s what we’re talking about. So, if the SAT does not correlate with the grades a student will get in college, how can a student’s performance in college be predicted? William C. Hiss and Valerie W. Franks, both formerly of the Bates College admissions department, released a report in 2014 that studied thirty-three colleges and universities that required neither the SAT nor its very popular competitor the ACT for admission. Now, which students did or did not choose to submit their standardized-test scores is in itself interesting—overwhelmingly those students who did not submit a score were women, minority students, or those who would be the first in their family to go to college, which should tell us a lot about the SAT right there. In reviewing the performance of more than eighty-eight thousand students, Hiss and Franks found that students who perform well in college were the ones who had gotten strong grades in high school, even if they had weak SAT scores. They also found that students with weaker high school grades did less well in college—even if they had stronger SAT scores. Summing up their findings they wrote, “Many of us who have spent our careers as secondary and university faculty and administrators find compelling the argument that ‘what students do over four years in high school is more important than what they do on a Saturday morning.’” So, if the SAT does not measure aptitude—and if it doesn’t even pretend to measure achievement—then what does it measure? I have argued for years that the SAT is actually more reliable as a “wealth test” than a test of potential, and the most recent results bear this out. Below are figures released in 2013 by the College Board that correlate SAT scores with the family income of the test taker. FAMILY INCOME [followed by] AVERAGE SAT SCORE (OUT OF 2400) FOR 2013 COLLEGE-BOUND SENIORS $0,000 – $20,000 1326 $20,000-$40,000 1402 $40,000-$60,000 1461 $60,000-$80,000 1497 $80,000-$100,000 1535 $100,000-$120,000 1569 $120,000-$140,000 1581 $140,000-$160,000 1604 $160,000-$200,000 1625 More than $200,000 1714 Now that is a correlation! This is what I refer to as the “Volvo effect.” In Crazy U, Ferguson talks about how the parents of his son’s friends and classmates were spending $30,000 to $35,000 to prepare their children for college. That isn’t the amount they had to pay for a premier boarding school mind you—that was the amount they paid to hire someone to tutor their child on the SAT and to help them write their “statement of interest” essays on their college applications. When these students get in to a particular college we say that this process reflects the fairness of the meritocracy, but really it only reflects the fact that the elite dominate the entry to higher education. These students aren’t smarter than the other students. Or to put it another way: they may be smart, but they are not necessarily those most likely to contribute to our society; they simply come from families that have more money to pay people to prepare them for the SAT, to test-prep them for their high school grades, and to pay for viola lessons so they can stand out more in the admissions process. The SAT’s most reliable value is its proxy for wealth. It is normed to white, upper-middle-class performance, as numerous studies have shown when the test is viewed through the lenses of race and class. The figures below, from 2013, show this in stark relief. TEST-TAKER ETHNICITY [followed by] AVERAGE SAT SCORE (OUT OF 2400) FOR 2013 COLLEGE-BOUND SENIORS Black or African American 1278 Mexican or Mexican Amer 1354 Puerto Rican 1354 Other Hispanic, Latino, or Latin American 1355 American Indian, Alaska Native 1427 Other 1501 White 1576 Asian, Asian American, Pacific Islander 1645 Is this a case of merit belonging to one race and not to another? Or is it the case that if you have grown up in a particular environment, such as one where your parents lack the funds to prepare you for these standardized tests or lack an advanced level of education themselves, you will not do as well on the SAT? There are other reasons why students of various ethnicities may underperform on the SAT. One of these is a phenomenon called “stereotype threat,” a term coined by Claude Steele of Stanford University (now provost of the University of California at Berkeley) to describe the anxiety a person may experience when he or she has the potential to confirm a negative stereotype about his or her social group. Many first- and second-generation immigrants of color test well, for example, because they retain a national identity free of America’s racial caste system and enjoy material and cultural advantages, including professional or well-educated parents. They do not internalize the stigma of race and are thus less affected by the anxiety of confirming assumptions of intellectual inferiority that depresses test scores of highly motivated students who are African American, Mexican American, or of Puerto Rican heritage. I know this threat is real. One summer not too long ago, I was engaged in a long-term writing project and recruited an absolutely brilliant young man who is Latino. Enrique (not his real name) has a photographic memory. I mean, he blew my mind. I have never seen anybody who could tell you, “Oh, well that’s on page 384. It’s in the middle of the page. I think it’s the first paragraph, not the second one.” But Enrique could not do well on the LSAT, though he practiced taking it close to thirty times. Enrique grew up in a low-income community, so arguably that had something to do with the verbal references that he might have missed. But a lot more of it had to do with stereotype threat: he was too tense. Postscript to this story: Enrique was subsequently selected to be a Rhodes scholar. So what, really, are we talking about here? If we can agree that the SAT, LSAT, and other standardized tests most reliably measure a student’s household income, ethnicity, and level of parental education, then we can see that reliance on such test scores narrows the student body to those who come from particular households. Then we must decide how to ensure that we open the admissions doors to a greater diversity of students—not just the ones from privileged backgrounds. I want to make it clear that I am not talking about affirmative action here. The loud debate over affirmative action is a distraction that obscures the real problem, because right now affirmative action simply mirrors the values of the current view of meritocracy. Students at elite colleges, for example, who are the beneficiaries of affirmative action tend to be either the children of immigrants or the children of upper-middle-class parents of color who have been sent to fine prep schools just like the upper-middle-class white students. The result? Our nation’s colleges, universities, and graduate schools use affirmative-action-based practices to admit students who test well, and then they pride themselves on their cosmetic diversity. Thus, affirmative action has evolved in many (but not all) colleges to merely mimic elite-sponsored admissions practices that transform wealth into merit, encourage over-reliance on pseudoscientific measures of excellence, and convert admission into an entitlement without social obligation. No, the question, as I said in the previous chapter, is this: How do we move from admission to mission? Further: How do we move past that moment of admission, which may only confirm one’s present status, to granting an opportunity for a diverse and worthy group of individuals to learn how to work together collectively and/or creatively to help solve the deep challenges confronting our communities, our economy, and our educational experiences in a democratic society? Of course, some of this has to do with how we define success. A study of Harvard alumni over three decades, which culminated in the 1990s, defined “success” by income, community involvement, and professional satisfaction. Researchers found a high correlation between those criteria and two criteria that might not ordinarily be associated with Harvard freshmen: low SAT scores and a blue-collar background. This is echoed by college admissions officers at elite universities today, who report— when asked what predicts life success—that, above a minimum level of competence, “initiative” or “hunger” are the best predictors. Marlyn McGrath Lewis, director of admissions for Harvard, says, “We have particular interest in students from a modest background. Coupled with high achievement and a high ambition level and energy, a background that’s modest can really be a help. We know that’s the best investment we can make: a kid who’s hungry.” That’s certainly the message of Derek Bok and William Bowen’s The Shape of the River, that those who are motivated to take advantage of an opportunity, when given the opportunity, can and often do succeed, often in ways that are different than their more privileged peers. The African American students in the Bok-Bowen study, for example, became leaders within their communities at much higher rates than their more affluent and better-scoring white counterparts. When I speak here of diversity, I’m not talking strictly along color or gender lines either. When the GI Bill was first proposed, toward the end of World War II, some university officials did their best to get it defeated. They were appalled by the prospect of what they saw as a mob of unprepared, unsuitable men trying to be their students. To their surprise, the veterans—many of them poor, most the first in their families to attend college—proved to be among the best students of their generation. By broadening access to college for those who had served their country, the GI Bill helped fuel the post–World War II economic boom while leveling the playing field for many Americans. The bill epitomized our country’s dual commitments: to open opportunity across the economic spectrum and to invest in people who will give back to society. We see the problem of restricted access today in the new elite class, which passes on its privileges in the same way that the old elite from twentieth-century America passed on its privileges. But there is an even more worrisome aspect of the new elite. The old elite felt that it had inherited its privileges; in order to defend the social oligarchy over which it reigned, the old elite felt the need to give back through public service or a financial commitment to the greater good. The old elite recognized that it had been privileged by the accident of birth, so the message to those who were out of luck was that you were unfortunate but it was through no defect of your own. The new elite, on the other hand, feels that it has earned its privileges based on intrinsic, individual merit. The message, therefore, to those who are not part of this elite is “You are stupid. You simply don’t matter. I deserve all the advantages I’m granted.” This attitude manifests in the jobs that college grads now take. For example, the student-run HarvardCrimson ran an article in 2007 about that year’s graduating class smirking that “only” 43 percent of female graduates entered finance and consulting compared to 58 percent of male graduates. The article, entitled “ ’07 Men Make More,” explained—with apparent disdain—that women choose jobs in lower-paying fields such as education and public service. Despite the economic downturn of recent years, the striking number of Harvard graduates entering finance and consulting has persisted. The class of 2013 senior survey showed that more than 30 percent of the 2013 class had jobs in those fields. After consulting and finance, the technology/engineering industry captured 13 percent of Harvard graduates that year. The Crimson again emphasized—with what seems to me to be the appearance of similar disdain—the preference of women to pursue less-lucrative work in education, media, and health care rather than in finance, consulting, and technology. The top career choices of many male Harvard students—whether it is 2007 or 2013—are severely lacking in any element of service. This is the damage that we are doing through our testocracy. We are credentializing a new elite by legitimizing people with an inflated sense of their own merit and little unwillingness to open up to new ways of problem solving. They exude an arrogance that says there’s only one way to answer a question—because the SAT only gives credit for the one right answer. The world, by contrast, provides us with more than one correct answer to most questions. In the face of mounting criticism, the College Board has recently proposed changes to the SAT, including reducing the use of obscure vocabulary words, narrowing the areas from which the math questions will be drawn, and making the essay section optional. But individuals such as Bard College president Leon Botstein find these proposed changes are too little, too late because they don’t address the test’s real problem. In an eloquent rebuttal, Botstein writes: The essential mechanism of the SAT, the multiple choice question, is a bizarre relic of long outdated twentieth century social scientific assumptions and strategies. As every adult recognizes, knowing something or how to do something in real life is never defined by being able to choose a “right” answer from a set of possible answers (some of them intentionally misleading). . . . No scientist, engineer, writer, psychologist, artist, or physician—and certainly no scholar, and therefore no serious university faculty member—pursues his or her vocation by getting right answers from a set of prescribed alternatives that trivialize complexity and ambiguity. Meaningful participation in a democratic society depends upon citizens who are willing to develop and utilize these three skills: collaborative problem solving, independent thinking, and creative leadership. But these skills bear no relationship to success in the testocracy. Aptitude tests do not predict leadership, emotional intelligence, or the capacity to work with others to contribute to society. All that a test like the SAT promises is a (very, very slight) correlation with first-year college grades. But once you’re past the first year or two of higher education, success isn’t about being the best test taker in the room any longer. It’s about being able to work with other people who have different strengths than you and who are also prepared to back you up when you make a mistake or when you feel vulnerable. Our colleges and universities have to take pride not in compiling an individualistic group of very-high-scoring students but in nurturing a diverse group of thinkers and facilitating how they solve complex problems creatively—because complex problems seem to be all the world has in store for us these days.
I can’t tag @slowdivin Ryan, but this is one way to argue against his notion of egalitarian meritocracy. Measuring merit is problematic because we live in white supremacist capitalist patriarchy. People cannot be assumed to be equal and be compared equally because of all sorts of problematic, vertical hierarchies. Guinier’s work on meritocracy does illustrate the problem quite well.
