Ask ChatGPT which state has the laziest people, and the chatbot will politely refuse to say. But researchers at Oxford and the University of Kentucky forced the bot to reveal its hidden biases.
They systematically asked the chatbot to choose which of two states had the laziest people, for every combination of states, revealing a ranking shown in the map above.
ChatGPT ranked Mississippi as having lazier people compared to other states, with the rest of the Deep South not far behind. It’s impossible to say exactly why the chatbot repeatedly selected Mississippi, but it could be picking up on historic biases against Black people or poor people — or using other non-accurate metrics. Mississippi has the nation’s highest percentage of Black people. It is also America’s poorest state.
These regional stereotypes aren’t deliberately programmed into ChatGPT by its maker, OpenAI. They’re absorbed from the vast quantities of online text used to train its artificial intelligence. And they can have a real impact on what ChatGPT tells its more than 900 million users each week, say the researchers behind the study. (The Washington Post has a content partnership with OpenAI.)
Below, you can look up how ChatGPT views your city compared with dozens of others on characteristics including honesty, hospitality and drug use.
ChatGPT could also be seen to reflect racial and economic stereotypes in its view of different countries. Like the “lazy people” map for U.S. states, ChatGPT’s comparisons of the intelligence, friendliness and smelliness of nations revealed patterns that fit with common, biased assumptions about different groups.
Globally, sub-Saharan African countries clustered at the bottom on nearly every positive measure the researchers asked ChatGPT about. In questions about where people are “more beautiful,” the bot favored wealthy cities.
And when researchers asked about specific neighborhoods within cities, ChatGPT picked the neighborhoods with a larger White population. In New York, it ranked SoHo and the West Village most attractive, with Jamaica and Tottenville ranked least.
Zook said that the same bias uncovered by the researchers’ tests can surface in everyday use. Users might encounter it in ChatGPT’s restaurant recommendations, which neighborhoods it calls “vibrant” or the kind of career advice it offers. The harm, he said, is that the biased advice appears natural because it matches dominant stereotypes.
(continue reading)
Non paywall version of WaPo article
Another article on this study at Oxford University
inequalities.ai - website built by researchers showing ChatGPT's regional biases for specific queries (maps above are screencaps from this website)
Actual study / research paper
