WEIRD research on WEIRD people

To start, you might want to look at the interesting article by James Surowiecki in this week's New Yorker on procrastination. Which brings me to my resolution: I am going to try to finish the rewrite of Working with Stories this month. My favorite part from Surowiecki's article is this part:

Instead [of fighting with ourselves], we should rely on what Joseph Heath and Joel Anderson, in their essay in [a book named] "The Thief of Time," call "the extended will"—external tools and techniques to help the parts of our selves that want to work. A classic illustration of the extended will at work is Ulysses' decision to have his men bind him to the mast of his ship. Ulysses knows that when he hears the Sirens he will be too weak to resist steering the ship onto the rocks in pursuit of them, so he has his men bind him, thereby forcing him to adhere to his long-term aims. 

So I have created an "extended will" of (a) publically saying I will do this, and (b) limiting myself to microscopic blog posts for the remainder of the month. Yes, it's bread and water for you folks this month. Here is this week's meagre fare.

WEIRD people and the researchers who love them

I've been having a wonderful time traipsing through this review paper. It's by Joseph Henrich, Steven J. Heine and Ara Norenzayan, and it explains how much of psychological and sociological research is biased by its exclusive attention to WEIRD people: that is, people from Western, educated, industrial, rich, democratic societies. Here are some of my favorite excerpts.

In the Journal of Personality and Social Psychology, the premier journal in social psychology—the sub‐discipline of psychology that should (arguably) be the most attentive to questions about the subjects’ backgrounds—67% of the American samples (and 80% of the samples from other countries) were composed solely of undergraduates in psychology courses (Arnett, 2008). In other words, a randomly selected American undergraduate is more than 4000 times more likely to be a research participant than is a randomly selected person from outside of the West.

That's perfectly spectacular bias, isn't it? Worse, the idea that American undergraduate students might not be representative of the species is rarely mentioned:

Leading scientific journals and university textbooks routinely publish research findings claiming to generalize to "humans" or "people" based on research done entirely with WEIRD undergraduates. In top journals such as Nature and Science researchers frequently extend their findings from undergraduates to the species—often declaring this generalization in their titles. These contributions typically lack even a cautionary footnote about these inferential extensions.

The authors review worldwide differences in visual perception, spatial cognition, cooperation, independence, choice, views of self, conformity, reasoning styles, and moral reasoning. Across all these areas the human ranges are wide, but the research selections and conclusions are narrow.

Carpentered minds

I found plenty of surprises here. For example, did you know that not everyone sees these lines as having different lengths? This is the famous Mueller-Lyer illusion, but apparently the San foragers of the Kalahari, among others, are "unaffected by the so-called 'illusion'."

The authors mentioned that this issue was previously discussed by Segall, so here I am looking up the referenced book (The Influence of Culture on Visual Perception) and assuming it has just come out, and I find out it was published in ... 1966. And it's out of print. What I end up wondering is why, 44 years later, nobody knows about this (unless I'm the only one surprised, which I doubt). I wonder what that says about science and culture. Did we not want to know that?

Henrich et al. say:

As discussed by Segall et. al., these findings suggest that visual exposure during ontogeny to factors such as the "carpentered corners" of modern environments may favor certain optical calibrations and visual habits that create and perpetuate this illusion. That is, the visual system ontogenetically adapts to the presence of recurrent features in the local visual environment. Since elements such as carpentered corners are products of particular cultural evolutionary trajectories, and were not part of most environments for most of human history, the Mueller‐Lyer illusion is a kind of culturally‐evolved byproduct....

Non-carpentered house

It certainly makes me look at my house differently, I'll tell you that. I have seen some designs of clay houses where a sharp corner cannot be found, and I wonder what it might be like to have the gift of not having grown up "cornered." What might my mind be like? What might I be capable of? What sorts of sensemaking frameworks might I come up with? What sorts of stories might I tell? What sorts of software might I design? And what would an internet designed by non-cornered minds look like? Maybe we should find out.

Nice bird

I will allow myself just one more excerpt (I told you it would be meagre), this one about "folkbiological reasoning." I like this one particularly because it intersects with something I've mentioned on this blog a few times. My hobby is reading old novels, and I've repeatedly been struck by how people spoke so casually then of skills we cannot fathom today, like starting a cake recipe by grinding the flour and solidifying the butter. And how our perceptions of everyday life have moved on a gradient from creating to consuming in many overlapping ways.

This part of the study matches that observation:

[There are] three robust findings from urban children: (1) inferential projections of properties from humans are stronger than projections from other living kinds, (2) inferences from humans to mammals emerge as stronger than inferences from mammals to humans, and (3) children’s inferences violate their own similarity judgments by, for example, providing stronger inference from humans to bugs than from bugs to bees....

However, when the folkbiological reasoning of children in rural Native American communities ... was investigated ... none of these three empirical patterns emerged.... In rural environments both exposure to, and interest in, the natural world is commonplace, unavoidable, and an inevitable part of the enculturation process. This suggests that the anthropocentric patterns seen in U.S. urban children results from insufficient cultural input and a lack of exposure to the natural world. The only real animal that most urban children know much about is Homo sapiens, so it is not surprising that this species dominates their inferential patterns.... Indeed, studying the cognitive development of folkbiology in urban children would seem the equivalent of studying "normal" physical growth in malnourished children.

So not only is cake made of things from the refrigerator and cabinet instead of the field and barn; animals and plants are things in zoos and parks instead of in woods and streams. And fridges and cabinets and zoos and parks are the things we come to know the most about (and assume everybody else knows about too). This statement also resonated with me:

This deficiency of input likely underpins the fact that the basic level folkbiological categories for WEIRD adults are life‐form categories (e.g., bird, fish, and mammal) and these are also the first categories learned by children: e.g., If you say "what’s that" (pointing at a maple tree), the common answer is "tree". However, in all small‐scale societies studied, the generic species (e.g., maple, trout, and fox) are the basic level category and the first learned by children....

I'm not a naturalist by any stretch, but I've been confused by an increasing tendency for people to think my minuscule ability to distinguish a few local species of trees, birds, flowers and herbs is more amazing than it is. When I was a young biologist, the number of birds and trees and things I knew was embarrassing, and still is among real naturalists. But among people who don't consider themselves biologists I've noticed a change over the decades. Today I almost feel a pressure to hide knowledge about species when I talk to people, as though they feel my saying "wow, did you hear that thrush singing" instead of "nice bird" is an insult. They want the larger categories. In my rural childhood we did learn the specific names for things first (many of our names were wrong, but that's beside the point: they were precise if not accurate). I've met many people since who were raised in urban environments and who habitually refer to "that flower" or "that bird" as though there were no sub-categories beneath the general class. I'm not saying there is anything wrong with that, but it does make me wonder what effect on general thinking about things it might have when your taxonomies stop halfway down. How does that impact the things people build? How should people design for people who have had different experiences than they have had? How should I design for urbanites, and how should urbanites design for me?

I can't help mentioning my Gogol mountain ash story again (sorry to those who have heard it already). I was reading Dead Souls, and I came across this bit:

At one particular spot the steep flank of the mountain range is covered with billowy verdure of denser growth than the rest; and here the aid of skilful planting, added to the shelter afforded by a rugged ravine, has enabled the flora of north and south so to be brought together that, twined about with sinuous hop-tendrils, the oak, the spruce fir, the wild pear, the maple, the cherry, the thorn, and the mountain ash either assist or check one another's growth, and everywhere cover the declivity with their straggling profusion.

It just so happened that on the day before I read this, I had been trying to figure out what a young tree growing in my yard was, and I fell into the "mountain ash" section of the field guide, never having heard of one before. What made me laugh that day was that evidently Gogol felt he could build a narrative metaphor through reference to the relationships among particular species of trees. It apparently never occurred to Gogol to think that he needed to explain to his readers how these species interact, because of course everybody would know those things. Mountain ash is an understory shrub that (at least in my yard) is tolerated to live beneath the larger oaks and maples which check its growth. People reading Dead Souls when it was written would have had a precise image of the relationships to which Gogol was referring, but we struggle to grasp them, even if we have mountain ash trees growing in our yards.

In every age and place we assume some knowledge is universal, and we are always wrong. Otherwise I wouldn't have two bookmarks in every old book I read -- one in the book proper, and one in the notes that explain all the things the author thought didn't need explaining when they wrote the book. We do the same thing as Gogol: we say "I got my email" expecting our readers to know that involves turning on a computer and using an email program. (Which will probably be a footnote two hundred years from now.) This sort of we-are-the-world thinking is interesting and fun when novelists do it. I love figuring out what authors are talking about and learning what people used to think everybody knew. But it's dangerous when scientists do it.

Henrich et al. conclude:

Journal editors and reviewers should press authors to both explicitly discuss and defend the generalizability of their findings..... The widespread practice of subtly implying universality by using statements like "people’s reasoning is biased..." should be avoided. "Which people?" should be a primary question asked by reviewers.... The sample of contemporary Western undergraduates that so overwhelms our database is not just an extraordinarily restricted sample of humanity; it is frequently a distinct outlier vis‐à‐vis other global samples. It may represent the worst population on which to base our understanding of Homo sapiens.

My suggestion is that the import of papers like these (this is not the first) goes further than even the authors state. Biased-sampling science over-generalizes explanation, but it also over-generalizes expectation. Studied things are important things, and people pay attention to research results.

One of the papers that had a big impact on me back when I was reading mostly about animal behavior was a study done on birds where the researchers used colored bands to distinguished their bird subjects. They had to halt the study because they discovered that the male birds with the red bands were receiving disproportionate attention from the female birds. In other words, the experimental design altered the phenomenon being studied. I remember another study on naked mole rats where the researchers observed a curious phenomenon that seemed to indicate a novel circadian rhythm, until they realized the activity rhythm coincided with the rhythm of ventilation noises in the building. If you are studying rocks you can probably choose any sample and say it is representative without the rocks responding, but with anything else responses to experimental manipulation might extend beyond what you thought you were experimenting with. Experiments that produce broad claims from tiny slices of humanity are conducting a wider experiment than they imagine.

Here's hoping attention to WEIRD research has an impact on the way people study people.