Not nutting up or shutting up:
Notes on the demographic disconnect in our field’s best practices conversation
Alison Ledgerwood, Elizabeth Haines, and Kate Ratliff
A few weeks ago, two of us chaired a symposium on best practices at SPSP focusing on concrete steps that researchers can take right now to maximize the information they get from the work that they do. Before starting, we paused briefly to ask a couple simple questions about the field’s ongoing conversation on these issues. Our goal was to take a step back for a moment and consider both who is doing the talking as well as how are we talking about these issues.
Apparently our brief pause sounded strident to some ears, and precipitated an email debate that was ultimately publicized on two blogs. The thing is, the issues we originally wanted to raise seemed to be getting a little lost in translation. And somehow, despite the absolute best of intentions of the two people having the (cordial, reasonable, interesting) debate, we had become literally invisible in the conversation that was taking place. So we thought maybe we would chime in, and Simine graciously allowed us to guest blog.*
As we said in our symposium, a conversation about where the field as a whole is going should involve the field as a whole. And yet, when we look at the demographics of the voices involved in the conversation on best practices and the demographics of the field, it’s clear that there’s a disconnect.**
For instance, the SPSP membership is about 56% female. Yet when you look at the speakers in best practices symposia at SPSP over the last four years, only about 30% are female. (That number drops to 23% if you remove our symposium.) Or take the ISCON Facebook page, where so many of these conversations take place. The membership is 40% female. The percent of comments on best practice-related posts made in the past year by females? 10%. You can see similar disconnects if you look at race/ethnicity, power, or institution type.
So here’s an interesting puzzle for the social scientist in all of us: Why is there this disconnect?
One might wonder: Is it intentional discrimination?*** As far as we know, there is zero evidence that anybody is sitting around intentionally excluding groups of people from the conversation. For instance, there happens to be an unofficial email list of more than 30 scholars in our field devoted to discussing issues in best practices as they arise. This email list is 100% white and 100% male.
But we find it extremely hard to imagine that any of them are actively trying to keep women or minorities off their listserv. In fact, we strongly suspect that most of them haven’t noticed the demographic composition of the listserv, and might be a little surprised if they did. Another example: That “Nut Up or Shut Up” blog post on replication last year was obviously not intended to make women feel alienated. (And nonetheless, it did.)
Here’s a much more compelling explanation for the demographic disconnect, and one that social psychological research can actually tell us a lot about: We suspect that a number of forces have combined to create unintentional barriers that limit the extent to which some scholars are involved and visible in the best practices conversation. For instance:
1. The question of whether to chime in (and whether that’s even a question)
Over the last few years, we’ve asked a number of scholars with strong methodological backgrounds to contribute to the best practices conversation, in talks or in articles, and we’ve heard many other people talk aloud about the prospect of chiming in.
Here’s what men tend to say: “Yes! I have a lot of opinions about this. Let me tell you about them.”
But here’s what women tend to say instead: “Hmm…I do have a lot of opinions about this…but I’m not sure I’m the most qualified person to be talking about them.”
In other words, whereas some scholars seem to see a logical link between having an opinion and wanting to express it, others seem to grapple with an extra question in the middle of that process: What gives me the legitimacy to say this?
These anecdotes shouldn’t be surprising to researchers who study topics like power and gender: We know that power tends to move people away from a deliberative mindset (e.g., “Should I say something? What are the pros and cons?”) and toward an implemental, action-oriented mindset (“I should do this now;” see Magee et al., 2005, for a review). And we know that in general, men are more likely to speak up, talk for longer, and interrupt more than women are (Dovidio et al., 1988; Hall & Sandler, 1982; Holmes, 1992; Zimmerman & West, 1975).
Certain demographics, then, are less likely to chime in because they get stalled somewhere halfway through a deliberative mindset. But notice what else this means. If people who feel higher in power are likely to jump in frequently and talk a lot, another unintentional consequence may be a crowding out of the less frequent voices. One of the people reacting to our symposium recently asked: Do the validity of arguments depend on the demographics of people making them? Definitely not. But if the airtime given to arguments depends on the demographics of people making them, maybe we should be paying attention to that.
2. How we’re talking.
The tone of the discussion and debates about best practices so far has been remarkable in its tenor and volume. Conference symposia, papers, and especially less formal posts on blogs and social media tend to be heated, polarizing, and often angry or moralizing. We challenge you to think of another topic in our field that could generate 289 comments in response to a single post on the ISCON Facebook page. Once again, social psychological research can tell us a lot about how this tone may be feeding into the demographic disconnect.
For instance, decades of research demonstrate what happens to women who act in powerful, directive, and authoritative ways. They are targets for backlash—the social, economic, and evaluative penalties of not being nice, kind, accommodating, and likable (e.g., Eagly & Karau, 2002; Heilman, 1983; Rudman, Moss-Racusin, & Nauts, 2010). Moreover, participating in conversations that get angry can be uniquely costly for women: When women (vs. men) appear angry, they are more likely to be written off as an “angry person” or “out of control”—in fact, whereas perceived anger tends to undermine women’s authority, men’s anger actually increases their perceived status (Brescoll & Uhlmann 2008). And when women are seen as talking too much, they are viewed as less competent and less suited for leadership than men who use the floor too much (Brescoll, 2011).
In other words, competitive discourse tends to hurt women and help men, in terms of how they are perceived by others. Women may therefore lean back from heated public discourse because they understand—consciously or unconsciously—the professional and social consequences of talking too much or talking with the intensity that has become de rigueur in the fields’ conversations about best practices.****
More broadly, framing debates in terms of who is a “real” or “good” scientist—a good group member—may tend to particularly discourage peripheral (vs. central) group members from expressing their views. Research suggests that peripheral group members tend to be more concerned about demonstrating that they are good, prototypical group members, and they may feel especially threatened by the prospect of being told that they are not (e.g., Jetten et al., 2006; Klein et al., 2007). In other words, to the extent that things like gender, ethnicity, power, and institution type correlate with how central or peripheral people feel as scientists or members of the field, the tone of the conversation may make the prospect of participating in the conversation feel more threatening or costly to some people more than others.
Of course, there are other reasons for the demographic disconnect, too. Feel free to chime in. Yep, that’s right. We mean you.
Why should we care?
Right now, our discipline is grappling with a number of complex and important issues on the best practices front. We are at a time when we need a lot of good and creative ideas. Diversity helps us achieve that goal—just look at all the research linking demographic diversity to improved decision-making (Smith et al., 2006), particularly in areas where innovation is important (Richard et al., 2003).
Including more voices also improves our chances of developing field-wide standards that are applicable and effective across a wide range of research areas. For instance, we don’t want just experimental social cognition researchers developing a set of field-wide standards based on their experience with running two-condition reaction time studies—we can’t assume that the same standards will apply equally well across areas like political psychology and social neuroscience and relationships research. The more voices we include in the discussion, the better our chances of incorporating a wide variety of researchers’ experiences with the complexities, nuances, and realities of data collection in their own subfields.
So what should we do?
Already have a spot at the table? Share it. Look around you for people with good ideas who might be hovering in the periphery, and invite them to pull up a chair (as an author, speaker, guest blogger, you name it). Be aware of when the airtime given to arguments tracks the demographics of people making them more than their expertise or thoughtfulness, and consider whether you could steer the conversation toward the latter more than the former.
Meanwhile, if you’re waiting silently, deliberating whether to weigh in: Say something. Think about how long you’ve been conducting research. You know something about research practices. You may not have conducted a dozen simulation studies, but neither have most people contributing to the current conversation...they’re basing their opinions on a consideration of their own experience in the field. So if you’re sitting there with thoughtful opinions, speak up! Our field is striving toward a better science—toward identifying and honing research practices that maximize the knowledge we get from the work that we do. That’s not an easy task. We need all hands on deck.
*Prerequisite: A shared, profound fondness for footnotes?
**Note that we haven’t said anything about WHY there’s a disconnect yet, nor did we say anything about it during our symposium. There can be all sorts of reasons for disconnects. We’ll get to some possible reasons for this one soon. But in the meantime, if you feel like we’re attacking you? Calm down. This is about as vanilla as suggesting that the demographics of cat owners don’t match the demographics of society as a whole. We’re just mentioning some descriptive statistics.
***We actually never wondered this, but apparently some people see the pie charts above as not just wondering it but claiming it, so let’s tackle this idea head on.
****Other women may jump into heated public discourse after a year or three of restraining themselves because they think maybe, just maybe, if they write this post, something will start to change.