Identifying Sex/Gender Bias in Research

Who Is the Researcher?

How does the identity of the researcher contribute to bias?

Many factors influence the research process, including who actually does the research. The phrase “the myth of the impartial researcher” refers to a common misperception—drawn from positivism—that people who conduct research are value-neutral and thoroughly objective. As discussed earlier, all psychological theories start from personal observations or previously established theories (i.e., concepts based on a previous researcher’s work, perhaps based on their own observations). Therefore, if science is primarily conducted by one group of people, it results in a narrow range of possible theories. Most senior authors of peer-reviewed publications are White, upper-class men who are senior professors at top research universities (Cundiff, 2012; Eagly & Riger, 2014). Although it is certainly valuable to develop theories that stem from the life experiences of White, upper-class men, it becomes a problem when that is the predominant worldview represented in psychological research.

As an example, Freud, the founder of psychoanalysis, said that a key component of women’s psychological development is their experience of penis envy (Freud, 1905/1949b). In the 1930s, Karen Horney, a psychoanalyst, criticized Freud’s inability to consider a woman’s perspective and called for a more woman-centered approach (Westkott, 1986). Astonishingly, Freud later responded to her critique by saying, “We shall not be very greatly surprised if a woman analyst, who has not been sufficiently convinced of the intensity of her own wish for a penis, also fails to attach the proper importance to that factor in her patients” (Freud, 1940/1949a, p. 107). How fascinating that Freud was accusing someone else of bias! This example demonstrates the limitations of having only one type of person conduct research.

In 1988, several feminist psychologists published “Guidelines for Avoiding Sexism in Psychological Research” (Denmark, Russo, Frieze, & Sechzer, 1988). They noted many examples of how the predominance of White male researchers caused a bias in what is studied and how questions are asked. For example, they noted that leadership had traditionally been understood as dominance and that topics relating to White men (e.g., television and aggression) were seen as more important than topics relating to women or people with other marginalized identities (e.g., pregnancy and menopause).

What Is the Research Question?

How do research questions themselves contribute to bias?

Who the researcher is will influence not only the formation of theories but also the specific research questions asked. In discussing research about sex differences, for example, psychologist Rhoda Unger (1979) argued that the very decision to study differences between women and men reinforced the status quo rather than allowing the discovery of anything new. It’s intriguing to consider how different the entire field might be if we changed the question to ask “How are women and men similar?” As another example, you might consider what would happen if we asked women to describe their experience of menstrual joy instead of menstrual discomfort.

There are also hidden assumptions in research questions. Let’s imagine, for example, a group of researchers exploring the effects of daycare on child development, with a specific interest in maternal working status. Their research question might be “Should mothers be employed out of the home early in their child’s life?” This question doesn’t include fathers, so the hidden assumption here is that mothers, not fathers, are the ideal caretakers, and that researchers should question a mother’s, but not a father’s, employment status during their child’s early years. How different do you suppose the findings would be if the researchers explored “Should mothers or fathers be employed out of the home early in their child’s life?”?

Who Are the Research Participants?

How does the identity of research participants contribute to bias?

Another area of potential bias is choice of participants. Psychological research has a long history of using male participants and then generalizing results to all humans (Eagly & Riger, 2014). For example, Carol Gilligan criticized fellow psychologist Lawrence Kohlberg because his theory of moral development was entirely based on interviews with wealthy, White boys and men (Gilligan, 1982). Kohlberg (1969) argued that children move through a fixed set of stages in which morality shifts from focusing on avoiding punishment, to focusing on following the rules or being “good,” to focusing at the highest stage on abstract ideals of individual rights and autonomy. In response, Gilligan argued that Kohlberg’s highest stage of development, based on individual autonomy, reflected an androcentric view and didn’t include the communal and care ethic often observed in girls and women. Gilligan was especially critical of the way Kohlberg used data collected from boys and men to explain human development. According to Gilligan, it was unreasonable to apply findings based on boys and men to girls and women.

Psychological research continues to be guilty of this tendency, which undermines generalizability, or the ability to use findings from a given study to explain phenomena that occur in the general population. Even today, the titles of studies rarely indicate when participants are exclusively men or exclusively White (Cundiff, 2012). Yet when participants deviate from the White male norm, researchers not only provide a rationale for their decision but also often signal it in the title of their article (Cundiff, 2012).

Recently, the number of female participants has substantially increased—probably due to their over-representation in psychology undergraduate participant pools, a source of convenience samples for many psychologists (Cundiff, 2012). As a result, most of what we know about human behavior is now based on a very specific type of woman. She’s likely an undergraduate student at a large university who is enrolled in an Introduction to Psychology course. Such sampling bias overlooks the experiences of all other people. If you think back to other psychology courses you’ve taken, it might be interesting to recall how often your textbooks indicated who was included in the samples used in the studies described. If you knew that a study sample was based only on undergraduate women, would that change your perception of the results?

Sampling bias is also a challenge within the field of psychology of women. In its early stages, feminist research didn’t fully address the diversity among women. Instead, it focused primarily on the experiences of White, well-educated, middle-class, heterosexual women who were disproportionality members of Western, industrialized, rich, democratic societies (Henrich, Heine, & Norenzayan, 2010). Within a field that was supposedly trying to undo bias, feminist researchers’ sole focus on such privileged women actually replicated power hierarchies. It’s easy to see how this can happen. When researchers use convenience samples, they tend to get a group of participants who resemble them or the students at their college or university. When research, even feminist research, is predominantly done by White women at predominantly White institutions, this can lead to systematic sampling bias that over-represents the concerns of White, well-educated women. For example, body anxiety centering on concerns about thinness is well researched within the feminist community. However, this research area might reflect concerns that are particularly important to White women, an issue we’ll revisit in Chapter 6.

Bias also exists in the ways demographic questions are asked—such as when marginalized women are clumped into one social identity category. The term LGBTQ (lesbian, gay, bisexual, transgender, and queer) places all sexual minority individuals into one category, ignoring differences among them. For example, there is tremendous diversity among bisexual women, but this will be overlooked if researchers don’t study bisexuality separately from other sexual-minority identities. Notice, too, that the term places all people who are sexual minorities in contrast with people who are heterosexual. In other words, all other people are being compared to heterosexual people. This reinforces the idea that some people are the norm against which all others should be compared—which creates a bias within research.

How Are the Variables Measured?

How does the operationalization of variables contribute to bias?

Another potential source of bias involves the way variables are measured. Recalling the example from above, body anxiety could be measured in many different ways, and decisions about how to operationalize a variable could be a source of bias. Let’s take something as seemingly straightforward as aggression. In the past, researchers have measured aggression in mostly overt ways (e.g., physical and verbal assaults; Brown, 2005). By doing this, their data showed that boys were more aggressive than girls. However, when researchers began to measure relational aggression (e.g., gossiping and socially isolating other girls), a different picture emerged: Girls were aggressive too (Crick & Grotpeter, 1995).

Ideally, consumers of psychological science should be informed about how variables are defined, although this information is rarely included in popular press summaries of research. The lack of information makes it hard to evaluate research claims because doing so requires tracking down the original article. How often have you looked for the original article of a study you read about on social media? It can be difficult to find the time to do this, and costly too, because academic journals generally aren’t free to the public. For example, in 2017 it cost $36 to purchase access to an individual article published in Psychology of Women Quarterly and $106 per year to subscribe to the journal as a whole. If you’re a student at a college or a university, you can probably access articles for free through your school’s library system—a privilege that not everyone has.

How Are the Data Analyzed?

How does the method used to analyze data contribute to bias?

Most psychological research relies on quantitative methods. As we’ve discussed, the reliance on statistics also perpetuates bias since the design of many statistics is to look for differences within a sample. Although quantitative research is the most widely used methodology among psychologists, some feminist scholars have advocated the use of another methodology. Qualitative methods produce descriptive data, with little attention to statistics. Qualitative researchers often rely on interviews, diaries, observations, and archival data. In these cases, data analysis involves identifying themes or patterns in participants’ responses in order to understand how participants interpret various aspects of their lives. Since themes emerge from the data, researchers are, theoretically, less likely to control or manipulate variables and are more flexible in making design changes alongside their study participants. In this approach, researchers and participants can develop close relationships, which is very different from the detached relationships usually associated with quantitative methods.

Feminist researchers are critical of the rigid separation of researchers from study participants, suggesting that it reflects power and control (Rutherford & Granek, 2010). In labs, researchers are considered experts who manipulate situations in order to study outcomes. This hierarchy may be problematic for collecting accurate data. However, qualitative methods offer an exciting alternative to the detached aspect of quantitative methods. One such qualitative method, participatory action research (PAR), includes participants in the decision process during every stage of the research (Yost & Chmielewski, 2013). For example, participants might use cameras to document their day-to-day experiences and then collaborate with the researcher about how to use this information to develop research questions and to design a study.

Despite their appeal, qualitative articles are far less likely than quantitative ones to appear in psychology journals. In a review of all psychology journals listed with the academic database PsycINFO from 1960 to 2009, only 8.7% of articles were qualitative (Eagly et al., 2012). Interestingly, most of these articles were written and published outside of the United States (e.g., in Canada, Australia, the United Kingdom), and there was a 26.2% increase in qualitative methods in journal articles that focused on gender (Eagly et. al., 2012). So does research need to be qualitative to be considered feminist? Not at all! Feminist researchers use both qualitative and quantitative methods. However, no matter what methods feminist researchers use, they pay attention to how their own experiences and perspectives influence the questions they ask and the results they find.

How Do Researchers Write about Their Results?

How does the way researchers write about their work contribute to bias?

Bias is also evident in the language used to present research. One concern is use of the generic man or he to describe humans. This usage is problematic because it both includes and excludes women (Smith et al., 2010). As we’ll discuss in Chapter 2, the generic he also excludes people who use pronouns that aren’t gender specific (e.g., they/them/their). Because of this, the APA publication manual advises against using generic pronouns (APA, 2009). Language can also reflect more subtle biases—for example, use of words that have masculine connotations (e.g., penetrate, dominate) or descriptions of sex-related differences in ways that are demeaning toward women (e.g., submissive). Even in writing this textbook, we came across an instance of our own bias. A reviewer of an early draft pointed out that we were consistently putting men first when we made a statement about “men and women.” In a textbook about women, she said, please start with women. These subtle biases might seem unimportant, but the cumulative effect suggests that men should be prioritized. In fact, in a review of psychological studies, male participants’ results were disproportionately presented before those of female participants (Miller, Taylor, & Buck, 1991).

Where Are the Results Published?

How does the place where research is published contribute to bias?

You might recall that when describing the hypothetical body anxiety study earlier, we suggested submitting the research article to a journal that focuses on gender. This would ensure that the reviewers had previous knowledge about sex and gender. Although doing this makes a lot of sense, the drawback of always doing this is that research about sex and gender would then only appear in specialized journals. For example, in a review of the literature in 2012, researchers found that 89% of articles published about sex and gender appeared exclusively in Psychology of Women Quarterly or Sex Roles (Eagly et al., 2012). This means that journals specializing in other aspects of psychology will be less likely than specialized publications to include feminist perspectives.

Further, according to feminist psychologist Stephanie Riger (1992), feminist research shouldn’t simply be used for the production of knowledge; it should also be used for social action and social justice. If research is only published in academic journals, it will only reach a small subset of the population that would benefit from it. Feminist researchers prefer to “give research away.” Examples include sharing results on social media or through e-mail, and using results in formal conversations with policy makers and legislative bodies that often overlook the psychological needs of marginalized individuals. These efforts are in distinct contrast to the role of detached and objective researcher. Indeed, feminist researchers become advocates as they share their findings and continue to ask questions in hopes of finding answers that can increase equity.

Another way to give research away is to talk about research findings in the popular press. Unfortunately, the popular press tends to simplify the findings of psychological research, particularly those that confirm popularly held stereotypes (O’Connor & Joffe, 2014). Television and social media predominantly feature stories that exaggerate the differences between women and men (Fine 2010; O’Connor & Joffe, 2014). They also generally ignore the diversity among women and often reinforce the idea that everyone can be clearly labeled as either a woman or a man (Golden, 2004). This probably happens because short, simple stories are more likely to be read, which ultimately increases revenue for the media outlets (O’Connor & Joffe, 2014).

Minimizing Bias in Feminist Research

What are key ways psychologists minimize bias in feminist research?

Because the foundation of psychology relies on conducting studies, feminist psychologists have worked to transform inherent biases in both the content and the methods of psychological research (Eagly & Riger, 2014). From their perspective, science is not counter to feminism because it has value in generating reliable information about women’s lives. While there is no one type of feminist research, two characteristics tie contemporary feminist researchers together:

• They are aware of power dynamics inherent in research. A core belief is that no research method is free of bias and that, therefore, it is essential to carefully consider potential problems at every stage of the research process (Crawford & Kimmel, 1999; Rutherford & Granek, 2010; Unger, 1983). For example, feminist researchers reject the myth of the impartial researcher and avoid making overly simplistic interpretations of their research results. Ironically, feminist researchers have commonly been criticized because they often explicitly acknowledge their biases. In reality, all research is biased, and feminists are just being honest.
• They consider intersectionality. In addition to attending to gender, feminist researchers take into account intersectionality (Cole, 2009; Rutherford & Granek, 2010). In order to ensure that psychological research addresses intersectionality, psychologist Elizabeth Cole (2009) proposed three questions that researchers should ask:

1. Who is included within this category? For example, at most universities, if student samples are used, then the experiences of predominantly White and middle-class women are likely to be taken for the experiences of all women. This practice should be avoided.
2. What role does inequality play? Due to historical and institutional forces, some social groups have more power than others, and these power differences can shape behavior and experiences. We will address this more in Chapter 2.
3. Where are the similarities? Even if people are from different social groups, they may have shared experiences or concerns. For example, people of diverse social identities may all be concerned with violence in their community, although their concerns may express themselves in different ways.

Table 1.2 shows how these questions can inform each stage of the research process.

TABLE 1.2 Questions to Encourage Intersectionality throughout the Research Process

Note. Adapted from Cole (2009).

In practice, every methodological choice has advantages and limitations. The goal is to constantly reflect on research decisions in order to minimize bias. No method can be completely neutral or objective. Therefore, it’s important to read every psychological study with a degree of caution.