Data Scientists Grapple With How to Ask, ‘What’s Your Gender?’

Dave Chappelle's take on the subject.

https://www.youtube.com/watch?v=jQ8Z00MfkcI

My take: You can call yourself whatever you want. I'm good with it. If you're a dude and want to compete in sports against biological females...sorry, boss. Not buying it.

Data Scientists Grapple With How to Ask, ‘What’s Your Gender?’

Traditional surveys can overlook the growing number of people who don’t identify as male or female

Josh Zumbrun, WSJ

Dec. 2, 2022 5:30 am ET

Right now, pollsters, academics, statisticians—anyone who cares about gathering good data—are finding that one of the trickiest questions to get right is this: “What’s your gender?”

A growing portion of the population doesn’t identify as male or female. Though researchers are still debating how best to measure the transgender or nonbinary populations, recent surveys have found 1-2% of the overall population identifies with the terms. As recently as 2017, an analysis of the limited surveying to that date had estimated only 0.4%. Adults under age 30 are especially likely to say they’re transgender—as high as 3-5% in recent surveys from Gallup Inc. and the Pew Research Center.

But with many surveys only giving the options of male or female, such individuals might answer incorrectly, or not at all. That means the data might be inaccurate, and those individuals’ needs go underserved.

Alexis Dinno, a professor at the OHSU-PSU School of Public Health in Portland, Ore., gives an example of the potential impact of such data holes: “I’m an epidemiologist, I’m an epidemiology professor who is transgender, and I cannot tell you what the top cause of death among the transgender population is, or what the major burden of disease is, because historically we have not asked.”

One challenge that researchers have faced is that a large share of the population doesn’t agree other options should even be provided. In 2018, the Pew Research Center asked in a survey whether forms should include options other than “man” and “woman” for people who don’t identify as either. A majority of respondents, 56%, said forms shouldn’t include other options; 42% said other options should be included.

The people who design surveys and work with their data say that revising the gender question isn’t intended to express a view on whether people’s opinions or identities are right or wrong, but merely to accurately measure the population.

“We didn’t want to exclude anybody who is nonbinary,” said Ashley Amaya, a senior survey methodologist at the Pew Research Center. “And we also don’t want to exclude folks who don’t believe gender is a social construct. We need to make sure we represent all those opinions.”

Part of what makes questions about gender identity so difficult is that they are being asked amid shifting social norms and deeply held convictions. Historically many people, and thus many surveys, have used sex and gender interchangeably. More recently, though, sex has come to refer to biological traits such as anatomy and chromosomes, while gender commonly refers to the social roles filled by men and women. Whether biological sex or gender is the more relevant trait might depend on what is being studied.

Figuring out how to change the surveys, however, is no easy task. Major polling organizations are experimenting with different approaches. The National Academies of Sciences, Engineering, and Medicine published a major study this year, at the request of the National Institutes of Health, to recommend how to collect such data in surveys and research, as well as in administrative, clinical and other health settings. The U.S. Census Bureau is considering testing changes in its largest annual survey, the American Community Survey.

Another challenge is what terminology to use. Because many survey respondents don’t distinguish between sex and gender, researchers have sought other ways to ask their questions.

Since 2020, Pew has settled on the question, “Do you describe yourself as a man, a woman, or in some other way?” in its regular surveys. The Gallup polling organization asks whether people are male, female or nonbinary. In occasional surveys, Pew and Gallup have also sometimes asked more detailed questions about views and attitudes regarding transgender or nonbinary populations.

The National Crime Victimization Survey, one of the first government surveys to ask a question on gender identity, beginning in 2016, asked whether people are male, female or transgender.

The General Social Survey in 2018 also included the option of “a gender not listed here” and allowed respondents to give a free-text response.

No measure is perfect. For example, some people identified as male on their birth certificates might now describe themselves as female, rather than transgender, though this generally fits most definitions of transgender. Some people consider themselves nonbinary, but not necessarily transgender.

Ms. Amaya of Pew notes that many Americans still don’t know what the term transgender means, and that this is especially challenging in Spanish-language polling.

When given free-form options, some people respond with protest responses such as “none of your damn business” that throw off data if such an answer is tabulated as a type of transgender.

Surveyors have found, however, that protest responses are less common than they expected.

Jeffrey Jones, a senior editor at Gallup, said the firm has seen higher refusal rates on questions about income than questions about LGBT identity. “People are more comfortable talking about sexual orientation and gender identity than how much money they make,” he said.

Familiarity with the terms is growing along with the number of people who don’t identify as male or female. Polling from Gallup and Pew shows a significant generational gap. Gallup’s questions find that among adults under 30, 3% say they are nonbinary, compared with 1% among those 30-49 years old and less than 1% among older groups, according to Mr. Jones. Pew has found 5.1% of adults under 30, 1.6% of adults age 30-49 and 0.3% of adults 50 and over.

When measuring overall lesbian, gay, bisexual and transgender identification, the generation gap is even larger. Among Generation Z, 20.8% tell Gallup they’re LGBT, compared with 10.5% of millennials and 2.6% of baby boomers.

Aliya Saperstein, a professor of sociology at Stanford University, and a leading researcher into survey methodologies, says a consensus is growing that the best approach to measuring gender is a two-question approach. This was the recommendation of the National Academies’ committee, of which she is a member.

For example, in 2021 the Census Bureau’s experimental Household Pulse Survey asked two questions: “What sex were you assigned at birth on your original birth certificate?” and then, “Do you currently describe yourself as male, female or transgender?”

The survey found 0.6% of adults identified as transgender. An additional 0.3% of people said they were female on birth certificates but now identify as male, and 0.4% of people said they were male on birth certificates and now identify as female.

It takes both questions to get at the full transgender population. The two-step approach has the added benefit of both reflecting the assessment of biological sex at birth and the gender identity people hold as adults.

In several of the Census Bureau’s major surveys, a respondent answers questions about everyone in their household. The bureau plans to study whether that biases responses about gender.

Dr. Saperstein said the two-step question could change, too. Some people object to being asked about their birth certificate, especially when it’s unclear how private their responses are. But, she said, that isn’t a reason not to ask these questions. Offering just two choices “collapses all sorts of important differences about people’s bodies and identities and how they live in the world,” she said. “We’ve already missed 10 years of massive change in this area and I hope we don’t miss another 10.”

Write to Josh Zumbrun at Josh.Zumbrun@wsj.com