Many research studies have reported on the elevated health risk and deviance of youth who identify as lesbian, gay, bisexual, or questioning (LGBQ). But a study using US national data suggests that many of those estimates may be overstated and that LGBQ youth risk and deviance is not as different from heterosexual youth as many studies claim.
The research was led by Joseph Cimpian, an associate professor of economics and education policy at New York University's Steinhardt School. The study found that "mischievous responders," research survey respondents who mislead researchers by providing extreme and untruthful responses to multiple items, have significantly affected overall estimates of health disparities between LGBQ and heterosexual youth.
"Not all adolescents take surveys as seriously as researchers would like," said Cimpian. "Some youth may find it funny to give exaggerated responses, reporting they are impossibly tall, have never been to the dentist, and eat carrots with extreme frequency. Problematically, these mischievous responders also find it funny to falsely claim they are 'LGBQ' and to exaggerate risky outcomes such as drug use, thereby inflating estimates of LGBQ youth drug use."
Cimpian utilised data from the Youth Risk Behaviour Survey (YRBS), a large US Centres for Disease Control and Prevention public health dataset on more than 140,000 high school students, to identify potentially mischievous responders and then examined how the inclusion of their responses may have an impact on estimates of health disparities in 20 different outcomes, from hopelessness to heroin use, between LGBQ and heterosexual youth.
Likely mischievous responders were identified using machine-learning techniques that seek out unusual patters of responses to items that should be theoretically unrelated to sexual identity (how often someone eats carrots, or their height) on self-administered questionnaires. After identifying the most likely mischievous youths,
Cimpian and his research team were able to assess the data for changes in the estimates of the youth health disparities.
After removing the most likely mischievous respondents, Cimpian and his colleagues found that youth health disparities between LGBQ and heterosexual males dropped by an average of 46%, while dropping by 23% on average among females. Cimpian pointed out, however, that not all outcomes were affected equally.
"We found that drug and alcohol related disparities were most affected by removing potentially mischievous responders. With the suspect cases, researchers would conclude that LGBQ youth were drinking alcohol and using drugs like heroin and ecstasy at substantially higher rates than their heterosexual peers. But when the suspect cases were removed, those disparities diminished considerably and vanished in some instances," reported Cimpian.
"Yet, outcomes like being the victim of bullying and considering suicide were virtually unaffected," Cimpian continued. "This suggests that there are important health disparities observed in the literature, largely related to the victimisation LGBQ youth experience, that are not caused by aberrant data, and that policymakers and health professionals need to pay careful attention to."
The researchers then repeated their analyses on a separate nationally representative dataset from the Centres for Disease Control and Prevention of over 14,000 students and replicated their initial findings, this time finding average reductions of 64% among males and 12 percent among females, further suggesting that mischievous responders affect male disparities more than female disparities. They also performed several additional statistical tests to ensure the patterns they observed were robust to research modifications. Overall, the research provides strong evidence that mischievous responders contribute to inflated estimates of the risk and deviance of LGBQ youth. The changes in disparity estimates are most pronounced for drug- and alcohol-use and also for suicide attempts, and also among males more than among females. Disparity estimates for being bullied, feeling sad or hopeless, and thinking about suicide were not noticeably affected.
"This research demonstrates why it is essential for researchers to perform a wide range of tests to ensure their survey data lead to robust findings," Cimpian added. "The research we put out can have a substantial effect on public policy and on what gets communicated to the public. It is critical that we take the necessary steps to make sure that what we present is accurate and to be forthcoming about the assumptions and limitations of research."
Abstract
Objectives: To determine how sensitive estimates of lesbian, gay, bisexual, or questioning (LGBQ)–heterosexual youth health disparities are to the presence of potentially mischievous responders.
Methods: We used US data from the 2015 Youth Risk Behavior Survey, pooled across jurisdictions that included a question about sexual identity for a total sample of 148 960 students. We used boosted regressions (a machine-learning technique) to identify unusual patterns of responses to 7 screener items presumably unrelated to LGBQ identification, which generated an index of suspected mischievousness. We estimated LGBQ–heterosexual youth disparities on 20 health outcomes; then we removed 1% of suspected mischievous responders at a time and re-estimated disparities to assess the robustness of original estimates.
Results: Accounting for suspected mischievousness reduced estimates of the average LGBQ–heterosexual youth health disparity by up to 46% for boys and 23% for girls; however, screening did not affect all outcomes equally. Drug- and alcohol-related disparities were most affected, particularly among boys, but bullying and suicidal ideation were unaffected.
Conclusions: Including screener items in public health data sets and performing rigorous sensitivity analyses can support the validity of youth health estimates.
Authors
Joseph R Cimpian, Jennifer D Timmer, Michelle A Birkett, Rachel L Marro, Blair C Turner, Gregory L Phillips
[link url="https://www.nyu.edu/about/news-publications/news/2018/november/-mischievous-responders–taint-lgbq-health-estimates-in-national.html"]New York University material[/link]
[link url="https://ajph.aphapublications.org/doi/10.2105/AJPH.2018.304407"]American Journal of Public Health abstract[/link]