A Survey of LGBT Americans
Attitudes, Experiences and Values in Changing Times
Appendix 1: Survey Methodology
About the Survey of LGBT Adults
The analysis of the experiences and attitudes of the LGBT population in this report is based on a survey conducted April 11-29, 2013, among a sample of 1,197 self-identified LGBT adults, 18 years of age or older. The survey was conducted by the GfK Group using KnowledgePanel, its nationally representative online research panel. KnowledgePanel members are recruited through probability sampling methods and include both those with internet access and those without (KnowledgePanel provides internet access for those who do not have it and, if needed, a device to access the internet when they join the panel). A combination of random digit dialing (RDD) and address-based sampling (ABS) methodologies have been used to recruit panel members (in 2009 KnowledgePanel switched its sampling methodology for recruiting panel members from RDD to ABS). The panel comprises households with landlines and cellular phones, including those only with cell phones, and those without a phone. Both the RDD and ABS samples were provided by Marketing Systems Group (MSG). KnowledgePanel continually recruits new panel members throughout the year to offset panel attrition as people leave the panel. The survey was conducted in English.
Panel members complete an annual profile survey that includes a range of demographic, attitudinal and behavioral questions. Among these are questions about sexual orientation and gender identity. A total of 3,645 panelists classified as LGBT in response to the profile questionnaire, although some are no longer active in the panel. All active members of the GfK panel who identified as LGBT in the profile survey were eligible for inclusion in this study, but only one person per household (where more than one LGBT individual resided), randomly selected, was recruited for the study. In all, 1,924 panelists were invited to take part in the survey. All sampled members received an initial email to notify them of the survey and provide a link to the survey questionnaire. Additional follow-up reminders were sent to those who had not yet responded as needed. In order to avoid any potential response bias related to the topic of the survey, the invitations did not describe the content or target population of the study.
The Pew Research survey relies on a random sample of adults who have agreed to be a part of the GfK KnowledgePanel and complete surveys in exchange for a modest monetary reward. Interviews are conducted online, with no interviewer present. This is not an anonymous survey, but the level of trust established between respondents and the survey organization is likely to be high. Consequently, it is possible that some LGBT individuals who would be unwilling to disclose their status in other contexts are willing to identify themselves as LGBT in this panel.41 Considerable research on sensitive issues (such as drug use, sexual behavior and even attendance at religious services) indicates that the online mode of survey administration is likely to elicit more honest answers from respondents on a range of topics.42 The percentage of GfK KnowledgePanel respondents who identify as LGBT is 5.2%.
Identifying and Interviewing LGBT Respondents
A total of 1,924 members of the GfK KnowledgePanel who had identified as LGBT in their profile survey were recruited to take the current survey; 1,422 of these completed enough of the interview to determine their eligibility for the study. All of those recruited to the study were offered a $10 incentive to complete the interview; near the end of the field period, the incentive was increased to $20 to reduce the rate of non-response. All respondents who completed the screening questions (below) were given the incentive, regardless of whether they were determined to be eligible for the study or not.
The survey opened with general political questions and other topics unrelated to LGBT identity and experiences. Before the survey shifted to questions of particular relevance to LGBT individuals, respondents were asked the following questions about their sexual orientation and gender identity in order to reconfirm their eligibility for the survey:
Those whose answers indicated that they were not LGBT were asked the following question:
Overall, the vast majority of gay men (99%) and lesbian women (98%) reconfirmed as LGBT in the screening questions, though a few indicated that they belonged to a different category in the current survey than in their profile. Similarly, 85% of bisexuals reconfirmed as LGBT, including 78% who said they were still bisexual (2% identified as lesbian, 3% as gay and 2% as transgender). But most respondents who had indicated that they were transgender in the profile survey failed to reconfirm in any LGBT category. Among all those contacted for the Pew Research survey who identified as transgender in the profile survey (N=181), 23% reconfirmed as transgender in the current survey and an additional 11% identified as lesbian, gay or bisexual.
Respondents who did not reconfirm as LGBT (15%) were thanked for their time and told that the interview was over. In addition, five respondents who appeared qualified for the survey on the basis of these screening questions subsequently indicated during the interview that they were not LGBT; they, too, are excluded from the survey.
Twenty other individuals appeared qualified for the survey but broke off and did not complete the questionnaire, yielding a break-off rate of 1.1%. The final qualified sample size is 1,197. In all, 73% of those invited to take the survey screened in as qualified and completed the interview, screened out as not qualified, or indicated in their answers to other questions that they were not qualified. The cumulative response rate (CUMRR1) based on the household recruitment rate for the panel, the household profile completion rate, and the cooperation rate for this study is 7.4%. Including the average panel attrition rate in the calculation (CUMRR2) produces a response rate of 2.6%.43
Respondents who indicated that they are both transgender and either gay, lesbian or bisexual were asked which term (e.g., transgender or gay) they would prefer that the survey use in referring to them in the rest of the survey. This term was then automatically inserted at the appropriate places in subsequent questions.
The questionnaire was developed by the Pew Research Center. The design of the questionnaire was informed by the advice and counsel of two external advisers on the project: Gary J. Gates of the Williams Institute at UCLA and M.V. Lee Badgett of the University of Massachusetts, Amherst. In addition, the development of the questionnaire was informed by a focus group discussion of LGBT adults in the Washington, DC, area that met on March 26, 2013, at the offices of the Pew Research Center. The online questionnaire also benefited from comments and advice by GfK staff. As a further check on the questionnaire, the survey was administered as a slow launch, i.e., with a small group of panelists at the beginning of the field period. These respondents were also asked an additional open-end question to solicit feedback on any problems with clarity or other issues in the questionnaire. The Pew Research Center had final authority and responsibility for the design of the questionnaire and retains sole responsibility for the analysis and interpretation of survey findings. The median length of the questionnaire was 19 minutes.
The final qualified sample was weighted in several steps. All members of the KnowledgePanel carry a weight designed to produce a nationally representative sample of the U.S. adult population. This weight uses an iterative technique that matches gender, age, race/ethnicity, education, region and metropolitan area to parameters from the April 2013 Census Bureau’s Current Population Survey (CPS). In addition, the sample is weighted to match household income and homeownership status to a parameter from the March 2012 CPS survey and to match current patterns of internet access from the July 2011 CPS survey. This weight is multiplied by an initial sampling or base weight that corrects for differences in the probability of selection of various segments of the sample and by a panel weight that adjusts for any biases due to nonresponse and noncoverage at the panel recruitment stage (using all of the parameters described above). Details about the KnowledgePanel panel-level weights can be found at http://www.knowledgenetworks.com/knpanel/KNPanel-Design-Summary.html.
Because the final sample is narrowed to LGBT members of the panel, an additional weighting step was taken to correct for any potential biases introduced at this stage. This step weighted the LGBT sample selected for the survey to match the characteristics of the full sample of 3,645 members of the panel who are identified as LGBT, with an adjustment to correct for the different rates by which lesbians, gays, bisexuals and transgender individuals reconfirmed their eligibility for the study when interviewed. The sample was weighted using an iterative technique that matches gender, age, race/ethnicity, education, region, metropolitan area and household income among LGBT individuals to targets derived from the weighted sample of all LGBT individuals in the panel.
An additional adjustment was made to align the Hispanic composition of the sample to a target based on a weighted average of the KnowledgePanel (N=3,645 LGBT individuals), the 2008-2012 General Social Surveys (LGBT N=188) and a telephone survey of 2,669 LGBT respondents from RDD telephone surveys conducted June-August 2012 by the Gallup Organization. The KnowledgePanel survey estimated the Hispanic share of the LGBT population at 22.5%, compared with 14.0% in the Gallup sample and 15.6% in the General Social Survey. The weighted average of the Hispanic share of LGBT respondents in these three surveys was computed by using the effective sample sizes of each survey to determine its relative contribution to the average.
Sampling errors and statistical tests of significance take into account the effect of weighting at each of these stages. The margin of sampling error at the 95% confidence level is plus or minus 4.1 percentage points for results based on all LGBT respondents (N=1,197). Sample sizes and margins of error for key subgroups of the sample are shown below. There were too few confirmed transgender respondents to tabulate separately (N=43). Sample sizes and sampling errors for other subgroups are available upon request.
In addition to sampling error, one should bear in mind that question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of opinion polls.
Profile of Pew Research Center 2013 LGBT Survey Respondents
The LGBT survey sample is evenly split by gender—50% of respondents identify as male, and 49% as female. Among bisexuals, almost three-fourths (73%) are females, and 27% are men.
Among respondents in the survey, 30% are ages 18-29. Some 39% are ages 30-49, and 23% are ages 50-64. Just 9% are ages 65 or older. Bisexuals are younger, on average, than gay and lesbian respondents. Fully 38% are younger than age 30, as compared with 31% of lesbians and only 21% of gay men.
Two-thirds (66%) of respondents are non-Hispanic whites, and 17% are Hispanic. Smaller shares are non-Hispanic blacks (10%). Bisexual respondents include a significantly higher share of blacks (14%) than do gay men (7%). Conversely, gay male respondents are significantly more likely to be Hispanic (21% are) than bisexual respondents (12% are).
Overall, some 93% of respondents in the LGBT sample were born in the United States. The remainder were born in a U.S. territory or a foreign country.
About one-third (32%) of LGBT respondents have a bachelor’s degree or more, and an additional 36% have some college experience. One-third (33%) have a high diploma or less. Bisexual respondents are far less likely than gay or lesbian respondents to have a bachelor’s degree (23% do); this is probably in part a reflection of their younger age profile.
Some 39% of LGBT respondents report annual family incomes of less than $30,000; another 39% report incomes of $30,000 to $74,999; and 20% have incomes of $75,000 or more. Gay male respondents are particularly affluent, with more than one-fourth (27%) having incomes of $75,000 or more, while bisexual respondents are less affluent—only 12% report such high family incomes.
Survey respondents are located throughout the U.S. Fully one-third (34%) live in the South, and 27% reside in the West. One-fifth (20%) are in the Northeast, and the same share resides in the Midwest. Respondents are concentrated in more urban areas—89% live within a metropolitan statistical area (MSA), while 11% reside in more rural parts of the country.
- Although providing only anecdotal evidence in support of this possibility, one respondent, age 66, wrote that he was “out” only to this survey panel. ↩
- See e.g., Frauke Kreuter, Stanley Presser and Roger Tourangeau. 2008. Social Desirability Bias in CATI, IVR, and Web Surveys: The Effects of Mode and Question Sensitivity. Public Opinion Quarterly, 72(5): 847-865. ↩
- Callegaro, Mario and Charles DiSogra, 2008. “Computing response metrics for online panels.” Public Opinion Quarterly 72(5). pp. 1008-1032. ↩