By Edward T. Rincón, Ph.D.
Polls and surveys continue to play an important role in monitoring public sentiments and guiding campaign strategy as we have witnessed in the current presidential election and the Census 2020. With changes in the growth and composition of the U.S. population, accurately predicting election outcomes or achieving a complete Census count has become more challenging.
Evidence of this trend is shown by the different results reported by pollsters in the current presidential election and the struggle that the Census Bureau has encountered in completing their questionnaires in hard-to-count Black and Hispanic communities.
In addition to conflicting results, conducting surveys in today’s diverse demographic environment has resulted in a noticeable decline in survey quality, including lower response rates, more missing data, response bias, greater measurement error, and misleading indicators. While such differences in suvey outcomes can result from a variety of factors, pollsters sometimes overlook common practices in surveys of Blacks, Hispanics and Asians that contribute to these survey quality issues.
In the new book, The Culture of Research, Dr. Edward T. Rincón discusses a forensic analytic approach that he uses to describe common polling practices that bias survey outcomes and data quality in studies that include multicultural persons. For instance:
- Identity Bias: Which race or ethnic labels should one use when introducing the survey to a respondent? Are these labels offensive or outdated? Not all Hispanics like the term “Hispanic” or “Latino” while not all Blacks like the labels “Black” or “African American.” Asians often prefer their country of origin as their racial identity. Using the wrong label can lead to a quick rejection by the respondent.
- Coverage Bias: Is your sample representative of all members of a community or universe, or does it exclude important segments? For example, online surveys are often conducted with non-random samples of people who join panels and agree to complete a survey for an incentive. These panels often exclude Hispanics, immigrants, lower-income, and non-English speakers. This coverage bias could seriously limit the usefulness of the survey results if the excluded segments are important to decision makers.
- Sampling Bias: To properly represent a community, a pollster usually takes a random sample of persons or households in a specific geographic area. Sometimes, improperly drawn samples can distort survey outcomes. In one study of crime perceptions, Black and Hispanic survey respondents revealed more negative attitudes toward law enforcement than white respondents. Closer inspection, however, revealed that the sample of Black and Hispanics was drawn from lower income, high-density areas that have more negative interactions with law enforcement, while white respondents were chosen from varying income areas.
- Mode Bias: Most polls and surveys are conducted in only one mode, such as telephone, online or mail. Persons with vision and hearing disabilities, lower literacy or Internet access are often excluded when a survey is presented in a mode that challenges their ability to respond. Health data shows that Blacks and Hispanics reveal higher levels of such barriers and less likely to complete a survey if they are not provided a mode option that they can understand given their limitations.
- Language Bias: Most polls and surveys are presented only in English although most Hispanic and Asian immigrants prefer a native language survey when provided the choice. Improved response rates and more valid survey responses result when a respondent can communicate in the language that they best understand. Both English and native language options, however, should be presented at the same time to the respondent and not delayed for a later point in time.
- Weighting Bias: If a survey produced an imbalance on a particular attribute, such as more females than males, a pollster will ordinarily restore the proper balance by applying a mathematical weight derived from a reliable source like the most current Census Bureau data. Sometimes, these weights are not reliable and can distort reality. For example, The George Lopez Show was introduced several years ago to a U.S. Hispanic television audience that was eager for more relevant programming. The Nielsen television ratings, however, were showing smaller than expected audiences and the show was slated to be terminated. Our company conducted a national survey of Hispanics and discovered that the audiences were much larger than revealed by the Nielsen ratings. The problem? The ratings were improperly weighted by Nielsen estimates of Spanish-speaking television viewers, which resulted in lower ratings for the show whose humor appealed primarily to English-speaking Hispanics. The ensuing news coverage and debates with Nielsen research staff concluded with a decision to continue the show.
- A Renewed Industry Focus: With the advent of online surveys, the survey industry has become complacent with data collection tools that are less costly and easier to implement, a trend that places less emphasis on the needs of the target audiences. Rather than rely on one language or one mode of data collection, we propose the use of our multi-modal, multilingual method (MMML) that provides more than one language and mode of data collection to accommodate the diverse needs of multicultural populations.
- Purchasing Agents: One chapter in the book is devoted to survey quality issues that emerge when buyers of research have limited knowledge about multicultural research and diverse communities. This lack of knowledge encourages the buyer to place more weight on the lowest bid and less weight on the experience and qualifications of the survey vendor. The book offers recommendations to minimize this barrier to survey quality.
- Academic Inertia: The survey industry would benefit greatly if academic institutions would introduce courses and textbooks that address the unique challenges of conducting surveys in linguistically and culturally diverse communities. No such courses currently exist at U.S. academic institutions.
In summary, the new book is an excellent companion for persons interested in surveys of U.S. multicultural populations, which may include: (a) students who are writing a thesis or dissertation that involves a survey of multicultural persons, (b) faculty who teach traditional survey methods courses who need a good way to add multicultural content, (c) attorneys who litigate court cases that involve opinion surveys, (d) purchasing agents who want to raise the standards when selecting survey vendors, and (e) survey practitioners in public and private institutions who are responsible for ensuring an adequate representation of multicultural persons in diverse communities.
The added knowledge gained with reading The Culture of Research will add a level of expertise that is greatly needed in the survey industry.
Dr. Edward T. Rincón, Ph.D., is president of Dallas-based Rincón & Associates LLC and is a research psychologist. In June of 2021, Rincón will teach the first course in the state of Texas dedicated to multicultural research methods. The online course will originate from the College of Business at the University of Texas at Arlington.
