A systematic review of publications assessing reliability and validity of the Behavioral Risk Factor Surveillance System (BRFSS), 2004-2011.
Pierannunzi C, Hu SS, Balluz L. · 2013
Read A systematic review of publications assessing reliability and validity of the Behavioral Risk Factor Surveillance System (BRFSS), 2004-2011. by Pierannunzi C, Hu SS, Balluz L. (2013) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc3622569.txt.
Introduction
Carol Pierannunzi, Shaohua Sean Hu, and Lina Balluz audited the trustworthiness of a giant American health survey in BMC Medical Research Methodology in 2013. The Behavioral Risk Factor Surveillance System polls adults by telephone about smoking, drinking, weight, and disease, guiding policy nationwide. Falling response rates and new cell-phone sampling forced method changes, raising fair questions about whether its numbers still hold. Their review gathered every reliability and validity study of the system published from 2004 to 2011.
Judged against a scoring rubric that favored repeated test-retest designs and comparisons with physical measurements, the literature mostly reassured. The survey's prevalence rates stood close to those of other national surveys built on self-report, with specific gaps noted category by category. Against surveys using examinations and instruments, agreement weakened, as expected when people describe themselves. The authors conclude the system remains a usable national gauge while its methods evolve.
Catalog record
| CREATOR | Pierannunzi C, Hu SS, Balluz L. |
|---|---|
| YEAR | 2013 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A systematic review of publications assessing reliability and validity of the Behavioral Risk Factor Surveillance System (BRFSS), 2004–2011 By Carol Pierannunzi, Shaohua Sean Hu, Lina Balluz Published in: BMC Medical Research Methodology DOI: 10.1186/1471-2288-13-49 ABSTRACT Background In recent years response rates on telephone surveys have been declining. Rates for the behavioral risk factor surveillance system (BRFSS) have also declined, prompting the use of new methods of weighting and the inclusion of cell phone sampling frames. A number of scholars and researchers have conducted studies of the reliability and validity of the BRFSS estimates in the context of these changes. As the BRFSS makes changes in its methods of sampling and weighting, a review of reliability and validity studies of the BRFSS is needed. Methods In order to assess the reliability and validity of prevalence estimates taken from the BRFSS, scholarship published from 2004–2011 dealing with tests of reliability and validity of BRFSS measures was compiled and presented by topics of health risk behavior. Assessments of the quality of each publication were undertaken using a categorical rubric. Higher rankings were achieved by authors who conducted reliability tests using repeated test/retest measures, or who conducted tests using multiple samples. A similar rubric was used to rank validity assessments. Validity tests which compared the BRFSS to physical measures were ranked higher than those comparing the BRFSS to other self-reported data. Literature which undertook more sophisticated statistical comparisons was also ranked higher. Results Overall findings indicated that BRFSS prevalence rates were comparable to other national surveys which rely on self-reports, although specific differences are noted for some categories of response. BRFSS prevalence rates were less similar to surveys which utilize physical measures in addition to self-reported data. There is very little research on reliability and validity for some health topics, but a great deal of information supporting the validity of the BRFSS data for others. Conclusions Limitations of the examination of the BRFSS were due to question differences among surveys used as comparisons, as well as mode of data collection differences. As the BRFSS moves to incorporating cell phone data and changing weighting methods, a review of reliability and validity research indicated that past BRFSS landline only data were reliable and valid as measured against other surveys. New analyses and comparisons of BRFSS data which include the new methodologies and cell phone data will be needed to ascertain the impact of these changes on estimates in the future. BACKGROUND Health officials recognize the need for accurate data for purposes of program planning, policy evaluation and estimation of health risk prevalence [1]. Telephone surveys have been a staple of data collection methods, in part due to their efficacy and reduced costs. The behavioral risk factor surveillance system (BRFSS) is a state-based telephone survey coordinated by the Centers for Disease Control and Prevention (CDC). Self-reported information regarding chronic conditions and health risk behaviors is collected throughout the year using telephone survey methods in all 50 states, Washington DC, Guam, Puerto Rico and the US Virgin Islands. More than 400,000 adults complete the survey annually, making the BRFSS the largest telephone survey in the world [2]. Individual states use data from the BFRSS to assess need and plan public health priorities. These data have been essential to states and local jurisdictions and have historically been shown to be useful as sources of information [3]. For many states the BRFSS is the sole source of health and health risk behavior data available to policy makers. Given the importance of the BRFSS data to its constituent jurisdictions, continuous validation of findings is requisite. CDC, perforce, conducts numerous internal checks on BRFSS data. Independent practitioners have also tested BRFSS reliability and validity within their areas of interest. A comprehensive reliability/ validity study of BRFSS was conducted earlier by Nelson [4] examining articles appearing in peer-reviewed journals through the 1990s. They found that most measures taken from the BRFSS were moderately reliable and valid and that many were highly reliable and valid. Using the Nelson study as a framework to examine reliability and validity studies by topic, this research compiled information on reliability and validity testing of BRFSS data from 2004 through 2011 by a number of researchers in peer- reviewed journals and assesses BRFSS data by question topic. In its current form, the BRFSS not only produces a large data set covering a number of health risk behaviors, but also provides a number of services to states which are engaged in the process of data collection [5]. Generation of samples, weighting to account for demographic and geographic variables and programming to support report writing are provided to state coordinators and their staffs. Traditionally the BRFSS was based exclusively on landline random digit dialing (RDD) samples of households. Random selection among adults within households was also conducted. In 2008 in response to the growing percentage of cell phone only households in the US [6], cell phone samples were piloted and in 2009 all states included cell phone samples in their data collection process. In 2011 the public release of the BRFSS included both landline and cell phone data for the first time. A second important change in 2011 was the move to a new weighting system which incorporates cell phone data as well as including new variables (education, marital status and home ownership) as controls. The BRFSS is one of several surveys which compile health data in a variety of modes and methods. Many researchers review BRFSS prevalence indicators in terms of prevalence rates from other surveys which can be used to produce national estimates. These include: National health interview study (NHIS) The NHIS is conducted continuously throughout the year using face-to-face interviews in respondents’ homes. Basic health information is collected for all family members, by proxy if necessary. Additional health and socio-demographic information, including health risk behaviors, is collected, by self-report, from one adult family member [2]. National health and nutrition examination survey (NHANES) The NHANES collects information on adults and children and combines face-to-face interviews with physical examination measures. The NHANES has been conducted periodically since the early 1960s. In 1999 the NHANES became a continuous survey with data released every two years [7]. National survey on drug use and health (NSDUH) The NSDUH is annually compiled from face-to-face interviews. It focuses primarily on substance abuse among respondents 12 years of age and older [1]. Current population survey (CPS) The CPS is conducted by the Bureau of Labor Statistics and the Census Bureau [8]. Data are combined from telephone survey and other modes of collection. Data are published for respondents over 15 years of age. National survey of family growth (NSFG) The NSFG gathers information using personal interviews [9]. Topics include family life, marriage and divorce, pregnancy, infertility, use of contraception, and men's and women's health. Adults and teenagers 15 and over are selected as participants. Despite studies which support findings from self-reported information [3,10], for some scholars and practitioners self-reported data are perceived to be unreliable estimates of health factor prevalence. Moreover in recent years, telephone survey response rates have declined [5,11]. BRFSS response rates have also declined from medians in the 70–75 percent in the 1980s to a median of 57 percent in 2010 [2], resulting in