Defining harmful news reporting on community firearm violence: A modified Delphi consensus study Authors: Jessica H. Beard, Evan L. Eschliman, Anita Wamakima, Christopher N. Morrison, Jim MacMillan, Jennifer Midberry, Claudio Terranova, Claudio Terranova, Claudio Terranova, Claudio Terranova Published: 2024 ABSTRACT Community firearm violence (CFV), including fatal and non-fatal shootings that result from interpersonal violence, disproportionately harms people from marginalized racial groups. News reporting on CFV can further exacerbate these harms. However, examining the effects of harmful news reporting on CFV on individuals, communities, and society is hindered by the lack of a consensus definition of harmful reporting on CFV. In this study, we aimed to define harmful reporting on CFV. We used a modified, three-round Delphi process to achieve consensus among diverse stakeholders. Round 1 sought to assess consensus on 12 potentially harmful news content elements for three levels of harm (individual, community, and society). Round 2 invited panelists to rate the severity of each news content element at each level of harm. Round 3 asked panelists to agree or disagree with the panel’s median severity rating of each element at each level of harm. Twenty-one panelists were recruited from three expertise groups (lived experience of CFV, journalism practice, scholarship) and all panelists completed all three rounds. In Round 1, no negative consensus was achieved for any of the proposed news content elements. In Round 2, panelists assigned moderate to severe harm ratings for all but two news content elements, and median harm ratings for each element varied across the different levels of harm. In Round 3, panelists reported high levels of agreement for each harm rating at each level. This modified Delphi process yielded a definition of the 12 elements that comprise harmful news reporting on CFV and severity ratings of harm caused by each element at each level according to expert consensus. Future work will use these results to evaluate and intervene on harmful reporting on CFV. Reducing harm from reporting on CFV can help address this health disparity and support evidence-based approaches to this urgent public health issue. Introduction Community firearm violence and structural racism Firearm-related injury is an increasing threat to public health in the United States (US) [1–6]. Community firearm violence (CFV), defined as fatal and non-fatal shootings that result from interpersonal violence, disproportionately harms people from structurally marginalized racial groups and occurs predominantly in communities where racist policies and disinvestment have contributed to the presence of concentrated poverty [7–14]. These race and place-based disparities in CFV incidence are rooted in the social and structural determinants of health, including historic and ongoing structural racism [7,8,10,14]. Racialized societal structures in the US perpetuate CFV through contemporary policies and actions including mass incarceration, state sanctioned violence, food insecurity, and inequitable access to health, education, and social services, among others [15–17]. These structural forces shape an individual’s risk for CFV and must be addressed to effectively decrease CFV incidence and promote community health and well-being [17]. News reporting on community firearm violence in the United States News coverage of CFV has the potential to deepen racialized structural inequities by shaping how the public and policy makers understand and respond to CFV [18–23]. In the US, news outlets rarely contextualize the complexity of CFV as an issue rooted in historic and contemporary structural racism [24–28]. Instead, news reports on violence typically emphasize individual blame over structural causes and have been critiqued for producing stereotypical narratives about Black people and communities [26,29–33]. The transmission of messaging about CFV—and its potential harms—depends in large part on how a news story is told, a concept called framing. Framing in news media occurs when journalists make choices about which content to include or exclude in a story, whose perspectives to highlight, and what attributes of an event to emphasize over others [27]. Those decisions make certain facets of a story more salient to audiences, and when the same messages are repeated in the news over time, people are likely to internalize those ideas [27,34,35]. A specific type of framing analysis that communication scholars have applied to news coverage of violence and crime is to distinguish between episodic and thematic framing [18,26,28]. Episodic framing of CFV is when a story focuses on a shooting event in isolation, while thematic framing places the event in its relevant social and structural contexts [18,24–26]. The primary source of a news report is also a key aspect of framing. In the case of CFV, most news reports rely on police sources and rarely include the perspectives of firearm-injured people and co-victims [26,28,36,37]. Decades of journalism and communication research have demonstrated that the crime frame is the dominant frame in news stories about violence [18,24–26,28,36]. Episodic crime framing is therefore news framing that is episodic, defines violence as a crime issue, and privileges police narrators above other viewpoints. Studies indicate that episodic crime framing in news reports about violence has detrimental societal level impacts; it can lead news audiences to blame individual victims, reinforce racist stereotypes about the people and places impacted, suggest an unfounded efficacy to policing as a means to prevent violence, and undermine effective public health responses [18,21,24–26,36,38,39]. Because of these theoretical and empirically demonstrated harms, scholars and journalism leaders have long advocated for reconsidering episodic crime framing in favor of other approaches that allow for more nuanced dialogue on complex issues like violence. One alternative frame posited by scholars and journalism educators for stories on CFV is the public health frame, which may include epidemiologic context and root causes along with public health narrators, visuals, and solutions [20,22,25,28,40]. Additionally, the Solutions Journalism Network provides guidance to journalists on how to rigorously report on social problems like violence, highlighting the importance of including complex narratives and critical evaluation of solutions to create more impactful news stories [41]. In addition to a relatively large body of media effects research on news about violence and crime at the societal level, a few recent studies have examined the impact of news reports on individuals and local communities. For example, an interview-based study investigated the mental health consequences for Black participants of engaging with news media about police brutality against Black Americans [42]. Participants explained that such stories caused them to feel debilitating sadness, remain in a state of hyper-arousal and perpetual fear, and experience a sense of helplessness [42]. Another recent study, focused specifically on CFV, analyzed interviews with firearm-injured people and found that episodic crime narratives of participants’ shootings made them feel dehumanized and added to their trauma [43]. Firearm-injured people in that study described how specific news elements, including graphic content, factual inaccuracies, and naming the treating hospital resulted in distress, harms to reputation, and threats to personal safety [43]. At the community level, some participants in the study expressed beliefs that harmful reporting may be driving increased CFV incidence in their neighborhoods by generating fear, which increases firearm purchasing and carrying [43]. Another community-level impact was observed in an online survey of Chicago residents, which found that participants who were more fearful of crime experienced higher levels of depression; however, this effect was dampened for individuals who paid closer attention to positive local news [44]. Taken together, these studies suggest that reducing harmful content elements in news coverage of CFV may ameliorate negative effects on individuals and communities faced with high rates of CFV [42–44]. Defining harmful reporting as a next step for intervention Events involving human suffering present journalists with the inherent tension of carrying out two key, but sometimes conflicting, principles of news: 1) to seek truth and report it and 2) to minimize harm [45]. Therefore, defining which specific content in CFV stories is harmful and developing reporting guidelines can illuminate (though not fully resolve) complicated newsroom debates about how journalists can best balance their duties to both inform the public and reduce harm. There is clear precedent for research exploring harmful news content to inform journalistic policy development as a public health intervention. With empirical support, journalistic guidelines that provide special instruction to minimize harm to victims and audiences cases of suicide, mass shootings, sexual assault, abuse, and crime involving minors have been widely accepted [45,46]. For example, studies demonstrating that harmful reporting approaches are associated with increases in suicide incidence led to the adoption of revised newsroom practices endorsed by public health experts [47–53]. These guidelines contain specific harm-reduction recommendations for media, including avoiding: prominent story placement, sensationalizing headline/content, glamorization or oversimplification of suicide, discussing the suicide method, and repeated reporting about the same suicide [47,49]. When media reports limit these harmful approaches, portray suicide as preventable, and provide resources, these modifications have been shown to result in population-level decreases in suicide rates [51–53]. Conceptualizations of harmful reporting on CFV remain far less developed than those that exist for reporting on suicide and mass shootings. Although there is clear evidence of negative societal level impacts of episodic crime framing and emerging research is uncovering the individual and community level harms of existing news coverage of CFV, guideline and intervention development remains hindered by a lack of a consensus definition of harmful reporting on CFV [18,24,26,43]. Codifying the definition of harmful reporting on CFV through expert consensus is an essential next step to inform measurement of the extent and direct effects of harmful reporting in US news and to support the creation of evidence-based strategies for journalists to minimize harmful reporting on CFV. Because CFV largely impacts structurally marginalized people and places, harmful reporting on CFV also exerts disproportionate impact on people and communities subjected to systemic disadvantage. As such, harmful reporting on CFV may contribute to health inequities and perpetuate sustained disadvantage in a mechanism similar to other discriminatory practices and policies [8,10,14]. If this is true, then modifying news media approaches to limit harmful elements is a potential target for mitigating contemporary structural racism, minimizing disparities in CFV incidence and for helping CFV prevention efforts. We utilized the social-ecological model of health as a framework to operationalize potential levels of harmful reporting on CFV [54–56]. We considered that individuals are nested within their social and physical environments at multiple levels (e.g., neighborhoods, community, society) and that harmful reporting of CFV may have adverse effects at each level [54–56]. The model presented in Fig 1 reflects these theorized levels and potential mechanisms of harm, informed by a review of previous research and work with our partners at the Philadelphia Center for Gun Violence Reporting (PCGVR), a community-based organization that supports ethical media reporting on CFV [18,24,26,38,39,43,57,58]. The three levels of harm corresponding to our social-ecological model considered throughout this study are defined as follows: Individual: Firearm-injured people and/or co-victims involved in the shooting being reported on; Individual: Firearm-injured people and/or co-victims involved in the shooting being reported on; Community: Firearm-injured people and/or co-victims who have been affected by previous shootings; Community: Firearm-injured people and/or co-victims who have been affected by previous shootings; Society: News audiences viewing, reading, and/or listening to the content and/or society at large. Society: News audiences viewing, reading, and/or listening to the content and/or society at large. Social-ecological model of impacts of harmful reporting on community firearm violence (CFV). Study objective In this study, we aimed to define harmful reporting on CFV using a modified Delphi method to achieve consensus among key expert-stakeholders identified in collaboration with community partners at PCGVR. We report on the Delphi process along with our findings about the specific news content elements that are potentially harmful to firearm-injured people, impacted communities, and society across our pre-defined three levels of harm (i.e. individual, community, and society). Materials and methods Panelists We conducted this modified Delphi consensus study in collaboration with journalists and individuals with lived experience of firearm violence at PCGVR. PCGVR is a Philadelphia-based organization that supports “ethical, empathetic, and impactful” reporting on CFV through novel media created by firearm-injury survivors, educational programs for journalists, and community-engaged research [58]. We defined expertise on harmful reporting on CFV broadly and inclusively, working with PCGVR leadership to identify key stakeholders from three distinct expertise backgrounds: (1) lived experience experts (people who have been shot and survived as well as co-victims, who are the loved ones of firearm-injured people); (2) journalism practice experts (journalists with experience covering CFV); and (3) scholars (researchers of news reporting on violence). We aimed to include 20 panelists from these stakeholder groups, a larger sample size for the Delphi method to accommodate potentially varied perspectives [59]. Identifying nationally renowned academics and drawing on PCGVR’s local network, we invited 25 potential adult panelists, including 9 lived experience experts, 8 journalism practice experts, and 8 scholars. Study design, procedures, and ethical statement The Delphi method is a well-established and systematic process for iteratively developing consensus among a panel of experts [59–61]. This approach is especially practical where knowledge is “uncertain or incomplete, and human expert judgment is better than individual opinion” [61]. One of the key strengths of the Delphi process is panelist anonymity, which allows panelists to share their perspectives without conflicts, removing biases that might come from individual dominance or group conformity [61]. Other features of the Delphi method include controlled feedback by the research team and iterative discussions between rounds [61]. Following each round, the Delphi research team analyzes both quantitative and qualitative results, presenting these to panelists and using the results to develop the next round. We chose the Delphi method for this study for several reasons. Discussing firearm violence can be an emotional topic, especially for those with lived experience. The Delphi method allows panelists to share their perspectives anonymously ensuring that each perspective is considered and incorporated without risk of interpersonal disagreement. Additionally, because the topic of harmful reporting is highly complex, the Delphi method provides the opportunity to bring diverse experts into nuanced conversation that would not be achieved through a traditional survey or other quantitative research methods. We designed this modified Delphi consensus study to include three rounds with a plan to stop the study after the third round [59,60]. No other stopping criteria were considered in the initial design. The study goal was to achieve expert consensus on what constitutes harmful reporting on CFV. In Round 1, the objective was to examine the degree of expert consensus on potential harmful reporting elements identified from prior research. In Round 2, the objective was to evaluate panelist perspectives on the severity of each potentially harmful reporting element that was not dropped due to negative consensus in Round 1. Following recommendations for Delphi consensus studies, the a priori objective of Round 3 was to allow panelists to evaluate the results of the previous rounds, revise their judgements, and specify any reasons for being outside consensus [59,60]. Potential panelists were contacted over email to introduce the study on October 31, 2023. They were provided a link to RedCap to complete Round 1, which took them to a written informed consent statement. Individuals who agreed to participate clicked “Yes” in response the consent question and then proceeded with answering the questions in Round 1. Basic demographic information was collected about each panelist along with information on their relevant expertise. The three rounds were carried out over approximately 5 months, allowing time between rounds for analysis, creation, and iteration of the subsequent rounds. The final round was completed on March 7, 2024. Panelists received $75 upon completion of each survey, for a total of $225 compensation for participation in all three rounds. Identifying information about the panelists, which was collected for the purposes of compensation, was stored separately from questionnaire responses to encourage open dialogue and responses and maintain anonymity. The Temple University Institutional Review Board approved this study. Round 1 development and analysis We identified a set of potentially harmful news content elements informed by the prior qualitative interview study with firearm-injured people and by review of the journalism and communication literature about the harms of episodic crime framing in reporting on crime and violence [18,20,24,26,43]. This list of elements and corresponding descriptions was compiled by the first author (JHB), edited in two rounds with another author (JM) and then presented to the research team for feedback and in-team consensus. This approach ensured that the perspectives of people with lived experience of CFV were prioritized in the development of content for Round 1 [43]. Potentially harmful news content elements included in Round 1 are summarized in Table 1. 10.1371/journal.pone.0316026.t001Table 1Potentially harmful news content elements included in Round 1 of the modified Delphi consensus study.Potentially Harmful News Content ElementDescriptionGraphic contentNews coverage includes graphic or explicit news content about firearm violence, such as a video of shooting or a detailed description of the crime scene.Clinical conditionNews coverage of a shooting includes information on the clinical condition of a firearm-injured person (e.g. “critical” or “stable”).Number of gunshot woundsNews coverage of a shooting includes information on the number of gunshot wounds of a firearm-injured person.Name of treating hospitalNews coverage of a shooting includes the name of the treating hospital of a firearm-injured person.Relationship between firearm-injured person and perpetratorNews coverage of a shooting includes information on the relationship between the firearm-injured person and the alleged perpetrator of the shooting.MugshotNews coverage of a shooting includes a mugshot of the alleged perpetratorAbsence of a follow-up storyThere is no follow-up story (e.g. an update on how a community has fared after a shooting or an interview with a survivor about their recovery) after the initial “breaking news” coverage.Episodic framingNews coverage of firearm violence that focuses only on a specific shooting event and does not include context, root causes, or solutions to firearm violence.Only law enforcement narratorsNews coverage of firearm violence that only or predominantly presents the perspectives of law enforcement representatives (e.g. police).Missing perspective of firearm-injured personNews coverage of a shooting that does not include the perspectives of the firearm-injured person and/or their loves ones.Missing community perspectiveNews coverage of firearm violence that does not include the perspectives of people from the impacted community.Does not explore solutionsNews coverage of firearm violence that does not explore potential solutions. Potentially harmful news content elements included in Round 1 of the modified Delphi consensus study. Traditionally, Delphi consensus studies are conducted in four rounds, with the first round focused on open-ended questions and idea generation [59–61]. Because the existing research on harmful reporting on violence is relatively robust, we elected to conduct a modified Delphi study in three rounds. As such, Round 1 was mostly structured with some space for open-ended responses from panelists throughout [59,60]. A 7-point Likert scale was used to assess the extent to which panelists agreed or disagreed that each potentially harmful news content element could cause harm at each level of harm (i.e. individual, community, society) [54–56]. In all rounds, panelists were instructed to answer questions from their perspective based on their expertise in personal experience, professional experience, and/or research. Consistent with recommendations for Delphi studies, consensus was defined a priori according to the proportion of responses within a range [59,60]. For Round 1, we were most concerned with negative consensus, defined as 80% or more of the panelists responding “Disagree” (6) or “Strongly Disagree” (7) to the question of potential harm of each news content element on the 7-point Likert scale [59,60]. We planned to drop any news content element at any level of harm from subsequent rounds in the event of negative consensus. Panelist demographic data were analyzed descriptively furnishing proportions, medians, and interquartile ranges. Kruskal-Wallis tests were used to examine differences in responses between expertise groups for each news content element and for each level of harm. Responses to open-ended questions were analyzed via a pragmatic analysis approach that included inductive thematic coding by a single team member (ELE) and full-team discussion of responses and emergent themes [62–64]. When all team members met together to discuss the open-ended responses, collaborative consensus was employed in order to identify any potentially harmful news content elements or additional question blocks that could be incorporated into future rounds [65]. Round 2 development and analysis Given that no news content element from Round 1 achieved negative consensus, we assessed the severity of all potentially harmful news content elements included in Table 1 in Round 2. We asked panelists to rate the severity of potential harm for each news content element for each of the three levels of harm on an 11-point scale of 0 to 10. We provided the same detailed instructions to all panelists and provided descriptive labels on intermediate values (i.e., 0 indicated no harm, 5 indicated some harm, and 10 indicated extreme harm). Panelists were then asked to select the top three most harmful news elements to allow panelists to emphasize which elements they found especially harmful (e.g., in case they rated more than three elements as “extreme harm”). Based on the high endorsement of the harm of graphic content across all levels and the thematic analysis of open-ended responses from Round 1, additional questions were included in Round 2 to identify specific content elements panelists would consider graphic and/or explicit and to assess how the medium of the news report may impact the level of harm of graphic and/or explicit content. Twelve potentially harmful content elements were included, and panelists were instructed that this content could be present in video, audio, or still photographs. Panelists were also asked to rate graphic and/or explicit content of firearm violence presented in four potential formats (disturbing video, disturbing still photographs, disturbing audio, and disturbing detailed verbal description) on the same 11-point harm scale by level of harm. Open-ended questions were included to allow panelists to provide support for their ratings of harmful content elements, offer further examples of harmful graphic and/or explicit content, and add information not covered in Round 2. We analyzed responses to Round 2 by calculating the median harmfulness score for each content element at each of the three levels of harm. The median scores for severity of harm of each content element at each level of harm were then categorized based on the definitions provided that corresponded to the 11-point scale. Scores of 8 to 10 were categorized as severe harm, 5 to 7 were categorized as moderate harm, and 4 and below were categorized as mild harm. As additional analyses, we calculated percentages of endorsement within the three harmfulness ranges and tabulated the number of times each element was mentioned in the top 3 most harmful news content elements. Differences in responses between expertise groups were examined using Kruskal-Wallis tests. Open-ended responses were analyzed thematically as previously described [62–65]. In this round, special attention was paid to organizing themes around explanations for harmfulness rating responses by level of harm and considerations related to graphic and explicit content. Round 3 development and analysis At the beginning of Round 3, panelists were presented with a table summarizing the median harmfulness scores tabulated from Round 2 along with each elements’ assigned severity rating (i.e. severe harm, moderate harm, mild harm). Panelists were given the opportunity to agree or disagree with the Delphi panelists’ median rating. Panelists who disagreed were asked to provide their reasoning for opposing the presented rating. Positive consensus was defined as ≥ 80% of panelists agreeing with the panel’s median rating while negative consensus was defined as ≥ 80% of panelists disagreeing. We calculated percent agreement with Round 2 median harmfulness ratings and assigned severity ratings for each potentially harmful news content element for each level of harm. We tested differences in rates of any disagreement and number of disagreements between expertise groups using Fisher’s exact tests and Kruskal-Wallis tests, respectively. Open-ended responses providing rationale for disagreement with the harmfulness ratings were analyzed thematically to determine whether panelists outside consensus felt the harmfulness ratings should be more or less severe and to identify perspectives outside consensus [62,63,65]. The full content of Rounds 1, 2, and 3 are available in the (S1–S3 Appendices). All analyses were conducted using Stata 18 (StataCorp LLC, 2024), and results of statistical tests for differences between expertise groups are reported below when p < .10. Results Panelists Of the 25 potential panelists invited, 21 individuals (84%) agreed to participate, and all 21 panelists completed all 3 rounds. The median age of the panelists was 48 years (IQR 33–61 years; range 29–75 years). Twelve panelists (57%) self-identified as Black, 2 (10%) as Latine, 6 (29%) as White, and 2 (10%) as Multiracial. Most of the panelists were women (n = 15, 71%). Six participants (29%) self-identified as lived experience experts, 12 were journalism practice experts (57%) and 6 (29%) were scholars. Of note, three panelists reported overlapping expertise: two panelists with lived experience and one scholar also self-identified as journalism practice experts. In the statistical tests for differences between expertise groups, these three panelists were classified based on their non-journalism expertise. Round 1 Table 2 presents the number and percentage of panelists who agreed (i.e., somewhat agreed, agreed, or strongly agreed) that the media content elements presented (Table 1) could cause harm to firearm-injured people, communities impacted by CFV, and society. Panelists had the highest proportion of agreement that the elements of graphic content (100%), number of gunshot wounds (95%), and mugshots, only law enforcement narratives, and not exploring solutions (all 90%) were likely to cause harm at the individual level. The same elements had the same highest proportions of agreement for harm at the community level, except for mugshots (76% for community level). The elements with the highest proportions of agreement for harm at the society level were graphic content (95%) and does not report solutions (95%). No potentially harmful news content element achieved negative consensus for any level, so all elements, at all levels, were advanced to the next round. There were no statistically significant differences in responses between expertise groups for any element at any level. 10.1371/journal.pone.0316026.t002Table 2Agreement regarding harmfulness of potentially harmful news content elements by level of harm.Potentially Harmful News Content ElementIndividual, n (%)aCommunity, n (%)bSociety, n (%)cGraphic content21 (100)21 (100)20 (95)Clinical condition15 (71)13 (62)10 (48)Number of gunshot wounds20 (95)20 (95)16 (76)Name of treating hospital16 (76)15 (71)9 (43)Relationship between firearm-injured person and perpetrator17 (81)15 (71)12 (57)Mugshot19 (90)16 (76)16 (76)Absence of a follow-up story15 (71)14 (67)16 (76)Episodic framing18 (86)18 (86)18 (86)Only law enforcement narrators19 (90)19 (90)18 (86)Missing perspective of firearm-injured person17 (81)17 (81)16 (76)Missing community perspective17 (81)17 (81)18 (86)Does not explore solutions19 (90)19 (90)20 (95)Numbers and percentages refer to panelists who somewhat agreed, agreed or strongly agreed that the news content element could cause potential harm.aIndividual defined as: Firearm-injured people and/or co-victims involved in the shooting being reported on.bCommunity defined as: Firearm-injured people and/or co-victims who have been affected by previous shootings. cSociety defined as: News audiences viewing, reading, and/or listening to the content and/or society at large. Agreement regarding harmfulness of potentially harmful news content elements by level of harm. Numbers and percentages refer to panelists who somewhat agreed, agreed or strongly agreed that the news content element could cause potential harm. aIndividual defined as: Firearm-injured people and/or co-victims involved in the shooting being reported on. bCommunity defined as: Firearm-injured people and/or co-victims who have been affected by previous shootings. cSociety defined as: News audiences viewing, reading, and/or listening to the content and/or society at large. In the open-ended responses, panelists discussed perceived mechanisms of harm by news reports and identified other potentially harmful elements. Several panelists indicated that graphic content could re-traumatize people impacted by CFV, while publicizing the health status or treating hospital of a firearm-injured person could result in threats to privacy and safety. Panelists also identified factual inaccuracies as a mechanism of harm, which are known to be more present in reports that favor the perspectives of law enforcement over firearm-injured people and their loved ones. Finally, panelists suggested that episodic framing in news stories without community perspectives can perpetuate racist stereotypes about firearm-injured people through stigmatization and dehumanization. Other potentially harmful elements identified by panelists in open-ended responses centered on concerns regarding journalistic practice; these included interviewing without consent, a lack of diversity among journalists and narrators, and logistical challenges related to deadlines and access to data. Several panelists noted that “insensitive” images that perpetuate stereotypes along with sensationalized headlines and narration are potentially harmful news content elements. Round 2 The median severity scores by content element and at each level of harm endorsed by the panelists are presented in Table 3, along with the corresponding rating (i.e., severe for 10–8, moderate for 7–5, or mild for 4 or lower). The highest number of median severity scores of severe harm ratings were endorsed at the individual level (i.e., for firearm-injured people included in the report). The news content elements that had median severity score ratings of severe harm across two or more levels were graphic content, episodic framing, reports that do not include solutions, and reports that include only or predominantly law enforcement perspectives. Tests for differences in severity scores by expertise groups did suggest some differences for some elements at some levels. Reported severity for clinical condition at the society level (p = .07), number of gunshot wounds at the society level (p = .03), name of treating hospital at the society level (p = .05), mugshot at the community level (p = .10), episodic framing at the individual level (p = .09), and does not explore solutions at the individual level (p = .02) all appeared to be rated differently by expertise groups. In each of these instances, the scholar expertise group rated the lowest average harm. 10.1371/journal.pone.0316026.t003Table 3Median harmfulness scores of harmful news content elements by level of harm and severity ratings.News Content ElementMedian Harmfulness Score (Rating)aIndividualbCommunitycSocietydGraphic content10 (Severe)9 (Severe)8 (Severe)Clinical condition7 (Moderate)5 (Moderate)4 (Mild)Number of gunshot wounds7 (Moderate)6 (Moderate)5 (Moderate)Name of treating hospital8 (Severe)5 (Moderate)3 (Mild)Relationship between firearm-injured person and perpetrator5 (Moderate)5 (Moderate)5 (Moderate)Mugshot7 (Moderate)6 (Moderate)6 (Moderate)Absence of a follow-up story8 (Severe)7 (Moderate)7 (Moderate)Episodic framing8 (Severe)8 (Severe)8 (Severe)Only law enforcement narrators8 (Severe)7 (Moderate)8 (Severe)Missing perspective of firearm-injured person8 (Severe)5 (Moderate)6 (Moderate)Missing community perspective8 (Severe)7 (Moderate)7 (Moderate)Does not explore solutions8 (Severe)8 (Severe)8 (Severe)aSeverity ratings are: Severe, 10–8; Moderate, 7–5; and Mild, 4 and below.bIndividual defined as: Firearm-injured people and/or co-victims involved in the shooting being reported on.cCommunity defined as: Firearm-injured people and/or co-victims who have been affected by previous shootings.dSociety defined as: News audiences viewing, reading, and/or listening to the content and/or society at large. Median harmfulness scores of harmful news content elements by level of harm and severity ratings. aSeverity ratings are: Severe, 10–8; Moderate, 7–5; and Mild, 4 and below. bIndividual defined as: Firearm-injured people and/or co-victims involved in the shooting being reported on. cCommunity defined as: Firearm-injured people and/or co-victims who have been affected by previous shootings. dSociety defined as: News audiences viewing, reading, and/or listening to the content and/or society at large. Fig 2 summarizes the number of times each element was included in panelists’ top three most harmful news elements for each level of harm. The elements most commonly included in panelists’ top three most harmful news elements were graphic content (14 times at individual level, 14 times at community level, 10 times at society level), episodic framing (12 times at society level), only law enforcement narrators (10 times at society level), missing perspective of firearm-injured person (11 times at individual level), and does not explore solutions (12 times at society level). Number of times element is mentioned in panelists’ top three most harmful elements by level. Relationship = Relationship between firearm-injured person and perpetrator. Individual defined as: Firearm-injured people and/or co-victims involved in the shooting being reported on. Community defined as: Firearm-injured people and/or co-victims who have been affected by previous shootings. Society defined as: News audiences viewing, reading, and/or listening to the content and/or society at large. Graphic content Seven of the 12 specific graphic content elements were considered graphic and/or explicit by at least half of the panelists. These elements were: the uncovered body of a deceased firearm-injured person(s) (n = 21, 100%); blood at the crime scene or on objects (n = 19, 90%); the covered body of a deceased firearm-injured person(s) (n = 18, 86%); the actual shooting incident as it unfolds (n = 18, 86%); someone being loaded into or out of an emergency transport vehicle (e.g. ambulance, police car) (n = 17, 81%), family or friends crying about a firearm-injured person (n = 13, 62%), an injured survivor (n = 11, 52%). Panelists rated all formats of graphic content (i.e. disturbing video, still images, audio, and detailed descriptions of firearm violence) as severely harmful (median scores of 8 and 9) across all levels of harm (i.e. individual, community, society) except for “disturbing detailed verbal description of firearm violence” at the society level (median score = 7, moderate harm). A figure depicting the perspectives of the panelists on which specific news elements constitute graphic and/or explicit content as well as a table detailing the median harmfulness score and corresponding severity rating for each of the four formats of graphic content at each level of harm are provided in the (S1 Fig and S1 Table). In the open-ended responses, panelists provided rationale for their assigned severity scores by level of harm. Multiple panelists indicated that not providing context or solutions could perpetuate negative stereotypes about firearm-injured people, emphasizing the importance of including these elements along with the need to center the perspectives of firearm-injured people, their loves ones, and community members in news reports on firearm violence to minimize bias. In considering harms at the societal level, some panelists noted that harmful content elements can perpetuate fear and reinforce violence, dehumanize firearm-injured people, and lead to “narrow-mindedness.” Several panelists emphasized that reliance on law enforcement narrators encourages audiences to understand firearm violence as a “criminal legal” issue rather than a public health issue. In terms of specific open-ended responses related to graphic content, panelists added that the following elements could also be considered graphic content: bullet holes in cars, audio of sirens or gunshots, stories about children being harmed, and “body counts” and/or statistics. Several important tensions were highlighted by panelists in the open-ended responses. Some panelists noted that some information about firearm-injured people may be important to share in news reports to raise awareness about the issue and support the police in finding the perpetrator. They indicated that these considerations need to be balanced with maintaining the safety and privacy of the firearm-injured person and ensuring that the community’s perspectives is centered in news reports about firearm violence. While the panelists uniformly agreed that graphic and explicit content were potentially harmful, a few panelists noted that some graphic content could also move people to action or prevent future shootings. Panelists noted that graphic and explicit content are more likely to cause harm to firearm-injured people, and that the recurring nature of graphic content in news reports on firearm violence likely results in desensitization of the public to firearm violence. One panelist suggested that a content warning ahead of graphic and/or explicit content might help reduce harm. Round 3 Table 4 summarizes the degree to which panelists agreed with the severity ratings resulting from the median harmfulness scores from Round 2. There was high agreement on the severity ratings, with all reaching positive consensus (i.e. ≥ 80% agreement). The lowest agreement was the rating of moderate harm for the element relationship between firearm-injured person and perpetrator at the individual level (81%). 10.1371/journal.pone.0316026.t004Table 4Percent agreement on Round 2 severity ratings for news content elements by level of harm.ElementPercent Agreement (%, n = 21)(Severity rating)aIndividualbCommunitycSocietydGraphic content100 (Severe)100 (Severe)95 (Severe)Clinical condition86 (Moderate)100 (Moderate)100 (Mild)Number of gunshot wounds90 (Moderate)86 (Moderate)90 (Moderate)Name of treating hospital90 (Severe)90 (Moderate)86 (Mild)Relationship between firearm-injured person and perpetrator81 (Moderate)100 (Moderate)95 (Moderate)Mugshot86 (Moderate)100 (Moderate)95 (Moderate)Absence of a follow-up story90 (Severe)95 (Moderate)90 (Moderate)Episodic framing100 (Severe)100 (Severe)100 (Severe)Only law enforcement narrators100 (Severe)90 (Moderate)95 (Severe)Missing perspective of firearm-injured person95 (Severe)95 (Moderate)95 (Moderate)Missing community perspective95 (Severe)100 (Moderate)95 (Moderate)Does not explore solutions86 (Severe)90 (Severe)90 (Severe)aSeverity ratings reflect median harmfulness scores from Round 2 and are: Severe, 10–8; Moderate, 7–5; and Mild, 4 and below.bIndividual level defined as: Firearm-injured people and/or co-victims involved in the shooting being reported on.cCommunity-level defined as: Firearm-injured people and/or co-victims who have been affected by previous shootings.dSociety-level defined as: News audiences viewing, reading, and/or listening to the content and/or society at large. Percent agreement on Round 2 severity ratings for news content elements by level of harm. aSeverity ratings reflect median harmfulness scores from Round 2 and are: Severe, 10–8; Moderate, 7–5; and Mild, 4 and below. bIndividual level defined as: Firearm-injured people and/or co-victims involved in the shooting being reported on. cCommunity-level defined as: Firearm-injured people and/or co-victims who have been affected by previous shootings. dSociety-level defined as: News audiences viewing, reading, and/or listening to the content and/or society at large. Only 48 out of the 756 responses (6%) indicated disagreement with the panelist severity rating. Of the 21 panelists, 12 (57%) disagreed with a panelist severity rating at least once, and there was no difference between expertise groups in likelihood of disagreeing at least once (p = .40). The average number of disagreements was 2.3 (SD = 3.0, range: 0–10; median: 1), and there was no difference between expertise groups in number of disagreements (p = .39). Thematic analysis found that 22 of the 48 disagreement open-ended responses reflected that the panelist felt the severity rating should be higher, while 26 reflected that the panelist felt the severity rating should be lower. In the disagreement open-ended responses, panelists discussed how the severity of harm from some news content elements (i.e., relationship between firearm-injured person and perpetrator, mugshot, and missing perspective of firearm-injured person) could be more or less severe depending on the circumstances of the shooting. Panelists also shared how they saw the harm of other news content elements (i.e., absence of a follow-up story and does not explore solutions) as depending on the nature of the reporting itself. For example, one panelist noted how covering solutions is only beneficial if the solutions covered are based in evidence or otherwise known to benefit firearm-injured people, communities affected by CFV, or audiences/society at large. Discussion In this modified Delphi study, 21 panelists representing a diverse set of stakeholder perspectives and expertise, including people with lived experience of CFV, journalists, and scholars determined through consensus that 12 specific news content elements are harmful to firearm-injured people, communities impacted by CFV, and society. Panelists agreed on a severity rating (mild, moderate, severe harm) for each news content element across three levels of harm (individual, community, society) and determined that news stories on CFV that include graphic content, episodic framing, and do not explore solutions have the potential to cause severe harm at all three levels of harm. In addition, panelists concluded that stories that only or predominantly include the perspectives of law enforcement have the potential to cause severe harm at the individual and society level and moderate harm at the community level, while news reports that neglect to include the perspectives of firearm-injured people and impacted community members were considered severely harmful at the individual level and moderately harmful at the community and society levels. Panelists found that harmful news content elements were most detrimental to firearm-injured people, with 8 of 12 elements rated as severe harm at that level. This is an especially important finding as traditional media effects theory and research largely considers the impact of news reports on news audiences more generally. These results highlight the need for journalists to consider the impacts of news reports on people directly impacted by CFV as a unique population requiring a trauma-informed approach to reporting [66]. Trauma-informed journalism employs methods that aim to recognize and respond effectively to the experiences of trauma survivors who are story sources or potential audience members with the goal to minimize further harm that the reporting may cause in the aftermath of a traumatic event [66–68]. Education and training in trauma-informed reporting practices for journalists is an important next step as journalists work to center the perspectives of survivors in news stories and minimize the news elements and practices known to cause harm to firearm-injured people [69]. This study has important implications for the study of news reporting on CFV. The level of consensus achieved indicates strong agreement regarding the potential for harm of the 12 specific news content elements among a diverse set of panelists. Panelists in this study agreed that reporting on CFV, which remains largely with episodic crime framing, is likely causing multilevel harms to firearm-injured people, their loved ones, communities impacted by CFV, and our society more generally [28]. This expert consensus provides strong rationale for the work of our community-based partners, PCGVR, in media narrative change and for this line of research examining harmful reporting as a health disparity that is contributing to the perpetuation of racial stereotypes and structural racism [57,58,69]. However, it is important to note that this study did not measure the direct impact of harmful reporting on individuals, impacted communities, or society, which remain important directions in future research. As a next step, we plan to use the results of this modified Delphi consensus to develop a novel instrument to measure harmful reporting along with a scale to document the severity of harm in news reports about CFV. We will then utilize this scale to examine whether there are racial and spatial disparities in harmful reporting on CFV. The ultimate goal of this line of research is to inform the measurement of direct effects of harmful reporting along with development and testing of interventions to minimize harmful reporting. While the examination of the Delphi panelists’ views on graphic and explicit content was not a stated goal of this study, our results do add a unique perspective to the rich and ongoing discussions surrounding the impact of news media portrayals of conflict and violence. Photojournalists have traditionally argued that war photographs depicting human suffering can evoke compassion and spur action in audiences; however, research on the societal impact of violent images indicates that audience responses are highly complex [70–73]. While images that depict the human-cost-of war do have the potential to evoke compassion, the ubiquitous presence of US police killings of Black people in recent years in news and social media has caused societal-level trauma and negative psychological effects on Black Americans [72,73]. Compassion fatigue, feelings of helplessness, and the desire for media to cover resolution efforts are also potential responses to conflict images [71,74]. The results of our study uncovered similar tensions between the idea that graphic and explicit content can raise awareness of CFV as a social problem and the concern that such disturbing content can compound the trauma of firearm-injured people and co-victims. Despite this, panelists were conclusive that graphic and/or explicit content is potentially harmful across all levels of harm; 95% of the panelists endorsed that graphic and/or explicit content could cause severe harm to news audiences and/or society and they uniformly agreed that graphic and/or explicit content could cause severe harm to firearm-injured people and impacted communities. Research indicates that graphic coverage is not uncommon in news reports about CFV. A quantitative content analysis of television news in Philadelphia found that 7% of news stories on firearm violence in 2021 contained video or audio of a shooting event, which 86% of Delphi panelists considered harmful graphic content [28]. The results of this Delphi consensus can further urge journalists to refrain from including audio, videos, photographs, or detailed verbal descriptions of a shooting event in news stories on CFV to prevent harms to firearm-injured people, impacted communities, and news audiences. Or, when that is not possible, journalists may be able to mitigate harm by including a content warning ahead of graphic and/or explicit content in news reports on CFV. This study has several strengths. All 21 panelists of this modified Delphi consensus study participated in all three rounds, indicating significant engagement and motivation among panelists in this study. While the study did achieve consensus on the severity ratings for all 12 news content elements, more than half of the panelists disagreed with at least one of the ratings. This suggests that while the consensus process was successful at synthesizing the viewpoints of the Delphi panelists, they felt free to disagree where they felt appropriate. One innovation of this study surrounds the definition of “expertise.” Informed by the work of our community partners at PCGVR, we considered expertise on reporting on CFV to include not just academic scholars but also individuals with lived experience of CFV along with journalism practice experience. This inclusive perspective on “expertise,” along with our engagement with community partners supports the validity of the Delphi panel findings. In this way, this research could serve as a model for other community-engaged efforts to achieve consensus on health or public health questions going forward. The study findings should be interpreted considering its limitations. While we followed Delphi study guidelines, there are some methodological adaptations to note [59–61]. For example, it is possible that we missed news content elements that could be potentially harmful by commencing the study with a structured questionnaire. Additionally, while we were able to draw on panelists from across the US with academic and journalism practice expertise, our lived experience experts were identified in collaboration with our community partners at PCGVR and were thus mostly local to Philadelphia. While the objective of a Delphi panel is not necessarily generalizability, more lived experience experts from other areas of the country would have likely added depth to the panelist responses and our analyses. Also, the inclusion of more women than men in the study is a potential limitation. It is important to note the main goal of a Delphi panel is to create expert consensus where none currently exists. In this study, the aim was to achieve expert consensus on what constitutes harmful reporting on community firearm violence, which inherently relies on the perceptions and perspectives of the panelists. As such, we did not seek to measure the direct multilevel impacts of potentially harmful reporting on health and psychological outcomes, which will be an important area of research going forward. Importantly, while all elements included were indicated as potentially harmful in our prior qualitative research with people with lived experience of CFV, it is possible that further investigations of the impacts of these elements may not find quantitative evidence of harms at some levels or may find quantitative evidence of harm at some levels but benefit at other levels.[43] Finally, this study did not evaluate the news content elements that should be present to support less harmful reporting, as there is far less existing research in this area. Thus, our results should not be interpreted to suggest that inclusion of firearm violence solutions is by definition “helpful,” as news reporting about ineffective solutions without a critical lens could cause harm. In conclusion, this study provides important and novel information about the elements that may constitute harmful news reporting on CFV during a significant national surge in CFV incidence in the US [1,2,4,6]. A Delphi panel with diverse expertise including lived experience of CFV, journalism practice, and academic scholarship reached consensus on 12 potentially harmful news content elements, rating the severity of harm attributed to these news content elements across 3 levels of harm (individual, community, and society). This study provides an important starting point from which to build scholarship around harmful reporting on firearm violence, work which will surely be iterative and evolve over time as the evidence base grows. The findings of this and subsequent studies will inform future investigations measuring the frequency and severity of harmful reporting on CFV in news reports, the direct impacts of potentially harmful news content elements across levels—and how these impacts may differ across levels—and in turn, inform interventions and best practices for journalists to minimize harmful reporting on CFV. Supporting information Round 1 content. (PDF) Round 2 content. (PDF) Round 3 content. (PDF) Endorsement of specific graphic and/or explicit content elements as harmful. (TIF) Median harmfulness scores of formats for graphic and/or explicit by level of harm and severity ratings. (DOCX) 17 Jul 2024 PONE-D-24-20002Defining Harmful News Reporting on Community Firearm Violence: A Modified Delphi Consensus StudyPLOS ONE Dear Dr. Beard, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Aug 30 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Claudio Terranova Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ" 3. Thank you for stating the following financial disclosure:"This research was funded by the Stoneleigh Foundation (https://stoneleighfoundation.org/) (JHB) and the National Institute on Minority Health and Health Disparities of the National Institutes of Health (JHB, CNM, JM) [grant number R21MD019088, https://www.nimhd.nih.gov/]. ELE is supported by the National Institute on Drug Abuse of the NIH [grant number T32DA031099, https://nida.nih.gov/]. The content is solely the responsibility of the authors and dose not necessarily represent the official views of the Stoneleigh Foundation or the NIH." Please state what role the funders took in the study. If the funders had no role, please state: ""The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."" If this statement is not correct you must amend it as needed. Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf. 4. Thank you for stating the following in the Acknowledgments Section of your manuscript: "This research was funded by the Stoneleigh Foundation (JHB) and the National Institute on Minority Health and Health Disparities of the National Institutes of Health (JHB, CNM,JM) [grant number R21MD019088]. ELE is supported by the National Institute on Drug Abuse of the NIH [grant number T32DA031099]. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Stoneleigh Foundation or the NIH." We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: "This research was funded by the Stoneleigh Foundation (https://stoneleighfoundation.org/) (JHB) and the National Institute on Minority Health and Health Disparities of the National Institutes of Health (JHB, CNM, JM) [grant number R21MD019088, https://www.nimhd.nih.gov/]. ELE is supported by the National Institute on Drug Abuse of the NIH [grant number T32DA031099, https://nida.nih.gov/]. The content is solely the responsibility of the authors and dose not necessarily represent the official views of the Stoneleigh Foundation or the NIH." Please include your amended statements within your cover letter; we will change the online submission form on your behalf. 5. In the online submission form, you indicated that data cannot be shared publicly to protect participant anonymity. Data are available upon request from the corresponding author for researchers who meet the criteria for access to confidential data.. All PLOS journals now require all data underlying the findings described in their manuscript to be freely available to other researchers, either 1. In a public repository, 2. Within the manuscript itself, or 3. Uploaded as supplementary information. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If your data cannot be made publicly available for ethical or legal reasons (e.g., public availability would compromise patient privacy), please explain your reasons on resubmission and your exemption request will be escalated for approval. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: No Reviewer #2: Yes ********** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ********** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ********** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ********** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: This paper raises some very important points about the role that media reporting can play in furthering the harms that are caused by firearm violence in communities. It is especially good to see attention being given to the disproportionate role that police sources play in shaping how news is framed. However, there is room for some improvement. The paper currently lacks nuance, and makes many statements that draw artificial binaries – for example, that gun violence is framed as a crime issue when it should be seen as a public health issue. In reality, it is both of these, and far more – issues such as historical racism and its contemporary consequences, severe poverty, and socioeconomic marginalization go far beyond crime or public health. I would like to see a much more sophisticated theoretical and empirical perspective that takes this complexity into account. Given that the entire purpose of the paper is to define ‘harmful’ reporting, it would be helpful to see more attention given to discussing current conceptualizations of, and evidence for, harm (or the lack of conceptualizations/evidence) and exploring how/why they differ or are unsatisfactory for research purposes. Further, there is a need to better explain whether and how the different ‘levels’ of harm were conceptualized/what guidelines were used to attempt to ensure some consistency between different individuals providing ratings – for instance, how can we be sure that different people in the different groups were using the same ‘yardstick’ to label something as an extreme level of harm? The results may only be saying that some people/groups see things as more harmful than others – rather than telling us anything about actual harm caused. This is particularly problematic in the context of lived experience participants, whose assessments could be influenced by a wide range of factors including trauma. This is touched on in the discussion, but only in the context of trauma-informed reporting (which means what, exactly?). The paper needs to be very careful about differentiating between subjective perceptions of harm, and whether those perceptions translate to harm in more ‘objective’ terms (for example, a person may believe that media coverage causes “narrow mindedness” – but is there anything to back this belief up?). I am a little concerned about the overall objectivity and relative lack of serious critical analysis presented in the work. For example, something that is not really acknowledged is that, at times, an element that may be deemed ‘harmful’ may nevertheless be an important part of a balanced, objective, and accurate news story. Similarly, some aspects that are implicitly considered to reduce harm – such as ‘providing solutions’ – may cause harm in their own right (e.g., ineffective ‘solutions’ may be promoted). Again, this speaks to the need for greater nuance and depth in how this paper is written. I am concerned, in relation to the lived experience participants in particular, that the small sample size is unlikely to capture the breadth of different experiences people may have with media reporting. The over-representation of female participants is also a limitation. Overall, the discussion takes an uncritical approach to the findings and accepts, seemingly without question, that the findings are valid, robust, and should be acted on. While clearly well intentioned, this comes across more as activism than scholarship. The paper also needs to be far more open about the limitations of the work. Reviewer #2: Insightful and much needed article on harmful news content and gun violence reporting. I believe this manuscript will make significant contributions to our knowledge on the ways media content specifically gun violence is framed and in what ways we can collectively address harmful content. Overall the paper is well written. However, I do have several minor revisions. Author(s) should explain in more detail the Delphi method and why did the authors choose this approach. In regard to the recruitment of the sample the author(s) did not address: How were participants invited? If I am reading this correctly, a total of six participants with lived experience participated in study. What were their demographics? I am concerned with merging the sub-groups to arrive at a median age because most victims of violence are younger than 48. What is the average age of a gun violence survivor in Philadelphia this should be reflective in the age of the lived experience group. Furthermore, this sub-group should have been disproportionately over-represented than the other sub-groups in the study. The sample should be more reflective of people injured by firearm Philadelphia. If one of the aims is to center the voices of those most impacted by gun violence (which was evident by one of the categories), then the community voices should be over-represented in sample instead there was far more representation of journalists and scholars. This is a limitation of the study that should be acknowledged. Author(s) should also define: What does lived experience mean? Did participants witness violence, were they victims and/or perpetrators of violence? ********** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. 15 Aug 2024 Please see file "Response to Reviewers" Submitted filename: Response to Reviewers.docx 9 Sep 2024 PONE-D-24-20002R1 Defining Harmful News Reporting on Community Firearm Violence: A Modified Delphi Consensus Study PLOS ONE Dear Dr. Beard, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not yet fully meet PLOS ONE’s publication criteria as it currently stands. One of the reviewers, as you can read below, believes that not all the comments highlighted in the first review have been addressed. Therefore, we invite you to submit a revised version of the manuscript that addresses all the points indicated by the Reviewer. Please submit your revised manuscript by Oct 24 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Claudio Terranova Academic Editor PLOS ONE Comments to the Author Reviewer #1: (No Response) Reviewer #2: All comments have been addressed ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Partly Reviewer #2: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? Reviewer #1: Yes Reviewer #2: Yes ********** 6. Review Comments to the Author Reviewer #1: While this is an improved paper, there remain a few points that have not been adequately addressed. Original comment: The over-representation of female participants is also a limitation. Authors response: This was not included as a limitation because the gender of experts is not relevant to the validity of their expertise. While this may be (somewhat) true of scholars and journalists, there is a growing body of research indicating that gender differences apply in how persons with lived experience respond to harm/emotion rating tasks, etc (which is unsurprising, given that these types of gender differences are found in general populations across a very wide range of socio-political contexts). Being a person who has experienced gun violence or a family member of a victim does not alter that. This in turn may have bearing on the results. This is a simple, reasonable, and evidence-based point to acknowledge. Revised sentence: In this study, the aim was to achieve expert consensus on what constitutes harmful reporting on firearm violence, which inherently relies on the perceptions and perspectives of the panelists. As such, we did not seek to measure the direct impact of harmful reporting on health or psychological outcomes, which will be an important area of research going forward.” This partially addresses my initial comment about the lack of objective information and potential mismatch between subjective and objective outcomes. However, this needs to be extended beyond health and psychological outcomes. Reporting may have psychological impacts for an individual. However, it may not have any negative broader community or societal impacts. Conversely, leaving an item out in order to avoid impacts on an individual may have negative community or social impacts. This requires more explicit acknowledgement (rather than just being pushed aside as a future direction). This also relates to the next point. Initial comment: I am a little concerned about the overall objectivity and relative lack of serious critical analysis presented in the work. For example, something that is not really acknowledged is that, at times, an element that may be deemed ‘harmful’ may nevertheless be an important part of a balanced, objective, and accurate news story. Authors response: This study was not meant to develop a list of things that should not be included, but instead to further advance research on this topic to inform guidelines and best practices to minimize any harms of reporting, which will inherently be iterative and evolve over time. We have now included statements to this effect, stating: “This study provides an important starting point from which to build scholarship around harmful reporting on firearm violence, work which will surely be iterative and evolve over time. The findings of this study will inform future investigations measuring the frequency and severity of harmful reporting on CFV in news reports, the direct impacts of harmful reporting on individuals and communities, along with interventions with journalists to minimize harmful reporting on CFV” (Page 23). While this goes part of the way to addressing my concern, it still does not capture the key issue – specifically, that something that may be perceived by some as “harmful”, may nevertheless be helpful to others. The revised paper is still not acknowledging this issue. It is a very straightforward point to include, and not too much to ask. Reviewer #2: The author(s) have addressed the necessary revisions, this paper is much needed in the gun violence reporting space. ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. 4 Oct 2024 See Response to Reviewers Document. 4 Dec 2024 Defining Harmful News Reporting on Community Firearm Violence: A Modified Delphi Consensus Study PONE-D-24-20002R2 Dear Dr. Beard, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Claudio Terranova Academic Editor PLOS ONE Additional Editor Comments Regarding the issue raised in your cover letter dated October 4, concerning a comment by Reviewer #1, I have carefully reviewed both the revisions and the authors' responses. I believe there is a misunderstanding between what the reviewer intended in the request and what the authors appear to interpret. As far as I can tell, the reviewer was simply asking the authors to acknowledge that there is a gender imbalance in their sample, and that this is related to a previous point (in the original review) regarding sample size, i.e., that this may not "capture the breadth of different experiences people may have." The authors' response is that "the gender of experts is not relevant to the validity of their expertise." However, I do not perceive the reviewers' concern to be that women lack expertise, but rather that their perspective may be over-represented. In conclusion, I believe that the reviewer has not suggested that women lack expertise, and that the request to add a sentence noting that there were unequal numbers of men and women does not constitute sexism. Therefore, I think the sentence you have introduced to address the reviewer’s comment can be maintained. Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: (No Response) ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No ********** 8 Dec 2024 PONE-D-24-20002R2 PLOS ONE Dear Dr. Beard, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Professor Claudio Terranova Academic Editor PLOS ONE