Please login to view this application
LoginDenver Department of Public Health & Environment/Denver Public Health
Fentanyl Early Warning System
Jean Finn
Program manager
jean.finn@denvergov.org
720.865.5503
Denver
Colorado
80202
Your summary must address all the questions below:
The City and County of Denver is a city of 716,500, a leading population and business center in the Rocky Mountain region and capitol of Colorado. It is served by the Denver Department of Public Health and Environment (DDPHE) with clinical, epidemiologic and informatics support from Denver Public Health (DPH), a department of Denver Health, an integrated public safety net healthcare authority that also includes a hospital, the paramedic system, behavioral health, and primary care.
In 2017, across the United States, 28,000 deaths involved synthetic opioids (other than methadone), more than from any other type of opioid, with much of the increase driven by illicitly manufactured fentanyl flowing into the illicit drug supply. However, fentanyl adulteration and associated overdose has remained relatively unusual in Denver (18 fentanyl-involved deaths in 2017, 17 deaths in 2018) but a sudden change in the drug supply could rapidly increase overdose mortality as it has done in cities across the US (e.g., an 81% increase in NYC 2014-17).
The objective of our project was to measure the incidence of fentanyl contamination and use in Denver and develop a communication plan if a rise in fentanyl were detected, with the ultimate goal of preventing sudden increases in opioid overdose mortality associated with unsuspected fentanyl adulteration of the illicit drug supply. This project was supported by Colorado Department of Public Health and Environment using the Centers for Disease Control and Prevention Opioid Emergency Response Cooperative Agreement. The project’s original budget period was September 1, 2018 to August 31, 2019, but was extended through November 30, 2019.
We began the project by analyzing multiple information streams to determine their feasibility as signals of increased fentanyl contamination or use. Thirteen diverse data streams were analyzed, of which five were selected using criteria such as frequency, simplicity, data quality, representativeness, and cost. Three represented novel data partnerships. All were integrated into a weekly dashboard designed to make trends and aberrations easily visible.
We convened focus groups and key contact interviews with people who use substances and other stakeholders to inform the communication plan if the system were to detect a sudden rise in fentanyl. We also reviewed products created elsewhere to inform the design, content and methods for rapid dissemination of alerts. A community-based notification strategy (via informal leaders, email, text and website for many organizations serving people who use substances), a mass-media notification strategy, and a health professional notification strategy (using preformatted messages for an established Health Alert Network) were implemented.
All objectives were successfully met between January and October 2019, through activities that included a comparative analysis of potential data sources; negotiation of all data use and quality issues for selected sources; iterative development and improvement of a dashboard; end-user acceptance testing and meetings with stakeholders; sharing of dashboard information with public health authorities; focus groups and key informant interviews; reviews of communications used in other communities; development of print and electronic messages to be disseminated; discussions with communication stakeholders; “fill in the blank” templates designed for Health Alert Network messaging to health care providers; and an evaluation report to the funders.
Shortly after project completion, Denver Police seized a large “brick” of black tar heroin, that was in fact comprised of fentanyl. This finding was made and reported outside the dashboard, but project relationships facilitated rapid inter-agency communication and decision-making. Within two business days, alerts were distributed using all three strategies. A community meeting, to follow-up with focus group participants and provide community overdose prevention education, had been scheduled for the evening of November 5, 2019. Staff received notice about the seizure of the counterfeit black tar heroin/fentanyl in the morning and received approval to issue an alert that day. At the community meeting, staff provided fentanyl harm reduction messages with news of the seizure of the counterfeit black tar heroin/fentanyl. In the following week, three of five dashboard metrics indicated a temporary spike.
Success factors for the early warning dashboard included: identification of candidate data sources, followed by orderly selection of indicators using a well-established evaluation approach for surveillance systems; leveraging existing data while rapidly and securely creating new data relationships; utilizing business intelligence software to create visually informative displays of multiple indicators, with easy data updating; and allowing for iterative dashboard reviews with both data contributors and users.
Success factors for early alerting included: implementing predictable weekly arrival of data and analysis; having people who use substances participate in message design and plan the dissemination; developing pre-formatted messages for rapid completion and distribution; and leveraging existing networks, including informal community leaders, email/text distribution lists and the Health Alert Network.
The DDPHE fentanyl site, including a link to the community alert, is available at https://www.denvergov.org/content/denvergov/en/environmental-health/community-health/office-of-behavioral-health-strategies/substance-misuse-overdose-prevention/fentanyl.html
A Model Practice must be responsive to a particular local public health problem or concern. An innovative practice must be -
Illicitly manufactured fentanyl and fentanyl analogs are largely responsible for a five-fold increase in US synthetic opioid overdose fatalities between 2013 and 2016. Sharp mortality spikes have been evident on both US coasts and in the Southeast and Midwest. Overdoses mortality involving fentanyl/analogs in Colorado remains relatively low, with 112 deaths statewide in 2017 of which 18 occurred in Denver. While people who use substances in many other communities have somewhat adjusted to widespread fentanyl/analog contamination over time, people who use substances in Denver are at high risk for unexpected changes in potency and lethality of illicit drugs. The overdose epidemic led to the CDC Opioid Emergency Response funding vehicle, which permitted the development of an early warning system before it was too late.
To our knowledge this is the first pre-event “early warning system” created specifically to detect and then warn affected populations and those in their social networks about the threat of fentanyl contamination in the illicit drug supply. With an estimated 4.82% of adult Coloradans having used an illicit drug other than marijuana in the past month, a sudden and unsuspected change in drug potency and lethality could be disastrous.
Upon reflection, no single used data stream could be expected to reliably and precisely reflect sudden changes in the makeup of the illicit drug supply, and none specifically targeted fentanyl/analogs. However, teammates had success with multi-indicator dashboards in the past and access to and experience with many possible sources of data. Therefore, we could integrate a variety of data, some of which might be leading but non-specific indicators, while others might be trailing indicators of great specificity. Since our vision was to alert the community at risk at the earliest opportunity, data would need to be available and interpretable in near real time. To avoid falling victim to our biases, we brainstormed a longer list of possible data indicators, and then narrowed them down, using CDC evaluation criteria proposed for surveillance systems. Based on these and other criteria we narrowed down 13 indicator candidates to five. While the data sources available to other communities might differ, any community can benefit by following a similar brainstorming and winnowing process informed by the same logic. Evaluations of data sources included describing each data source in terms of how it was collected, the format of the data, frequency in which it was received, whether it was a leading or lagging indicator, usefulness, simplicity, flexibility, data quality, acceptability (including cost), sensitivity, positive predictive value, representativeness, and timeliness.
Two selected data sources are common and well-established: emergency department syndromic surveillance for opioid overdoses (using a standard CDC query) and medical examiner overdose cases. Three more novel data sources were also included: drug seizures by local police, EMS cases, and urine drug screens from patients in addiction treatment. Local police performed gas chromatography/mass spectroscopy (GC/MS) on seized drugs for cases going to trial, but the addition of modest funding allowed them to test a random selection of drugs for fentanyl/analog contamination. We looked at EMS cases that responded only after unusually high doses of naloxone were administered, as an indirect indicator of overdose involving a potent opiate. Finally, a laboratory company (Precision Diagnostics, San Diego, CA) that routinely uses GC/MS on all urine drug tests enabled us to look at cumulative, de-identified results from patients in drug rehabilitation. Since we had little reason to believe data from other municipalities in the metropolitan area should systematically deviate from Denver County we utilized regional data when available. Data from broader surrounding regions and states made available by the laboratory company allowed us to look at emerging trends that might be approaching from further north, south, east and west.
Historic data was used, when available, to set control limits to help detect unusual trends, accented by color changes in the display. Retrospective analysis of consistency (Cronbach’s alpha) reassured that they measured related trends. Indicator metrics for each week were stacked vertically so patterns could be easily observed. Careful attention to “at a glance” visibility into the data is another principle other LHDs could utilize.
We reviewed data and dashboards several times, both with those who provided and best understood their data and other experts (including people who use drugs and those who care for them). This helped avoid misinterpretation and facilitated improved data display, while also enhancing easy communications among our various partners. Such iterative review by diverse stakeholders is another step other LHDs could use to their benefit.
Weekly periodicity of indicators was selected to increase the stability of estimates while allowing prompt detection. This balance has worked well with the data sets we selected.
Dashboards have been created weekly for 32 weeks (since May 5, 2019). Adding additional weekly data requires only a few keystrokes, so it has been possible to maintain operations in the absence of ongoing funding for several weeks (an alternate funding source will support forensic tests on a random sample of drug seizures). The Colorado Department of Public Health and Environment has agreed to fund maintenance and extension of the dashboard to other issues like synthetic marijuana in coming months, using CDC Overdose Data 2 Action cooperative agreement funds.
In parallel, DDPHE conducted focus groups and key informant interviews to help define message content and branding and identify how messages should effectively be delivered. One important conclusion of this assessment was to message around safe use, and not a call to abstinence that might cause users to disengage. It was discovered that the use of government logos would not increase distrust, but that official websites might be avoided by some segments of the population who use drugs.
Distribution of information to over 100 community-based entities will help ensure the message can be delivered by partners trusted by different segments of the target population. Social media also enables echoing of messages by partners who might appeal more to those who do not trust government. In short, many lessons were learned from direct and respectful communication directly with affected stakeholders, which also helped our team leverage existing relationships for communication rather than recreate them. The November 5th alert generated 313 mentions on radio, television, newspapers, Facebook, Twitter, and You Tube with a reach of 31,610,286 people and an earned medial value of $292,395. Past Health Alert Network alerts to health care providers have reached over 650 addressees, with a 99.8% delivery rate and 38% open rate.
As mentioned, the early warning system was tested by a real event in November. A single large seized specimen of counterfeit heroin was found to contain fentanyl, and Denver Police Department immediately notified health authorities. This notification preceded signal of a spike in the weekly dashboard, but partners agreed that community, media, and health professional notification should occur anyway due to the size and unusual nature of the seizure. Following the alert, record, or near-record, elevations in fentanyl positive urines, EMS rescues requiring high naloxone doses, and emergency department encounters for opioid overdose were noted for two weeks, then returning toward normal. This provided reassurance that the system was sensitive to changes in the drug supply, and that either the period of risk was short term or that alerting of the affected populations had been effective.
The LHD should have a role in the practice’s development and/or implementation. Additionally, the practice should demonstrate broadbased involvement and participation of community partners (e.g., government, local residents, business, healthcare, and academia). If the practice is internal to the LHD, it should demonstrate cooperation and participation within the agency (i.e., other LHD staff) and other outside entities, if relevant. An effective implementation strategy includes outlined, actionable steps that are taken to complete the goals and objectives and put the practice into action within the community.
The project was jointly designed by DDPHE’s Community and Behavioral Health Division and DPH’s Division of Epidemiology, Preparedness, Informatics and Data Science. DDPHE was the prime contractor for project funding from the Colorado Department of Public Health and Environment (CDPHE), and subcontracted dashboard and health professional alerting activities to DPH while completing the development of a communication plan and associated alerting activities.
The Community and Behavioral Health Division sponsors a Collective Impact group to address opioids and other substances with a leadership committee representing 27 diverse area agencies and including representatives with lived experience of substance use and recovery, and representatives of surrounding communities. The Collective Impact and its subcommittees weighed in on project design at several points and continue to do so, with special attention to the group’s values of reducing stigma, eliminating barriers, ensuring equity, creating opportunity and building resiliency. The fentanyl project addressed the “Reduce Harm” goal, “Reduce Rate of Overdose Deaths” strategy and “Monitor presence of fentanyl in Denver” as an activity in the County’s Opioid Response Strategic Plan 2018-2023.
The project was funded on a deliverable basis by CDPHE. Deliverables included assessing the utility and feasibility and increasing the quality of several data streams; establishing a minimum of two data sources (five were established) for dashboard inclusion; developing a fentanyl dashboard; resolving issues related to ownership and user-ship of data and the dashboard; implementing the dashboard and setting notification thresholds; developing alert messaging and response for a variety of populations; and creating and executing an evaluation plan to assess the effect of the program. The contract period was from September 2018 through November 2019, but both teams commenced work several months in advance given the time-importance and possible life-saving nature of the project. All deliverables were fully executed to the satisfaction of the funder.
As noted, multiple data sources were assessed prior to final selection of those used. Valuable consultations were held with the Rocky Mountain Poison and Drug Safety, the Harm Reduction Action Center, Denver Health emergency department, paramedics and laboratory staff, Denver Police Department, Office of the Medical Examiner, the Colorado Health Observations Regional Data Service (CHORDS) and various units of DDPHE. These dialogs helped build relationships that may spawn future information-to-action partnerships.
Creation of the dashboard strengthened relationships between our two agency divisions, and with the Denver Police Department, the Office of the Medical Examiner, the Denver Health Paramedic Division and Precision Diagnostics. The North Central Region syndromic surveillance collaborative of health departments sharing near real time emergency room data over six (and now an increasing number) Colorado counties was also leveraged to provide data on overdose visits for the dashboard.
A user acceptance conference included all data contributors and direct dashboard users, and one follow-up meeting was held to explore findings of uncertain significance. The project was also presented to and reviewed by Colorado’s Consortium for Prescription Drug Abuse Prevention, a leading statewide forum on the opioid crisis response, and meetings with emergency medicine and EMS leaders. DDPHE staff held two community meetings in November to follow-up on the seven focus groups and four interviews with recovery support providers conducted in the formative stage of the project. As previously mentioned, the two community meetings coincided with the November 5th alert and were well attended. Participants had an opportunity to film short videos that describe how they have been impacted by the opioid epidemic.
In short, there continues to be a high level of involvement in the evolving design and operation of the early warning system by people at risk, people and agencies who care for them, and data contributors. The Collective Impact leadership team provides an ongoing forum for knowledgeable input and dialog.
Alerts resulting from the November 5th fentanyl discovery were shared with emergency public health managers from surrounding counties, who also distributed them and expressed thanks for the increase in situational awareness.
This project received $135,312 from CDC via CDPHE for this project.
Evaluation assesses the value of the practice and the potential worth it has to other LHDs and the populations they serve. It is also an effective means to assess the credibility of the practice. Evaluation helps public health practice maintain standards and improve practice. Two types of evaluation are process and outcome. Process evaluation assesses the effectiveness of the steps taken to achieve the desired practice outcomes. Outcome evaluation summarizes the results of the practice efforts. Results may be long-term, such as an improvement in health status, or short-term, such as an improvement in knowledge/awareness, a policy change, an increase in numbers reached, etc. Results may be quantitative (empirical data such as percentages or numerical counts) and/or qualitative (e.g., focus group results, in-depth interviews, or anecdotal evidence).
The objectives of our project were 1) to evaluate data streams across the region for their usefulness and specificity in detecting an increase of fentanyl in the local illicit drug supply, 2) display various data streams using one common dashboard to gain a better picture of the problem, and 3) develop a communication plan to reach people who use substances and their support systems in the event that a spike in fentanyl use and effects were to be detected in the community. All objectives were achieved, using various methods to evaluate their success.
To evaluate the thirteen different data streams assessed as part of this project, we used established CDC guidelines for evaluating public health surveillance systems, describing each data stream in terms of simplicity, flexibility, data quality, acceptability, sensitivity, predictive value positive, representativeness, timeliness, and stability. In order to better assess each stream, sample data was collected when possible to better determine ease of use and the ability to have data updated frequently. Following evaluation of each stream, a recommendation on whether to include or exclude from the final dashboard was made.
After evaluation of each data stream and five diverse data streams were selected for inclusion, development of the dashboard was started. The dashboard was iteratively updated to better reflect data trends and to improve the functionality of the dashboard in order to detect meaningful changes. As more was learned about the nuances of each data stream, the display of the data was adjusted to account for this updated understanding. In order to better display trends in positive urine drug screens from Precision Diagnostics, a line was added to the initial graph of positive counts to show the percent of tests coming back positive for fentanyl and/or fentanyl analogs across time. The addition of the line allowed for more standardization as the number of tests changed week over week and across time based on customer contracts, the specific testing volume for the week, and other factors unrelated to fentanyl/analog use. Following initial dashboard creation, meetings were held with data contributors and other stakeholders to gauge the value of the data as displayed and to generate suggestions on further improvements. Several changes were implemented to the dashboard to incorporate the feedback gained during these meetings. Positive fentanyl/analog laboratory results from additional surrounding states were added to the dashboard. Initially the thought was that any new fentanyl contamination in the drug supply would first be seen in New Mexico and southern Colorado, but after discussions with the group we determined that that assumption might not be correct given the new and unprecedented rise of fentanyl in other areas of the country. With the assistance of Precision Diagnostics, we were able to modify the data request to incorporate more surrounding states and display the percent of tests coming back positive for fentanyl in those areas to better detect if spikes are first being seen in areas in any direction outside Denver. Our initial display of data included lines for two and three standard deviations above the mean, but based on feedback from team members at Denver Public Health the display was changed to show a line at the 90th percentile instead. This helps established values above what has historically occurred and allows for a lower and more standardized threshold than using standard deviation, especially in light of the non-normal distributions and limited timeframe for some data streams. To better help analysts and others gain a quick summary of the previous week’s data, metrics were added to the top of the dashboard to indicate the number of data streams that are above normal for the previous week. The metrics summarized the number of values on display from Precision Diagnostics, Denver Health Paramedic Division (DHPD), and ESSENCE which are at or above the 75th and 90th percentile based on historical values from 2019. The metrics are color coded and placed at the top of the dashboard to quickly draw the eye and summarize the data below. Incorporating feedback from multiple stakeholders allowed for creation of a more user-centered design of the dashboard. Based on recommendations from DPH staff, a test of internal consistency for the differing data points was done using Cronbach’s alpha in SAS Enterprise Guide 7.15. Using data from Precision Diagnostics, DHPD, ESSENCE, and deaths recorded by the Office of the Medical Examiner, a Cronbach’s alpha value of 0.697 across the 116 weeks for which data existed in all four sources was found, indicating some degree of internal consistency between these difference data streams. Sharing the dashboard with each data contributor and other stakeholders helped foster relationships and allowed for better understanding of data and gave the opportunity for additional useful data to be shared across organizations. Using data from diverse sources (police, addiction treatment, medical reporting, and deaths) allowed us to better gauge the prevalence of fentanyl in our community and shows that currently no one source of data is able to detect a true signal of a fentanyl outbreak alone.
Formal evaluation of the dashboard also assessed the full dashboard using the same CDC principles that were previously used to evaluate each individual data stream. Formal evaluation was completed prior to an alert being issued. However, an alert was issued in early November 2019 related to seizure of a previously unseen “brick” of counterfeit heroin containing fentanyl. When data was summarized the following week in the dashboard, all three streams analyzed for the overall dashboard metrics (Precision Diagnostics, DHPD, and ESSENCE) were at or above levels seen previously and the 90th percentile metric was reached for all three data streams. In the past, the various data streams had not spiked at the same time and at this level, indicating that potentially the dashboard was able to detect a new substance entering the local illicit drug market.
Communication of the alert was informed by lessons learned from earlier work done by DDPHE as part of this project. DDPHE had an existing list of organizations/agencies to be reached, but by holding focus groups as part of this project additional gatekeepers and other businesses and organizations were identified as needing to also receive the alert. The current distribution list for the alert includes over 100 contacts in and around Denver. Based on input from focus groups, a postcard-sized flyer was also developed with 1,000 copies pre-printed for quick distribution. A QR code was also included on the flyer to allow for direct linking to DDPHE’s fentanyl website for additional information. The website DDPHE developed also includes space at the top of the page in red where issued alerts can be displayed. As previously described, the November 5th alert had a significant reach with 313 mentions and a total reach of 31,610,286. In addition to the community-based notification, the Denver Health Alert Network (D-HAN) allows messaging to health care professionals. DPH maintains a system to create, review and distribute alerts of urgent interest to health care providers in the county. DPH maintains an email distribution list of 340 healthcare and emergency response personnel who have requested to receive alerts. Distributions occur to multiple medical specialties. As part of the communication plan for this project, a template D-HAN was created to allow for rapid dissemination once an alert was issued. The flyer developed by DDPHE was also attached to the D-HAN to allow providers to print and keep them in their offices, lobbies, etc. for patient consumption. Previous Denver Health Alert Network alerts to health care providers have reached over 650 addressees, with a 99.8% delivery rate and 38% open rate. Information was also released to the broad public via press releases from DDPHE. For the alert that occurred on November 5, 2019, a joint press release was created by DDPHE and Denver Police Department.
Sustainability is determined by the availability of adequate resources. In addition, the practice should be designed so that the stakeholders are invested in its maintenance and to ensure it is sustained after initial development (NACCHO acknowledges that fiscal challenges may limit the feasibility of a practice's continuation.)
The success of the dashboard is attributable in large part to the ease with which the organizations providing data have allowed DPH to access data on an as needed basis. Data collected from paramedics and ESSENCE are highly stable, and members of DPH staff have access to both data sources and are able to query both at any time. Precision Diagnostics has allowed DPH analysts access to de-identified data on all tests performed in the multi-state region surrounding Colorado. Analysts are able to query that data and adjust criteria as necessary. The Office of the Medical Examiner has improved their data reporting system and given access to DDPHE analysts to pull data as it is loaded into a business intelligence tool. DDPHE staff is able to query the data and provide de-identified results directly to DPH analysts for inclusion in the dashboard. Random seized drug testing by Denver Police Department (DPD) Crime Lab has initially occurred only as part of this grant program, requiring grant moneys in order to fund a forensic scientist testing a random sample of seized drugs by Denver Police. City and County of Denver funds will continue this testing through December 2020. Bringing together all data contributors and allowing them to review and provide feedback on the dashboard has allowed for greater communication across organizations and built relationships between contributors and analysts at DPH. With most of the data being easily accessible by DPH and/or DDPHE staff, the maintenance required by data contributors has been relatively low. The ability to query most data at any time has also allowed DPH analysts to pull data more frequently, including following detection of the fentanyl “brick” where data was viewed daily by DPH analysts to closely monitor rises in adverse events following identification of this new source of fentanyl.
Using a stable, easily modifiable display for the dashboard (Tableau Software, Seattle, WA) has allowed DPH analysts the ability to rapidly improve the dashboard based on feedback from stakeholder meetings. It also allowed for standardization of metrics across data sources and more unification in how data from diverse data streams is displayed by aligning data by week to quickly see if a spike in one source is also seen in another. Because of the flexibility of the data providers and the software, even after the end of the funding period, the dashboard has been able to be maintained with minimal work from analysts at DPH.
The dashboard has generated a great deal of interest and calls to expand to other substances. Plans are being developed to examine kratom and synthetic marijuana data in a similarly rigorous way by first taking an inventory of available data then bringing together data into one dashboard for comparison across time and across data sources.
DDPHE/DPH staff applied for funding to support community education specifically around fentanyl and measures people can take to protect themselves, friends, and family.
