Data Sources for Signal Detection

Signal detection in pharmacovigilance is the process of actively searching for and identifying safety signals from a wide variety of data sources. Signal Detection in GVP Module IX is one of the core stages of GVP Module IX for Signal Management. [link to blog]. This guide will explain the sources of data and information used in signal detection, the statistical aspects of signal detection, documenting the signal detection process, and what next generation signal detection software looks like.

Read more about the paradigm shift occurring in pharmacovigilance signal detection software in a recent article in Applied Clinical Trials, Pharmacovigilance Software's "Salesforce" Moment.

The most common source for signals comes from spontaneous reporting systems that are required by regulators to be kept by biopharma companies. These data are stored in a database that is either hosted and maintained by the company themselves, or outsourced to a technology or full services contract research organization (CRO). Signal detection on this database is required as a part of periodic monitoring. Additionally, data from national databases like the FDA Adverse Event Reporting System (FAERS), EudraVigilance, and VigiBase are important sources of potential signals.

While spontaneous reporting systems remain the gold standard for signal detection, other sources including, but not limited to, scientific literature that discuss adverse events and active monitoring systems such as the FDA Sentinel system should be considered.

Signal Detection Methods

Once a drug is on market and most products quickly amass a number of ICSRs that is too large to reasonably review at the case the level for the purposes of signal detection. Thus, appropriate prioritization thresholds must be put in place to focus attention on groups of ICSRs. There are both descriptive as well as quantitative ways of achieving prioritization.

Adverse events should be categorized using the MedDRA hierarchy to be able to identify signals using different levels of granularity. Preferred Term (PT) level of analysis has been shown to have the best combination of sensitivity and positive predictive value for signal detection. [Hill R, Hopstadius J, Lerch M, Noren GN An attempt to expedite signal detection by grouping related adverse reaction terms. Drug Saf. 2012; 35:1194-1195.]

Also, to focus efforts on serious and important events, the Important Medical Event (IME) list should be used to filter results that need further medical review. The IME list can be even further refined by mapping a list of Designated Medical Events (DME) that should always be prioritized. In addition, it is important for those conducting signal detection to have access to a company’s core common data sheet (CCDS) file of labeled events. This can help to ensure that further work on validating and assessing a signal is not done for information that is already known.

Quantitative signal detection is most commonly done through disproportionality statistics; The ratio of the proportion of spontaneous ICSRs of a specific drug-event combination to the proportion that would be expected if no association existed between the product and the event. There are various different ways to calculate disproportionality, with the most common using frequentist methods such as the reporting odds ratio (ROR) or the proportional reporting ration (PRR) and using Bayesian methods such as the Empirical Bayes Geometric Mean (EBGM) and the Information Component (IC). ROR for example is calculated:

Variables: a = the number of primary case reports involving Drug X and Adverse Event Yb = the number of primary case reports involving Drug X and NOT Adverse Event Yc = the number of primary case reports involving NOT Drug X and Adverse Event Yd = the number of primary case reports involving NOT Drug X and NOT Adverse Event Y.


a = 4 cases of nausea reported for Invokana (sitagliptin) b = 50 cases of all other adverse events reported for Invokana (sitagliptin) c = 100 cases of nausea reported across all other drugs in Evidex d = 1000 cases of all other adverse events reported across all other drugs in Evidex

ROR = (a/b)/(c/d) = (4/50)/(100/1000) = .08/.10 = .80

Any value of disproportionality greater than 1 indicates that the drug-event combination is being reported more than expected. It is common to use the lower bound of a 95% confidence interval greater than one or the absolute mean value greater than two as a prioritization threshold. [Candore G, Juhlin K, Manlik K, Thakrar B, Quarcoo N, Seabroke S, et al. Comparison of statistical signal detection methods within and across spontaneous reporting databases. Drug Saf. 2015; 38: 577–587]

Trends can also be evaluated on a drug-event combination, for both frequency and disproportionality to better understand changes over time. Large, unexpected jumps in periodic reporting could, for example, highlight a manufacturing quality issue that a more subtle change in disproportionality would not show.

Furthermore, applying signal detection specifically for individual populations of patients should be considered. Pediatric and geriatric populations, for example, have special characteristics that should be differentiated.

Documenting the Signal Detection Process

Good Pharmacovigilance Practices (GVP) are a set of measures put into practice in 2012 to facilitate the performance of pharmacovigilance in the European Union (EU). GVP is broken out into several modules that govern different aspects of pharmacovigilance processes. GVP Module IX - Signal Management (GVP IX) provides general guidance and requirements on scientific and quality aspects of signal management.

The guidelines apply to Marketing Authorization Holders (MAH) for medicines authorized by the European Medicines Agency (EMA) and EU Member States. However, in the absence of formal regulation on the process of signal management by other health authorities, such as FDA, these guidelines have become the de facto global standard.

There are no specifications on how these requirements are met, but in the age of software-as-a-service (SaaS), technology is playing an ever bigger role.

In addition to signal detection in GVP IX, there is the concept of signal validation, prioritization, and assessment. A full overview of the GVP IX signal management process can be found in Advera’s, GVP Module IX for Signal Management: The Complete Guide. The overarching theme throughout all stages of GVP IX, is that the processes used should be adequately documented and the steps taken in the management of a specific signal should be tracked and auditable. In short, the system needs to provide for transparency into the who, what, why, and when of the process. 

Next Generation Signal Detection Software

Next generation pharmacovigilance signal detection is built with a view toward how data and analytics software can better interact not only with traditional sources like ICSR databases, FAERS, VigiBase, and clinical trial data, but with emerging disparate sources such as social media, claims, EHR and other unstructured data. Bringing these pools of information together creates an opportunity to enhance signaling algorithms, make validations and assessment more efficient, and ultimately get answers to drug safety questions faster.

To be able to bring these data together, extensive ontologies need to be created to link all of the data. Drug name and active ingredient represented as NDC, RxCUI or ICSR drugs are resolved to one record. Adverse event coding in MedDRA is mapped back to verbatim labeling and ICD-10 codes. Drugs are characterized by ATC classifications, NDF-RT, label status, and more.  These ontologies and mappings need to be provided off-the-shelf to be able to further drive immediate, actionable insight.

Arguably more important than how software interacts with the data, is how end users engage with software. Complicated, slow, and unintuitive software leads to a poor user experience. Legacy software and platforms that were built during web 1.0 are no longer acceptable. And datamining in 2019 and beyond should not require an end user to be a data scientist. Next generations signal detection software need to reinvent how end users feel about the tools they use for their day-to-day job. Bottom line, a safety reviewer has to want to use and engage with the software, rather than see it as a burden. When this happens, an organization’s pharmacovigilance software shifts from not only just a system of record, or just one of engagement, but truly a system of intelligence that was not previously capable with legacy platforms. 

Advera Health’s Evidex, a cloud-based, software-as-a-service (SaaS) drug safety data, analytics and signal detection platform shifts the paradigm of pharmacovigilance software and enables the science of pharmacovigilance to advance at a rapid pace. Further innovation will come when end users are empowered to take advantage of disparate data sources through a modern user experience. More information on next generation pharmacovigilance software can be found in Advera's White Paper: Pharmacovigilance Software in 2019.

Talk to us.

See Advera Health's Evidex in action, get questions answered, or just talk drug safety with like minded pharmacovigilantes.