Most analyses of adverse events (AEs) are based on the presence or absence of events at the patient level. For example, if a patient experiences one or more headaches, they are considered a “success” in that the event occurred (= 1). Patients who do not experience a headache are considered failures (= 0). In traditional summaries of safety, the number of patients who experience one or more headaches are tabulated for each treatment arm, and the proportion (or odds ratio or risk ratio) of subjects experiencing a headache are compared between treatment arms. This analysis has important limitations. First, patients who experience one event versus many events are weighted identically with no distinction made for the number of events a patient may experience (except for patients who experience no events). Second, the timing at which these events occur is entirely ignored within the analysis. Study teams have no insight into whether events occur early in treatment compared to late, or if the occurrence of events is constant, increasing, or decreasing over time. The Recurrence Report in the forthcoming JMP Clinical 19.0 makes it easy to screen a clinical trial for noteworthy safety concerns. Throughout this blog post, we illustrate methodologies using data from patients with probable mild-to-moderate Alzheimer’s disease.
When using CDISC standards, the AE or ADAE data sets summarize the events that were experienced by patients in the study, but more information is needed to perform a recurrence analysis. Dates representing the start and end of an appropriate follow-up period need to be defined for all patients, not just those who experience a particular AE. Further, the derivation of the appropriate follow-up dates will depend on the types of AEs of interest for analysis (Figure 1).
Figure 1. Follow-up for Particular Adverse Events
- All Events include any AEs that may occur on study once informed consent is signed until the patient ends study participation.
- Treatment Emergent Events include any AEs that may occur on or after the first dose of study therapy determined by the treatment arm.
- On Treatment Events include any AEs that may occur on or after the first dose of study therapy and before or at the last dose of study therapy.
- Pre-Treatment Events include any AEs that may occur prior to the first dose of study therapy.
- Off Treatment Events include any AEs that may occur after the last dose of study therapy.
Ohidul Siddiqui describes the limitations of traditional analyses for AEs and how analysis of the mean cumulative function (MCF) can provide a more informative analysis. Briefly, the MCF is the mean number of cumulative events up to a particular time. This is a non-parametric analysis with no assumptions about the form of the MCF, and at any given point along the step function, the MCF can be interpreted as the average of individual cumulative AE curves across all patients. Analyses of recurrence using MCFs have been used extensively in manufacturing applications to assess the likelihood of product failures over time, and the Recurrence Analysis platform within JMP performs these analyses. However, these analyses have not been as readily performed in medical product development and subsequent regulatory review. There may be several reasons for this:
- Limited Software. As Siddiqui describes, these methodologies are available in PROC RELIABILITY which is a part of SAS QC software. In my experience, this software was rarely available to statistical teams involved in the design and analysis of clinical trials. PROC PHREG has the ability to perform recurrence analyses with some additional assumptions and with what I think is a challenging way to communicate the follow-up period. While this may have been a greater limitation in the SAS-dominated regulatory environment of the past, recurrence analyses still do not appear to be regularly performed. For example, they are not part of FDA standard safety outputs.
- Data Visualization Burden. Perhaps the lack of regular usage of recurrence analysis is due to the necessity of the data visualizations to help communicate the story behind the data. For example, has anyone anywhere ever produced a Kaplan-Meier analysis without the accompanying curve to communicate a particular endpoint? This is highly unlikely. But given the large number of AEs present in the typical clinical trial, this presents a very large analysis burden. Further, there needs to be a straightforward way to review multiple recurrence analyses quickly to identify the most important signals.
- Rare Events. While it is true that the most problematic AEs tend to be rare making recurrence analysis a bit of overkill, there are some therapeutic areas with a deluge of AEs that traditional approaches to AE analysis do a disservice. In addition, there are many commonly occurring AEs that would benefit from more informative summaries. Even in situations where events are rare, the explicit timing of those events are considered which is more informative than merely summarizing incidence.
- Analysis Literacy. Perhaps these methods are not being used because statisticians and data scientists have done a poor job explaining why these analyses are needed and how the resulting output should be interpreted. Here, I am very hopeful. Clinicians have grown very comfortable with Kaplan-Meier curves and plots of MCF are not extremely different.
JMP Clinical 19.0 solves many of the above issues. The analysis is quick, highly graphical, and interactive. Further, the Recurrence Report leads with a summary to help identify particular AEs with notably large MCFs or differences between treatment arms.
Treatment emergent adverse events are summarized in the analysis that follows.
