Interpretation of surrogate endpoints in the era of the 21st Century Cures ActBMJ 2016; 355 doi: https://doi.org/10.1136/bmj.i6286 (Published 30 December 2016) Cite this as: BMJ 2016;355:i6286
All rapid responses
Drs. Knopf et al  express concerns over speeding drug approval using additional indicators of treatment benefit. Currently, regulatory decisions on new drugs occur within six or ten months’ time based on priority assigned by FDA following submission of the complete new drug application (NDA). Will this Act enhance new drug approvals? In 2001, with the evidence overwhelmingly positive and unprecedented, the FDA approved a new drug, imatinib, within 6 weeks of NDA receipt. Since the mid-1990s, a regulatory pathway called accelerated approval (similar to conditional approval in the EU) has been available to expedite approval when efficacy and safety are sufficiently well-characterized. With decades of experience, this option allows speedier approval actions by utilizing less definitive but well-studied surrogate/intermediate outcome measures. These measures remain controversial to some; implementing new endpoints for even faster regulatory decision-making may be challenging. Accelerated approval is not a pass to full approval; sponsors must subsequently confirm a definitive clinical benefit, and failure here may warrant accelerated removal.
Agreed, the most important, compelling outcomes in clinical trials are improved quality or quantity of life (preferably both); however, limitations to feasibility of assessing these ultimate outcomes may undermine or abrogate their use. Long time intervals for survival assessment, evolving changes in active or control regimens or supportive care, and changing disease diagnostic criteria may make the definitive data obsolete or uninterpretable when finally available. New subsequent therapies may eclipse the effects of the earlier therapy on overall survival (OS). Symptoms of therapy may overlap symptoms of disease. The quest for more perfect evidence to support a definitive approval might hinder availability of newer, better therapies.
These ultimate goals are the right ones, but is rigidity necessary for all drugs all the time? If there is no subsequent therapy of promise for a serious or life-threatening disease, should the accelerated approval bar be lower? Also, the FDA is not empowered to assess or consider the price of a drug in the approval decision; cost decisions occur elsewhere in the U.S. Knopf et al  describe several cogent situations supporting return to definitive endpoints and more conclusive quantitation of benefit; however, while regulatory agencies may express preferences for specific trial designs, analyses, and endpoints, in the absence of a clear safety concern, sponsors may conduct trials as they choose to without regulatory concurrence or comfort.
Regardless of trial intent, the answer to the utility of an intermediate endpoint may be: “it depends”. First, regarding surrogates commonly used in oncology that may indicate drug activity or clinical benefit such as response rate, time to disease progression, or progression-free survival (PFS, a composite of tumor progression or death from any cause), there is surrogacy and then there is surrogacy. Established surrogates, such as blood pressure control or reduction of LDL cholesterol, have been extensively validated in thousands of patients in many settings sufficient to conclude that a favorable effect on the surrogate (BP or LDL-C) reliably produces a favorable effect on the clinical benefit endpoint, such as reduction in risk of stroke, myocardial infarction, or premature death, sufficient to approve a drug based on its effects on the validated surrogate. For accelerated approval, a trial may use a surrogate endpoint which is required only to be judged “reasonably likely” to produce the desired effect on the clinical benefit endpoint, such as overall survival (OS). How successful is the FDA track record on applying this judgment?
The actual experience of the past 15 years for FDA decision making is informative. Among more than 200 drugs granted accelerated approvals since implementation of this authority about 24 years ago, only one oncology drug, gemtuzumab ozogamicin, was subsequently withdrawn from marketing approval (by its sponsor). The drug’s planned confirmatory trial, intended to demonstrate the surrogacy of response to predict OS in acute leukemia, failed to do so. Otherwise, under actual field conditions, the Agency’s ability to judge a new drug “reasonably likely” to produce definitive clinical benefit has been well supported by subsequent trials confirming clinical benefit. The “reasonably likely” threshold considers the duration and size of the effect, toxicities, and available alternative therapies. Following U.S. accelerated approval, post-marketing requirements and commitments negotiated with the approval specifics set the clock for ongoing review, absent interim safety concerns. The European Medicines Agency re-assesses safety and efficacy annually for conditional approval drugs.
Following independent FDA review and analysis of the drug dossier, evidence for small increments in response or in PFS may be judged insufficient to meet a clinical threshold level sufficiently persuasive to grant approval. In other cases, the strength of evidence may be uncertain and may lead to public presentation at an Advisory Committee meeting seeking independent expert opinion on the strength of the evidence and whether the evidence is “reasonably likely to predict”. Even a definitive trial outcome, e.g., improvement in overall survival, is not always clear as to its clinical benefit, although the endpoint itself is definitive. Is an increment of a day, a week, or a month always clinically meaningful? Further, the risks of therapy may be substantial, unpredictable, or inadequately characterized to label for safe use. It is fair for Knopf and others to question intellectually if an extension of life by a median amount of 10 days is meaningful, but this peek at the evidence overlooks the totality of the overall survival impact of a drug, better expressed as hazard ratio or restricted median survival time. For PFS, minimizing bias in assessing clinically meaningful improvements mandates a concurrent control arm of appropriate standard therapy against which to judge time-to-events with attention to who is measuring the tumor, by what means, how often, and how symmetrically on study arms.    At times, the magnitude of improvement in PFS (with acceptable safety) may justify traditional approval.  In trials powered for OS, a PFS difference may appear statistically very robust (more events); however, in addition to being statistically persuasive, a PFS difference should be clinically persuasive if the PFS result is intended to support an initial accelerated approval. Beyond the surrogacy issue and the statistical obscurities, even more daunting is to assign a value to some quantity of change in symptom relief or survival, especially for ourselves. Clinical judgment explicitly expressed should be favored over absolutism.
1 Knopf K, Baum M, Shimp WS, et al. Interpretation of surrogate endpoints in the era of the 21st century cures act. BMJ 2016;355:i6286 doi: 10.1136/bmj.i6286
2 Carroll KJ. Analysis of progression-free survival in oncology trials: Some common statistical issues. Pharmaceut. Statist. 2007; 6: 99–113.
3 Panageas KS , Ben-Porat L , Dickler MN , Chapman PB , Schrag D . When you look matters: the effect of assessment schedule on progression-free survival. J Natl Cancer Inst 2007;99:428–32.
4 Kane RC. When You Look Matters: The Effect of Assessment Schedule on Progression-free Survival. JNCI 2007; 99:1131-32.
5 Kane RC. The clinical significance of statistical significance. The Oncologist 2008; 13: 1129-33.
Competing interests: No competing interests