Early stopping of clinical trials

Abstract

Early stopping of clinical trials in favour of a new treatment creates ethical and scientific difficulties, which are different from those associated with early stopping due to toxicity or futility. Two major breast cancer trials have recently taken such a decision, and the problem is relevant for several ongoing trials. Here we argue that such a decision should be taken with the utmost gravity and should be based on a clear overall clinical benefit for the new treatment, and not as an automatic response to crossing a predefined threshold. Predefined rules can be used to trigger a debate within the Independent Data Monitoring and Safety Committee (IDMC) about early stopping, but the IDMC should retain the responsibility of assessing overall clinical benefit in making its recommendation.

Introduction

The conduct of clinical trials requires a careful and meticulous consideration of the rights of the individual to autonomy and self-determination on one hand and, on the other, the needs of society to learn which treatment is most effective for a particular disease. This tension is present from the very outset, when patients are asked to consent to randomization, and continues throughout the trial, as early events accrue and results for other related studies are reported. Independent Data Monitoring and Safety Committees (IDMCs) are now de rigueur for all trials, and are charged with regularly reviewing this balance and protecting the interests of patients.

A key tenet of this whole activity is the concept of uncertainty. This principle underpins the credibility of all clinical trials, and considerable experience and maturity are required for a full understanding of its ramifications. In simplest terms this principle holds that randomization is appropriate for an individual patient if the clinician has substantial uncertainty as to which of two (or more) treatments is likely to provide greatest clinical benefit to him or her. In most cases this is not a precise knife-edge balancing of risks and benefits, but a determination of whether the patient fits in a large grey area where there is uncertainty.

In the past, differences in approaches to treatment led to separate schools of thought as to how best to treat a disease. At best, only indirect, often unbalanced, comparisons were available for determining the superiority of one treatment over another (or even any effectiveness beyond a placebo effect). About 50 years ago, enlightened physicians realized the weakness of this approach; they agreed to suspend their personal prejudices temporarily and subject their preferred treatment to a head-on comparison with a treatment championed by other doctors. The randomized clinical trial was born, and with it came all the ethical and social dilemmas associated with suspending personal prejudices and with temporarily accepting uncertainty until clear evidence emerges for a clinical benefit for one or another treatment. We still struggle with this concept and with recent attempts to more formally provide procedures for evaluating uncertainty that have led to unease among ourselves and others.

It seems to us that the uncertainty principle requires that clear evidence of clinical benefit of a new treatment should be obtained before a trial is stopped with a recommendation to accept a new treatment. There is a clear asymmetry here, in that stopping a trial early in favour of a new treatment is different from stopping a trial with a negative recommendation. The primary role of the IDMC is to protect the safety of the patients. Any unexpected or excessive toxicity of a new treatment requires prompt action – either to halt the trial temporarily while a remedy is sought, or to abandon it altogether. Additionally, early stopping when the efficacy of a new treatment seems worse, or even when it becomes clear that it is unlikely to be significantly better than standard treatment (so-called futility analyses) is also an important consideration for the IDMC.

The HABITS ('hormonal replacement therapy after breast cancer – is it safe?') trial is a good example of this. It was expected that the use of hormone replacement therapy would have minimal impact on recurrence in breast cancer patients, but early results showed a threefold increase [1]. Although some individuals question the knock-on effect of early stopping on other ongoing trials in which no excess was seen, safety considerations dictate a cautious approach in this instance.

It is clear that the threshold for action here should be lower than early stopping for success. Especially for futility analyses, it is important to consider whether the trial can usefully contribute to an overview that could provide a definitive answer, but when the patient population is limited or there are no similar ongoing trials a decision to stop a trial in favour of standard treatment (or no treatment) can, in our view, be appropriate even when there is overall clinical uncertainty. A good example of this was the combination arm in the ATAC (arimidex, tamoxifen alone or in combination) trial. It was dropped when the combination did not seem to be materially different from tamoxifen, but the use of anastrozole alone was superior [2].

However, the threshold for action when a new treatment looks promising should be much higher. Of particular concern to us has been the early stopping of trials when early endpoints (such as disease-free survival or time to recurrence) are positive. This has precluded any opportunity of clearly establishing overall benefit. The expansion of randomized clinical trials into areas requiring prolonged treatment of patients with a relatively good prognosis has exacerbated this problem, because many patients are still on active treatment when these interim analyses are done, and the time to definitive outcome (death or distant recurrence) is long.

Paradoxically, the early stopping of trials with apparently successful outcomes can delay full acceptance of the treatment. When uncertainty still persists in a substantial segment of the medical community, the trial has not achieved its primary goal, and the main aim of the trial remains unanswered. There seems to be a trend towards giving IDMCs inflexible rules as to when they must stop trials, and this is taking precedence over their judgement about whether there is a clear clinical benefit associated with the new treatment.

What is to be done about this?

It seems to us that there are two actions that would help to rectify the situation.

First, IDMCs should be given back the duty of making judgements about the clinical benefit of any differences seen in early endpoints. Stopping rules are useful for setting triggers that would initiate this activity, but IDMCs should then review safety, toxicity and other potential long-term evidence before coming to a recommendation about irrevocably abandoning a trial.

Second, when early indicators are positive but it is felt that clinical benefit has not been achieved, procedures for reporting these results in a way that does not require unblinding should be developed. This will require informing patients of the interim results, indicating that further follow-up is necessary to obtain definitive results, and seeking re-consent to continue in this trial. This has been successfully achieved in both the first International Breast Cancer Intervention Study (IBIS-I) [3] and ATAC trials [4] and is very much in the spirit of treating patients as partners in research, rather than simply as subjects exposed to new treatments. This approach is particularly important for trials of prolonged treatment in patients with a good prognosis (for example adjuvant hormone and prevention trials) in which interim results are likely to be available at a time when many patients still have a substantial period of active treatment ahead of them.

Such a partnership requires developing a different relationship with patients at the time of diagnosis and keeping them informed about the progress of the trial. It would be helpful if the consent form signed at the outset embodied these principles. Ideally, awareness of this partnership approach to clinical trials should be publicized and debated quite generally so that patients are aware of and comfortable with it before disease is apparent.

Could this approach have been employed in the trials that were stopped early?

For the National Surgical Adjuvant Breast and Bowel Project P-1 (NSABP-P1) tamoxifen prevention trial [5], the IBIS-I experience suggests that with an appropriate advance information and understanding, continuation would have been well tolerated and even valued.

The ATAC trial also demonstrates the ability to report positive results for new treatment and yet not unblind the patients. Very few patients on tamoxifen or anastrozole have requested unblinding. Should this have been done for the MA17 trial of extended treatment with letrozole after tamoxifen [6]? The original question of 5 years of treatment after 5 years of tamoxifen asked in that trial has not been answered, although it is now being used as the standard against which to compare 5 years of letrozole against 10 years of letrozole after initial treatment with 5 years of tamoxifen. No mortality benefit is yet apparent and it is unclear whether the present difference in recurrence will translate into a real difference. The recent report [7] of a significant mortality benefit in node-positive patients also noted a non-significant detriment in node-negative patients, with a non-significant overall mortality benefit. What are we to make of this? The implications of unblinding would be a switch of placebo patients to letrozole after a 29-month median gap. Is this beneficial? Given these uncertainties would it not have been better to report the result but give the patients the opportunity to remain on blinded treatment if they wished?

Similar questions arose in the Intergroup Exemestane Study trial [8], which compared 5 years of tamoxifen with switching to exemestane after 2 to 3 years of tamoxifen. A highly significant difference in favour of switching was reported, but most patients had completed treatment or would do so within a year, so the results were published and patients were informed. However, the blinding and treatment schedules were maintained and randomized treatment was continued in consenting patients. This is very important and should allow a long-term assessment of the effect on mortality to be determined.

Similar questions will now arise in the BIG 1–98 trial [9] in which an early benefit for letrozole over tamoxifen has been reported. The situation is somewhat different here, because more evidence is available from other trials on the subject, and after the initial 2 years of treatment women may also switch treatment or continue for a further 3 years. The positive interim results from three adjuvant trials of trastuzumab (Herceptin) recently announced at ASCO [10] will also require careful consideration of these issues, although significant mortality benefits were also seen here.

These issues will always be a matter of judgement and differing opinion, but we feel that the IDMC's role must include an assessment of clinical benefit in addition to looking at the primary endpoint, and it is in everyone's interest that the decision to abandon a trial in favour of the new treatment is taken with the utmost gravity. When early indicators are strongly positive, but an effect on definitive endpoints has not yet been established, we need to find a way to announce these early findings without damaging the chances of obtaining the conclusive results that the trial set out to provide. Perhaps we should involve the patients more in this process and provide them with the opportunity to remainin the trial until clinical benefit is clearly established.