Conference Agenda

Background / Introduction: Recently, several health technology assessment bodies across European Union countries have begun incorporating users' demands for quicker access to digital medical devices (DMDs). Introducing a conditional fast-track pathway that enables early access and reimbursement presents an attractive solution for patients, healthcare professionals, and the medical device industry. However, regulators must ensure they have enough data to justify provisional reimbursement and user access, even if complete clinical evidence is not yet available. By the time manufacturers complete the clinical study, they will have also gathered real-world data (RWD) from the DMD's use in the population. This dual source of information—clinical trial data and RWD—provides regulators with a richer evidence base for final decision-making, a significant improvement over the traditional reliance solely on clinical trial data. We propose a statistical framework that integrates clinical trial data and RWD, allowing a more rigorous evaluation of DMDs in compliance with European fast-track regulatory requirements.

Methods: Our framework includes three stages: (1) an interim analysis of clinical trial data to support temporary regulatory authorization and enable the collection of RWD; (2) a final analysis of the clinical trial data; and (3) a meta-analysis that integrates RWD with clinical trial data, provided temporary authorization was be granted. Various metrics are introduced to optimize the timing of the interim analysis and the application for temporary authorization. The framework was evaluated using a simulation study.

Results: Using a significance level of 0.025 and a power of 0.9 for a one-sided two-sample test for proportions, (0.7, 0.4) as the true response rates for the intervention and control arms in the clinical trial population, c* = 0.8 as the threshold for the conditional power (CP), the timing of the interim analysis is deduced as t = 0.5 and the CP is greater than c* in 70% of cases. In these situations, the temporary regulatory authorization is obtained, and therefore RWD can be collected in parallel with the second part of the clinical trial. Then, integrating RWD into the final analysis allows a gain in effective sample size compared to the traditional approach.

Conclusion: This framework should include a post-market evaluation of the DMD after its widespread adoption, in line with the principles of phase IV studies. While the primary purpose of the final augmented analysis is to refine the findings of clinical trials, it can also help to assess the generalizability of those results in real-world settings.

In randomized clinical trials that use a long-term efficacy endpoint T, the follow-up time necessary to observe T may be substantial. This may limit the timing of interim analyses based on T. In such trials, an attractive option is to consider an interim analysis based solely on an early outcome S that could be used to expedite the evaluation of treatment's efficacy.

Garcia Barrado and Burzykowski (Pharmaceutical Statistics 2024) developed a methodology that allows introducing such an early interim analysis for any combination of S and T types. It appears that such a design may offer substantial gains in terms of both the expected trial duration and the expected sample size. A prerequisite, though, is that the treatment effect on S has to be strongly correlated with the treatment effect on T, i.e., the early outcome is a good trial‑level surrogate for the long-term endpoint.

When developing their methodology, Garcia Barrado and Burzykowski assumed that the coefficients defining the (trial‑level) model used to evaluate the properties of S as a surrogate for T were known. However, in practice, only estimates of the coefficients, obtained by using data from a meta-analysis, would be available. This fact importantly limits the applicability of the methodology. In the current manuscript, we address this issue.

Methods

To adjust for the fact of estimation of the trial‑level surrogacy model, the variance of the interim‑analysis test‑statistic has to be inflated by a term related to the variance‑covariance of the estimated model coefficients. We obtain an explicit expression for the term by using measurement‑error modelling. By applying the expression to a set of hypothetical scenarios for a clinical trial, we evaluate the gain in operating characteristics of a trial with an interim analysis based solely on data for S, with and without the adjustment for the estimation of the model. We also illustrate the application of the developed results by using a real‑life clinical trial.

Results

As expected, the adjustment leads to a reduction in the gain in operating characteristics. It appears, however, that if S is a good surrogate for T, the relative reduction may be small.

Conclusion

The obtained results allow for designing trials with an interim analysis based only on an early outcome, while properly adjusting for the error resulting from the estimation of the model capturing properties of the outcome as a surrogate for the long‑term endpoint.

Futility analysis is an effective adaptive design that enables trials to potentially be terminated at a predetermined interim stage. For pharmaceutical companies and clinical trial sponsors, balancing the risk of prematurely stopping a trial for a potentially successful drug, against the risk of continuing a trial when collected interim data suggests the drug is ineffective, is a complex challenge. The distinct nature, significance, financial and other implications of these risks make them too intricate to be easily aggregated for decision-making purposes. Furthermore, the inclusion, timing, and strictness of futility designs have a considerable spillover effect on the entire portfolio. The opportunity cost of making no (or suboptimal) interim futility gating decisions, which could free up financial and human resources for reinvestment in other opportunities in the pipeline, exemplifies a portfolio-wide impact.

This work aims to introduce a refined framework to identify optimal futility designs for individual trials but viewed from a portfolio-level perspective. The framework seeks to balance the errors of falsely continuing and falsely stopping trials by weighing them according to their financial impact, both at the trial level and, more importantly, at the portfolio level.

Consequently, this framework quantitatively determines the extent of leniency and prudence in risk-taking based on the total portfolio-level financial impact of interim decisions while considering the mutual interconnectivity among various trials within the portfolio. Besides accounting for the unique financial characteristics of each trial, the proposed framework also evaluates the alternative uses of freed-up resources within the portfolio by assessing what could have been achieved if some trials had been prematurely stopped. This scheme can subsequently suggest suitable futility designs for each trial whilst ensuring overall portfolio-level optimization.

To demonstrate the sensitivity of the framework, several sensitivity analyses were conducted. These analyses primarily addressed the uncertainty surrounding the assumed drug effect at the trial's design stage, as well as the uncertainty related to cost evaluation caused by the potential loss of not investing in other opportunities within the portfolio due to the continuation of less effective and riskier trials.

Background: In the conduct of interim analysis to adapt trial design, the U.S. FDA 2019 guidance "Adaptive Designs for Clinical Trials of Drugs and Biologics" emphasizes the need to control the risk of drawing erroneous conclusions and to reliably estimate the underlying treatment effect. Statistical methodology to achieve these objectives has long-been available to trial statisticians; however, they carry the essential assumption that the underlying treatment effect is homogeneous across the stages of the trial.

Methods: Two settings in which homogeneity may become implausible are the presence of (1) underlying disease endotypes and (2) outcomes that depend on the patients' perception about their assigned treatment. Focusing on these scenarios and a variety of interim analysis objectives, we examine how knowledge about the heterogeneity may be incorporated into either quantifying objective measures of risk or devising design adaptations.

Results: With reasonably broad knowledge about the extent treatment effect heterogeneity, we derive an upper bound on error associated with decisions based on interim analysis for futility. When relatively little is known about the heterogeneity beyond its presence, we devise some general strategies to mitigate risk and stage effects and illustrate in the setting of design adaptations on multi-arm trials.

Conclusions: Potential departures from treatment effect homogeneity must be carefully considered in planning any comparative interim analysis. Although heterogeneity may pose an insurmountable challenge for certain design adaptations, there are settings where sound decisions on the basis of interim analysis may still be made - particularly if one is willing to trade off benefits and costs for the sake of caution.

Introduction

Suppose an adaptive Phase III trial has an interim analysis scheduled at a given information fraction, e.g., 50%. The key question is: When will it reach 50% information? In a non-longitudinal setting, the information level for a continuous endpoint can be approximated by the fraction of patients with the endpoint data at the interim analysis relative to the final analysis. However, longitudinal trials with repeated measures and missing data require more nuanced methods to estimate the information level accurately. The question then becomes: When will there be 50% information in the presence of missing data? Is it when half of the patients reach the final visit, or could it be earlier?

Methods

We propose an approach for projecting the information fraction at an interim analysis in a continuous longitudinal trial analysed using the MMRM framework. We establish a relationship between information time and calendar time, providing practical guidance. At the design stage, a prediction for the timing of interim analysis is based on assumptions about enrolment rate, total sample size, dropout rate, visit timing, and the correlation matrix between visits. Once some data are available, this prediction is refined using the observed enrolment rates, dropout patterns, and updated correlation estimates, yielding a more accurate estimate of the current information level and an updated timeline for the interim analysis.

Results

We demonstrate that we can project information timelines at the design stage and refine them as data accrues. In the context of a worked example, we show how to navigate different missing data patterns, assess the current information level, and set a reliable timeline for the interim analysis.

Conclusion

Accurately estimating the timing of the interim analysis in longitudinal trials with missing data is essential for optimizing trial conduct, especially in terms of ethics and allocation of efforts and resources. Leveraging both initial design assumptions and accumulating trial data, our approach enhances decision-making, ensuring that interim analyses occur at the intended information fraction.