Conference Agenda

The adoption of complex innovative clinical trial designs has been increasing in recent years. These are trial designs that have one or more unconventional features, which aim to improve upon conventional trial designs. The motivation for these designs may not be difficult to follow, but their set-up and implementation is usually more challenging. Statistical properties of these designs can also be difficult to compute. Clinical trial simulation (CTS), which uses software to generate artificial data for learning, can be conducted to identify the (optimal) setting of a clinical trial, evaluate the design's statistical properties, perform ``what-if" analyses, and compare different design set-ups and data analysis strategies, all of which contributes to a better understanding of the value of unconventional features before implementing the design in an actual clinical trial. It is also a tool for methodologists to investigate a novel design. Existing literature on simulation primarily focuses on the evaluation of statistical analysis methods, with less attention on the detailed specification and planning of CTS. This work presents a step-by-step planning process for CTS in the context of complex innovative trials. The target audience comprises both trial statisticians who are involved in designing and analysing clinical trials, and statistical methodologists who focus on the development of trial designs and of analysis methods.

Background

Chimeric Antigen Receptor (CAR)-T cell is an immunotherapy which revolutionised the treatment of relapsed/refractory lymphoma and leukemia. It is shown to have higher response rate, higher mid-to-long term overall survival, and lower toxicity than standard treatments. However, due to lack of dose-limiting toxicity (DLT) and unclear dose-effect relationship, traditional phase I designs of clinical trials cannot lead to accurate selections of the optimal dose (OD). Among the reported trials, the design of phase I clinical trials are frequently unclear, and early phase designs specifically for CAR-T cells are needed. Beside clinical outcomes, the CAR-T cell expansion from serial blood samples is measured at various timepoints. Two main profiles of cell-evolution have been reported: the injected cells exhaust and are progressively eliminated from the blood, or they proliferate and are maintained over some duration before being eliminated.

Methods

We propose a novel early phase dose-finding design for CAR-T, using both toxicity and activity endpoints to locate the OD, the dose with highest activity among safe doses. The CAR-T cells expansion is used to indicate activity, which is more sensitive than traditional clinical responses. A Bi-Exponential model is used to model the expansion trajectory, which approximates the CAR-T cells growth dynamics, is simple enough to be estimated with small sample sizes, and is flexible enough to accommodate the reported cell-evolutions. Three criteria for activity are considered: 1) number of cells at specific time, 2) duration before all cells are eliminated, 3) area under the cell-expansion curve. A non-parametric benchmark has been developed to evaluate the performance of the proposed design.

Results

Simulations show that the OD can be selected with high accuracy even under small sample sizes. All three activity criteria work well when the model is correctly specified. Furthermore, the model is robust under model-misspecification if the appropriate activity criterion is used. Sensitivity analyses show that the proposed design is robust against missing measurements of CAR-T cells expansion and increased noise level.

Conclusion

Both toxicity and activity endpoints should be used for CAR-T cells, and the CAR-T cells expansion is a more sensitive and specialised measure for biological activity compared to clinical outcomes. Depending on the activity criteria, higher doses do not necessarily give higher activities.

Motivation:

Evaluating operating characteristics (OCs), such as expected sample size, power, and type I error, is essential for designing clinical trials. These OCs guide critical design decisions and are vital for interactions between the study team, regulatory agencies, and other stakeholders.

Traditionally, OCs are estimated using Monte Carlo simulations. This approach is based on repeatedly sampling the clinical trial under a specific scenario selected by the researcher. The OCs are then computed as functions of the aggregated results. This approach, however, can be computationally expensive, especially for complex trial designs (e.g., adaptive trials) or models requiring intensive inference (e.g., MCMC-based Bayesian methods). These computational demands can make thorough design evaluation impractical.

Our proposal:

We introduce the Q-approximation, a method for rapidly approximating OCs by leveraging three key principles:

1. Many clinical trial designs adhere to the likelihood principle, meaning all necessary information for decision-making is contained in the likelihood function.
2. Under mild regularity conditions, the log-likelihood is approximately quadratic, and the likelihood is approximately Gaussian.
3. The distributions of the center and curvature of the quadratic approximation can be derived using standard asymptotic theory.

These considerations allow approximating OCs by directly simulating likelihood functions instead of simulating entire datasets as in traditional Monte Carlo analyses. More specifically, the Q-approximation can be much faster than the Monte Carlo approximation for two main reasons:

• Dimensionality Reduction: Instead of simulating entire datasets,
  we sample two-dimensional vectors, representing the center and the curvature of the approximation.

• Fast computation of inferences: Because the approximate likelihood is Gaussian, inferences can be computed analytically (e.g., no need for MCMC in a Bayesian logistic model).

Applicability and results:

The Q-approximation can be applied to more complex settings, including multi-stage adaptive designs, Bayesian adaptive randomization, and trials incorporating external data. We demonstrate its effectiveness across three trial settings: (a) non-adaptive two-arm randomized controlled trials (RCTs), (b) adaptive RCTs leveraging external data, and (c) multi-arm RCTs with Bayesian adaptive randomization. Our results show that the Q-approximation provides accurate OC estimates and reduces computational time by up to a thousandfold compared to Monte Carlo methods. This speed-up can make it much more practical to explore operating characteristics thoroughly and recommend complex designs.

Background

Due to the complex causal pathways involved, quantifying the effects of screening, particularly the effects of early detection, can be problematic. In the context of screening, modelling studies can be used as a way of linking intermediate and long-term health outcomes when RCT data are limited. A previously conducted methodological review of modelling studies, using intermediate trial outcomes for estimating morbidity/mortality reductions of screening interventions, found serious methodological limitations which hinders their usability. Consequently, a new a framework for quantifying the overall (i.e. combined effects of early detection and treatment) and early detection effects of screening is introduced.

Methods

In a typical screening trial participants allocated to screening are invited for testing while control participants are tested when they present with symptoms. In both groups those who are detected are treated using similar treatment protocols. Therefore, if the patients referred for treatment are prognostically similar between groups, there is no reason to expect the treatment effects to differ between control and screen detected patients.

If trial participants are randomised for a second time, then an unbiased effect estimate of early detection can be derived by comparing the number of events observed in screened arm as per original randomisation to expected number of events in re-randomised control arm. The overall effect of screening can be estimated by comparing the number of outcome events in screen compared to control groups as per original randomisation.

Such a re-randomisation approach to estimating the effect of screening can be implemented using simulation. The feasibility of such an approach is illustrated using data extracted from five cancer (breast, colorectal, cervical, lung, and prostate) screening trial publications.

Results

The expected mortality reductions, estimated using a re-randomisation approach using intermediate outcomes, are in-line with mortality reductions observed in the trials with the ratio of relative risks (observed to expected) ranging from 0.66 to 0.99. However, there was a greater variability in the estimated expected mortality reductions when there was a larger difference in number of cases with late-stage disease (i.e. lymph node and/or distant metastatic disease at the first presentation) between the trial arms.

Conclusion

The framework introduced and applied here provides a unified analysis approach for quantifying the overall and early detection effects of screening. A simulation study to assess the performance of the proposed approach across a range of scenarios is on-going.

Background: Early-phase clinical trials in oncology are characterized by high uncertainty and small sample sizes, making decision-making challenging. The overall success rate of development programs, as measured by the likelihood of approval, was previously estimated to be ~26% (DiMasi et al 2005). Informed decision making is therefore crucial for success, both in terms of speed and quality of the decisions taken. Focus has shifted from traditional hypothesis testing to quantitative decision-making frameworks that acknowledge the early development challenges. Such frameworks not only enhance the decision-making process but also empower statisticians to contribute to critical development decisions.

Methods: A Quantitative Decision-Making (QDM) framework was established which leverages Bayesian approaches to quantify the probability of success in early-phase studies. It integrates relevant criteria and risk thresholds based on internal targets (target-product-profile), as well as external/published data. The goal of the framework is to offer statistical guidance for cohort expansion or discontinuation, and possible transition to late development.

The talk will detail the steps taken to integrate methodology into clinical planning practices and to expand the role of statisticians.

Results: The implementation of the QDM framework facilitates a systematic evaluation of (preliminary) efficacy based on empirical evidence obtained in the ongoing early-phase clinical trial(s) as well as from external or prior evidence, when available. Based on these data the framework assesses the probability of achieving clinically meaningful outcomes (DiScala et al 2013). Using pre-defined criteria and thresholds, decisions regarding continuation or stopping studies are better informed by quantification of the risks involved.

The talk will demonstrate the calculation of decision criteria for a Phase I/II study and show how the operating characteristics are assessed to ensure the robust decision criteria. The framework has been implemented within governance meetings, providing a pathway for statisticians to actively shape the clinical development strategy.

Conclusions: The QDM framework enhances the decision-making process in early development oncology by providing a robust and actionable analytical approach to quantify uncertainty and risk. By adopting this framework, drug development can become more consistent, allowing for informed decisions that optimize the potential for successful clinical outcomes in a competitive landscape.

The talk will address challenges to widespread adoption, projecting increased use of such decision criteria in the future.

References

DiMasi, Grabowski. Economics of new oncology drug development. J Clin Oncol, 25(2007)

Di Scala, Kerman, Neuenschwander. Collection, synthesis, and interpretation of evidence: a proof-of-concept study in COPD. Statistics in Medicine, 32(2013)