Conference Agenda

Background

Estimation of so-called hypothetical estimands in clinical trials has historically not made use of data that may be collected after the intercurrent event (ICE). Some recent papers (e.g. Lasch et al 2023) have shown that such data can be used for estimation of hypothetical estimands using causal inference estimators (e.g. G-formula and G-estimation), and that these can be more precise (and thus have higher power) compared to using estimators that only use data before the ICE. This raises the question of whether trials should routinely be using such post-ICE data when estimating hypothetical estimands.

Methods

We critically examine missing data and causal inference methods for estimating hypothetical estimands which do, and do not, make use of post-ICE data, and in particular what assumptions must be made in order for precision to be increased. We derive asymptotic variance expressions for the estimators to quantify the potential gain in precision and power. Simulations are used to verify these.

Results

We first show that G-formula and G-estimators for hypothetical estimands are identical in some important special cases, and as such in these cases their precision/bias properties are identical. We show that estimators that use post-ICE data can only increase precision by assuming the effect of the ICE on outcome is not modified by those variables that affect the occurrence of the ICE and the final outcome, and we argue that such an assumption will often not be plausible. Moreover, our asymptotic variance calculations reveal that in practice the gain in power to detect an effect, even if such assumptions are made, will typically be modest.

Conclusion

Given that the assumptions required to improve precision will, in our view, often not be plausible, and that even if one makes such assumptions, the gain in precision will typically be modest, we recommend that in general estimation of hypothetical estimands continue to be performed using estimators which only use data up until the occurrence of the ICE.

Reference

Lasch F, Guizzaro L, Pétavy F, Gallo C (2023). A Simulation Study on the Estimation of the Effect in the Hypothetical Scenario of No Use of Symptomatic Treatment in Trials for Disease-Modifying Agents for Alzheimer’s Disease, Statistics in Biopharmaceutical Research, 15:2, 386-399, DOI: 10.1080/19466315.2022.2055633

The estimand framework described in the ICH E9 (R1) addendum, causal inference, and pharmacometrics modeling and simulation (M&S) are tools that can help to frame and answer causal questions about the efficacy and safety of clinical products. We aim at integrating these approaches to leverage their respective advantages. The estimands framework translates clinical questions into precisely defined quantitative measures, the estimands. The framework considers post-randomization intercurrent events (IE) that may confound the relationship between the drug intake and the response. Causal inference offers theory, assumptions and methods to express estimands in terms of statistical quantities that can be inferred from observed data. However, often the available data on the clinical endpoint of interest is limited such that an estimand might not be identifiable. This is particularly true when the question of interest concerns a hypothetical estimand, interpolation, e.g. an intermediate dose level, or extrapolation, e.g. to a pediatric population. Pharmacometrics modeling and simulation aims at bridging this gap by taking advantage of possibly available longitudinal pharmacokinetic (PK) and pharmacodynamic (PD) data and external knowledge on the drug and disease in the form of semi-mechanistic models.

We have shown that standard nonlinear mixed effects modeling (NLME M&S) is an implementation of a well-known method in causal inference, standardization. Standardization corrects for confounding by analyzing and combining results from groups of similar patients. In the NLME M&S implementation, conditioning occurs on individual parameters described by the random effects of the NLME model.

Currently we are evaluating the potential and limitations of NLME M&S to correct for confounding. The focus is on studies in which intercurrent events related to either efficacy or tolerability may lead to deviations from the assigned treatment schedule, and on hypothetical estimands of the efficacy at the end of the trial assuming adherence to a given treatment regimen.

We found that if the PK and PD of a drug are well understood and the clinical studies provide rich PK and PD data, NLME M&S is reliable, even with unobserved confounders affecting both the outcome and the dosing. For the more common situation that PK is well understood and supported by data, but only limited data or knowledge is available for the PD, we show that NLME M&S may provide reliable estimates in some situations but fails in others.

Introduction: Treatment effects are typically defined as measures comparing the marginal outcome distributions observed in two or more study arms to assess the efficacy of a novel therapy. Unbiased marginal estimates can be obtained under proper randomisation without adjustments, but previous work showed that covariate adjustments can improve precision even in randomised controlled trials. For non-collapsible effect measures - such as Cohen's d in linear regression, log-odds ratios in binary logistic regression or log-hazard ratios in the Cox model - conditioning on covariates changes the interpretation of treatment effects, making conditional and marginal effects incomparable.

Method: We propose a novel nonparanormal model formulation for adjusted marginal inference allowing the estimation of the joint distribution of outcome and covariates featuring the intended marginally defined treatment effect parameter whose interpretation is unaffected by adjustment. Corresponding marginal distributions are modelled by transformation models allowing broad applicability to diverse outcome types (including, non-normal continuous, binary, ordinal or right-censored survival outcomes). For the special case of Cohen's d under normally distributed outcomes and covariates, we present a closed-form expression of the standard error under covariate adjustment to investigate the potential for sample size reductions theoretically.

Experiments: We evaluated the ability of our proposed method to increase precision under different prognostic strengths of covariates and under different numbers of noise variables in a simulation study. We compared the results with a model conducting unadjusted marginal inference, conditional inference, and previously proposed adjustment methods for marginal inference.

Results: Our proposed approach obtained unbiased parameter estimates of marginally defined parameters with covariate adjustment leading to reduced standard errors, and, thus, narrower Wald confidence intervals, compared to unadjusted marginal inference. The advantages became more pronounced as the prognostic strength increased. Adding noise variables had no large effects on the adjusted parameter estimates.

Conclusion: Overall, this reveals the potential of our method to bypass the problems induced by non-collapsibility of practically important treatment effect measures such as Cohen's d, log-odds ratios, or log-hazard ratios. The approach allows for extensions to estimate heterogeneous treatment effect estimates potentially under observational data offering exciting opportunities for further research.

Introduction

ICH E9 (R1) makes a distinction between what is to be estimated (the estimand) and how to estimate it (the analysis). The estimand should include detail about post-baseline events (intercurrent events) and how they will be handled. The impact intercurrent events (IEs) have on the effect targeted will depend on the handling strategy chosen. A common handling strategy is treatment policy. This means patients' outcome measures post-IE are deemed clinically relevant to estimate the treatment effect and information after the IE should continue to be collected and included in the final analysis. In this setting, the occurrence of missing data post-IE complicates estimation. Multiple imputation is one approach to account for post-IE missing information and research has been done so far for continuous, time-to-event and recurrent event endpoints. We show how this type of estimand can be estimated in the binary setting while accounting for pre- and post-IE information and in the presence of missing data.

Methods

We present the results from a simulation study where we create binary repeated outcomes investigating four different models: a basic missing at random (MAR) model that does not account for the occurrence of the IE, a pre- and post-IE model, a pattern linear model and pattern full model.

Results

Using a basic MAR model showed it was the most biased model which increases as the rate of the IE increases and as the missing data increases. Models that attempt to account for the IE appear to work well, provided there is enough post-IE information recovered, but runs into computational issues for the most complex model.

Discussion

In general, basic MAR models are poor options for estimating effects on binary outcomes that use a treatment policy strategy for IEs. The choice of the most appropriate model will depend on the disease area and the expected rate of collecting post-IE information and it is critical to consider these when choosing an estimation method.

Intercurrent events, such as treatment switching, rescue treatment, and truncation by death, pose significant challenges to the interpretation of treatment effects in randomized clinical trials. Intention-to-treat analyses often fail under these conditions, potentially resulting in misleading conclusions. Existing methods, including hypothetical estimands and survivor average causal effects, address some challenges but rely on strong assumptions, are prone to positivity violations, and struggle with time-varying confounders. In this talk, I will present a novel methodology for analyzing longitudinal clinical trial data impacted by intercurrent events. Our approach does not require data on time-varying confounders and does not exclude positivity violations on the intercurrent events. It relies on a weak structural assumption about the occurrence of intercurrent events and is found to deliver only small bias under its violation. We propose asymptotically efficient, model-free tests of the null hypothesis of no treatment effect, which make use of data-adaptive nuisance parameter estimates. In the context of randomized experiments, we moreover propose asymptotically efficient tests in a subclass of tests that have greater robustness properties. The methodology's empirical performance is demonstrated through simulation studies and the re-analysis of a recent diabetes trial, which is complicated by truncation due to death.