Conference Agenda

The use of odds ratios and hazard ratios as primary effect measures in randomized trials and observational studies has faced significant criticism due to challenges in interpretation and communication. Recent advancements have shifted focus toward model-free estimands for marginal treatment effects, employing data-adaptive statistical modeling and machine learning to address imbalances in randomized trials or confounding in observational studies. However, these methods often lack direct applicability to patient-centered decision-making. While conventional treatment effect measures, such as odds ratios, can map conditional counterfactual mean outcomes (given baseline covariates) from untreated to treated scenarios, this involves reliance on parametric structures, making them susceptible to bias from model misspecification.

We address this challenge by introducing a novel debiased machine learning strategy that minimizes the squared bias induced by such mappings. Our approach ensures minimal bias by construction, leverages flexible data-adaptive methods to estimate conditional outcome means, accommodates model uncertainty (e.g., variable selection uncertainty), and provides insights into subject-specific treatment effects, enabling transparent communication with patients and clinicians through tailored visualizations.

Our proposal drastically advances previously proposed assumption-lean modeling strategies by maintaining interpretability under model misspecification, delivering more efficient estimators, and offering insight into treatment effect heterogeneity. We demonstrate the advantages of our method through simulation studies and an analysis of a recent diabetes trial.

A concentration index, a standardised covariance between a health variable and relative income ranks, is often used to quantify income-related health inequalities. There is a lack of formal approach to study the effect of an exposure, e.g., education, on such measures of inequality. In this paper we contribute by filling this gap and developing the necessary theory and method. Thus, we define a counterfactual concentration index for different levels of an exposure. We give conditions for the identification of this complex estimand, and then deduce its efficient influence function. This allows us to propose estimators, which are regular asymptotic linear under certain conditions. In particular, we show that these estimators are $sqrt n$-consistent and asymptotically normal, as well as locally efficient. The implementation of the estimators is based on the fit of several nuisance functions. The estimators proposed have rate robustness properties allowing for convergence rates slower than $sqrt{n}$-rate for some of the nuisance function fits. The relevance of the asymptotic results for finite samples is studied with simulation experiments. We also present a case study of the effect of education on income-related health inequalities for a Swedish cohort.

Quantifying causal effects in the presence of complex and multivariate outcomes is a key challenge to evaluate treatment effects. For hierarchical multivarariates outcomes, the FDA recommends the Win Ratio and Generalized Pairwise Comparisons approaches. However, as far as we know, these empirical methods lack causal or statistical foundations to justify their broader use in recent studies. To address this gap, we establish causal foundations for hierarchical comparison methods. We define related causal effect measures, and highlight that depending on the methodology used to compute Win Ratios or Net Benefits of treatments, the causal estimand targeted can be different, as proved by our consistency results. Quite dramatically, it appears that the causal estimand related to the historical estimation approach can yield reversed and incorrect treatment recommendations in heterogeneous populations, as we illustrate through striking examples. In order to compensate for this fallacy, we introduce a novel, individual-level yet identifiable causal effect measure that better approximates the ideal, non-identifiable individual-level estimand. We prove that computing Win Ratio or Net Benefits using a Nearest Neighbor pairing approach between treated and controlled patients, an approach that can be seen as an extreme form of stratification, leads to estimating this new causal estimand measure. We extend our methods to observational settings via propensity weighting, distributional regression to address the curse of dimensionality, and a doubly robust framework. We prove the consistency of our methods, and the double robustness of our augmented estimator. These methods are straightforward to implement, making them accessible to practitioners.

Background As prediction algorithms are increasingly used to support individual decision making in healthcare, the fairness, trustworthiness and transparency of such algorithms are paramount. To prevent biases of regular prediction algorithms developed on observational data, causal prediction (also referred to as counterfactual prediction or prediction under interventions) has been proposed as a basis for decision support. Causal prediction relies on untestable assumptions about the data underlying the algorithm. In view of algorithm trustworthiness, the plausibility of these assumptions should be evaluated as well as the statistical uncertainty associated with the algorithm. Crucially, both can vary widely between individual patients. We aimed to develop a metric that helps to evaluate and communicate the trustworthiness of a causal prediction algorithm on an individual patient level.

Methods For regular prediction algorithms, the effective sample size for an individual can be interpreted as the number of similar individuals (i.e., with similar predictor values) that their prediction is effectively based on. To translate the concept of effective sample size to a causal prediction setting, we explored the connections between the effective sample size and the causal inference assumptions of exchangeability and positivity. Furthermore, we developed methods to estimate effective sample sizes for causal predictions. These methods were applied to a clinical dataset, leading to a prototype of how causal effective sample size might be presented to end-users as part of a decision aid used in diabetes care.

Results In contrast to the regular effective sample size, the causal version may consist of multiple effective sample sizes for the same individual, i.e. one for each (hypothetical) intervention option considered. In addition, the causal effective sample size should extend the definition of ‘similar individuals’ to cover adjustment factors (confounders) that are not part of the predictor set. As such, the causal effective sample size can be used to consider the plausibility of the positivity assumption for an individual. In the clinical data, we observed large between-individual differences in causal effective sample sizes.

Conclusion While causal assumptions are traditionally only evaluated globally, we have shown how they can also be evaluated for individuals. This improves the transparency of causal prediction algorithms and allows end-users to determine on an individual basis whether they trust and want to use the predictions from the algorithm in their decision-making.

Funding This project has received funding from the Dutch Organisation for Scientific Research (NWO) under Grant ID (DOI) 10.61686/DFECP93059.

Background: Randomized controlled trials (RCTs) are the gold standard for estimating causal effects. However, their validity is compromised by noncompliance, particularly when unmeasured confounders influence both compliance and outcomes. In cardiovascular trials, prioritized composite outcomes are common, with the win ratio being a popular method for estimating treatment effects in such cases. The win ratio compares treated and control pairs to determine the winner based on the higher-priority event. If a winner cannot be determined, the secondary event is used for the comparison. Recently developed win ratio methods incorporating propensity scores cannot account for unmeasured confounders, making them unsuitable in the presence of noncompliance. Common approaches like intention-to-treat (ITT), as-treated (AT), and per-protocol (PP) analyses also fail to provide valid causal estimates when noncompliance occurs.

Methods: This study proposes a win ratio estimator that incorporates instrumental variable (IV) to address noncompliance. The idea is to fit a regression model of the treatment received on treatment assigned and extract the residuals. These residuals are then used as a proxy of the unmeasured confounders and then consequently adjusted for to calculate the win ratio. A simple form of the IV win ratio estimator is derived, which does not require any information on the unmeasured confounder.

Results: The proposed IV win ratio is compared with simpler alternatives like ITT, AT, and PP using simulations, focusing on bias, standard error, and coverage. Under no treatment effect, AT and PP show high bias and poor coverage even with 5% noncompliance, while ITT and IV remain robust. For strong treatment effects, all estimators perform well under full compliance, but ITT, AT, and PP degrade as compliance decreases. At 85% compliance, their coverage drops to 67%, 23%, and 60%, while IV maintains near-nominal coverage of about 95%. Applying the proposed methods to the JOBS II randomized field experiment data reveals a significant effect of the job training on reemployment and depression among unemployed subjects. Specifically, individuals are 21% more likely to achieve a more favorable outcome (either reemployment or reduced depression) if they received the job-skills training compared to if they just received a booklet.

Conclusion: Noncompliance in trials with prioritized composite outcomes is often overlooked. This study introduces an IV win ratio estimator, highlighting its superiority over ITT, AT, and PP win ratios. It provides practical guidance for choosing the most suitable methods for analyzing prioritized composite outcomes in the presence of noncompliance.