Blog
November 29, 2023
8 mins

Federated learning scales real-world data access for external control arms

The clinical development of a new drug passes through three distinct phases imposed by regulatory agencies and the whole cycle requires 12 years on average to complete. Oncology trials suffer from poor success rates with more than 80% of cancer therapies in phase III trials failing to meet their primary endpoint. This failure rate has a cascade of negative effects across healthcare systems as clinical trials are very expensive and the costs are indirectly passed on to patients and hospitals. Increasing success rates would not only lower costs of drug development but potentially also accelerate the pace at which new drugs can reach patients and improve their lives.

Phase I and II trials primarily aim to measure the maximum tolerated dose and the safety of the drug, not the efficacy of a drug’s benefits for patients. One of the major reasons why trials miss their endpoints is not catching potential issues in these earlier phases. This could be improved by building a Phase “2.5” using real world data in an external control arm analysis, which would help prevent wasting time and patient chances in Phase III that we know will fail and provide intermediate evidence to the regulators for promising new drugs before phase III. 

Randomized clinical trials (RCTs) are the gold standard for evaluating the causal effect of a drug. Traditional clinical trials use concurrent control arms, where participants are randomly assigned to either the experimental group (receiving the new drug) or the control group (receiving a placebo or current standard-of-care treatment). However, in practice, it can be challenging to recruit enough patients to conduct a concurrent control group, especially when conducting rare disease trials or using a drug that targets specific patient subpopulations. Researchers can overcome this challenge by leveraging external control arms, which use other patient data to mimic the characteristics of a concurrent control arm.

What is an external control arm?

External control arms (ECAs) use data from previous clinical trials or real world data in combination with phase II trial data to estimate what would have happened to the participants in the control group, while accounting for sampling biases due to the lack of randomization. They can be particularly valuable in situations where conducting a traditional concurrent control arm is logistically or ethically challenging, such as in the study of rare diseases or in cases where a new treatment is significantly better than existing options for a particular population. Read more about how ECAs work

Federated learning unlocks data access to fuel external control arms

Unfortunately, it is often hard to obtain enough data to power external control arms as it is distributed in different medical institutions and subject to regulations such as GDPR and HIPAA which makes pooling the data significantly more complex from a contractual and logistical perspective. In order to access diverse and representative data for use in external control arms there is a need for privacy-enhancing technologies (PET) to protect both patient confidentiality and competitive advantage for pharmaceutical companies. One such PET - federated learning - enables researchers to harness the knowledge from several institutions while leaving patients’ data on premises by only sharing mathematical quantities known as gradients instead of the raw data. This technology allows researchers to perform data analysis while facilitating compliance with regulations and protecting intellectual property (IP) and data ownership.

Watch the video to learn how federated learning works

Owkin scientists have investigated using recent advances in federated learning in combination with ECA techniques to create different methodologies for distributed external control arms with time-to-event outcomes, while limiting patient data exposure. Inverse probability of treatment weighting (IPTW) is a popular ECA method used to tap into real-world data or historical clinical trials to build a control group for a single-arm trial.

What is ‘inverse probability of treatment weighting’ (IPTW)?

IPTW is a statistical technique that can be used when trying to estimate a causal effect or outcome that mimics randomized control trials. It can help researchers reduce the risk of bias by adjusting for potential confounding variables. First, the propensity score - the probability of being exposed to the target variable of interest given a patient’s characteristics - is calculated for each individual in the study. Then, weights are calculated as the inverse of this propensity score and statistical analysis is performed on this weighted dataset.

Our research introduces a federated learning IPTW method for time-to-event outcomes, which we call FedECA

What is a ‘time-to-event’ outcome?

In medical research, it’s necessary to calculate the time to a specific event in order to understand key outcomes such as disease progression (time until a disease reaches a certain stage) response time (time until a patient responds to treatment) or survival (time until death). RCTs are often focused on understanding time-to-event outcomes.

FedECA extends WebDISCO - an existing method permitting federated learning of time-to-event models in ECA contexts, by integrating propensity scoring and causal inference to the learning process. We illustrated the feasibility of this approach by performing experiments on simulated and real data using Substra, an open source federated learning technology with proven experience in privacy-sensitive contexts in healthcare

Pooled IPTW and FedECA methodologies lead to similar performance results. The errors were computed on simulated data with 100 repetitions.

Our results demonstrated that FedECA is equivalent to the commonly used IPTW method on pooled data and that it even outperforms federated analytics counterparts in specific contexts. We have made code publicly available with unrestricted use for researchers to reproduce the analyses included in the paper and invite the community to contribute. We also explore the use of differential privacy in order to limit potential privacy leakage induced by the repeated sharing of risk sets and its impact on the performance of the analysis. To see the full results, read the paper here.

Jean du Terrail, first author of this research paper said:

By leveraging the latest developments in federated learning (FL) research and open-source softwares, it is now possible to build distributed data analysis pipelines that are both as easy to use and produce results of similar quality as one would get if one could pool all the data in a single place, which is not often not feasible in practice. While there is still a lot to be done on the research side to strengthen the privacy of such analyses even further, the FL community is continuously making progress on that ground and Owkin is proud to contribute to that open and collaborative effort.
Improving external control arms to de-risk and accelerate clinical trials

External control arms are a promising solution to improve clinical trial outcomes and FedECA can be a powerful approach to unlock the data from multiple sources needed to make them more effective in real world settings. Reducing the number of failed oncology trials could greatly positively impact the pharmaceutical industry, healthcare systems and ultimately patients. By increasing scientific and clinical accuracy, biopharma companies can also improve investor and regulatory perception of their assets. Owkin are experts in both clinical trial optimization and the application of federated learning in healthcare settings, and can help organizations navigate new approaches in this rapidly changing landscape.

Read more about how Owkin’s external control arms increase the probability of success of clinical trials

Authors
Jean du Terrail
Liz Allen
Blog
November 29, 2023
8 mins

Federated learning scales real-world data access for external control arms

The clinical development of a new drug passes through three distinct phases imposed by regulatory agencies and the whole cycle requires 12 years on average to complete. Oncology trials suffer from poor success rates with more than 80% of cancer therapies in phase III trials failing to meet their primary endpoint. This failure rate has a cascade of negative effects across healthcare systems as clinical trials are very expensive and the costs are indirectly passed on to patients and hospitals. Increasing success rates would not only lower costs of drug development but potentially also accelerate the pace at which new drugs can reach patients and improve their lives.

Phase I and II trials primarily aim to measure the maximum tolerated dose and the safety of the drug, not the efficacy of a drug’s benefits for patients. One of the major reasons why trials miss their endpoints is not catching potential issues in these earlier phases. This could be improved by building a Phase “2.5” using real world data in an external control arm analysis, which would help prevent wasting time and patient chances in Phase III that we know will fail and provide intermediate evidence to the regulators for promising new drugs before phase III. 

Randomized clinical trials (RCTs) are the gold standard for evaluating the causal effect of a drug. Traditional clinical trials use concurrent control arms, where participants are randomly assigned to either the experimental group (receiving the new drug) or the control group (receiving a placebo or current standard-of-care treatment). However, in practice, it can be challenging to recruit enough patients to conduct a concurrent control group, especially when conducting rare disease trials or using a drug that targets specific patient subpopulations. Researchers can overcome this challenge by leveraging external control arms, which use other patient data to mimic the characteristics of a concurrent control arm.

What is an external control arm?

External control arms (ECAs) use data from previous clinical trials or real world data in combination with phase II trial data to estimate what would have happened to the participants in the control group, while accounting for sampling biases due to the lack of randomization. They can be particularly valuable in situations where conducting a traditional concurrent control arm is logistically or ethically challenging, such as in the study of rare diseases or in cases where a new treatment is significantly better than existing options for a particular population. Read more about how ECAs work

Federated learning unlocks data access to fuel external control arms

Unfortunately, it is often hard to obtain enough data to power external control arms as it is distributed in different medical institutions and subject to regulations such as GDPR and HIPAA which makes pooling the data significantly more complex from a contractual and logistical perspective. In order to access diverse and representative data for use in external control arms there is a need for privacy-enhancing technologies (PET) to protect both patient confidentiality and competitive advantage for pharmaceutical companies. One such PET - federated learning - enables researchers to harness the knowledge from several institutions while leaving patients’ data on premises by only sharing mathematical quantities known as gradients instead of the raw data. This technology allows researchers to perform data analysis while facilitating compliance with regulations and protecting intellectual property (IP) and data ownership.

Watch the video to learn how federated learning works

Owkin scientists have investigated using recent advances in federated learning in combination with ECA techniques to create different methodologies for distributed external control arms with time-to-event outcomes, while limiting patient data exposure. Inverse probability of treatment weighting (IPTW) is a popular ECA method used to tap into real-world data or historical clinical trials to build a control group for a single-arm trial.

What is ‘inverse probability of treatment weighting’ (IPTW)?

IPTW is a statistical technique that can be used when trying to estimate a causal effect or outcome that mimics randomized control trials. It can help researchers reduce the risk of bias by adjusting for potential confounding variables. First, the propensity score - the probability of being exposed to the target variable of interest given a patient’s characteristics - is calculated for each individual in the study. Then, weights are calculated as the inverse of this propensity score and statistical analysis is performed on this weighted dataset.

Our research introduces a federated learning IPTW method for time-to-event outcomes, which we call FedECA

What is a ‘time-to-event’ outcome?

In medical research, it’s necessary to calculate the time to a specific event in order to understand key outcomes such as disease progression (time until a disease reaches a certain stage) response time (time until a patient responds to treatment) or survival (time until death). RCTs are often focused on understanding time-to-event outcomes.

FedECA extends WebDISCO - an existing method permitting federated learning of time-to-event models in ECA contexts, by integrating propensity scoring and causal inference to the learning process. We illustrated the feasibility of this approach by performing experiments on simulated and real data using Substra, an open source federated learning technology with proven experience in privacy-sensitive contexts in healthcare

Pooled IPTW and FedECA methodologies lead to similar performance results. The errors were computed on simulated data with 100 repetitions.

Our results demonstrated that FedECA is equivalent to the commonly used IPTW method on pooled data and that it even outperforms federated analytics counterparts in specific contexts. We have made code publicly available with unrestricted use for researchers to reproduce the analyses included in the paper and invite the community to contribute. We also explore the use of differential privacy in order to limit potential privacy leakage induced by the repeated sharing of risk sets and its impact on the performance of the analysis. To see the full results, read the paper here.

Jean du Terrail, first author of this research paper said:

By leveraging the latest developments in federated learning (FL) research and open-source softwares, it is now possible to build distributed data analysis pipelines that are both as easy to use and produce results of similar quality as one would get if one could pool all the data in a single place, which is not often not feasible in practice. While there is still a lot to be done on the research side to strengthen the privacy of such analyses even further, the FL community is continuously making progress on that ground and Owkin is proud to contribute to that open and collaborative effort.
Improving external control arms to de-risk and accelerate clinical trials

External control arms are a promising solution to improve clinical trial outcomes and FedECA can be a powerful approach to unlock the data from multiple sources needed to make them more effective in real world settings. Reducing the number of failed oncology trials could greatly positively impact the pharmaceutical industry, healthcare systems and ultimately patients. By increasing scientific and clinical accuracy, biopharma companies can also improve investor and regulatory perception of their assets. Owkin are experts in both clinical trial optimization and the application of federated learning in healthcare settings, and can help organizations navigate new approaches in this rapidly changing landscape.

Read more about how Owkin’s external control arms increase the probability of success of clinical trials

Authors
Jean du Terrail
Liz Allen
About Owkin

Owkin is the first end-to-end TechBio company on a mission to understand complex biology and derive new multimodal biomarkers through AI. We identify precision therapeutics, de-risk and accelerate clinical trials and develop diagnostics using AI trained on world-class patient data through privacy-enhancing federated technologies.

We merge wet lab experiments with advanced AI techniques to create a powerful feedback loop for accelerated discovery and innovation in oncology, cardiovascular, immunity and inflammation. Owkin also founded MOSAIC, the world’s largest spatial multi-omics atlas for cancer research.

Owkin has raised over $300 million through investments from leading biopharma companies, including Sanofi and BMS, and venture funds like Fidelity, GV and Bpifrance, among others.

Federated learning scales real-world data access for external control arms

No items found.
No items found.
No items found.