E-values in JCA – a rising star for robustness checks of MAICs?

 

🎓️ In two recent JCA reports, the assessors have pointed out the importance of correctly using E-values for binary and survival outcomes and in conjunction with robustness checks of MAICs.

 

❓️ So what is an E-value, and how is it used?

 

Statistical Method:

📊 The E-value is a way to deal with unmeasured confounders.

📊 The E-value quantifies how strongly an unmeasured confounder would have to be associated with both the intervention and the outcome to fully explain away the observed intervention-outcome association.

📊 It can be calculated directly from the risk ratio.

📊 For example, a risk ratio of 2 corresponds to an E-value of 3.4. Confounders weaker than that could still partially account for the association but not explain it entirely.

 

Easy interpretation:

💡The E-value shows how strong a hidden bias would need to be to erase your result — the higher the E-value, the more robust (and trustworthy) your finding.

 

Lessons learned:

🔹 As the JCA assessors have pointed out, if E-values are calculated, this must be done consistently and with care. E-values can help assess whether intervention-outcome associations are likely due to confounders. However, they cannot confirm this, since the actual strength of unmeasured confounders is never known. They are one more tool in a statistician’s toolkit.

🔹 Point-estimate E-values alone aren't enough — always calculate and report the E-value for the confidence interval limit closest to the null too, since a smaller degree of unmeasured confounding can already overturn statistical significance even when the point estimate itself looks robust; omitting this leaves assessors unable to judge the true fragility of the results.

🔹 There's no fixed E-value threshold for "robust enough." Without knowing the strength of association between missing prognostic variables/effect modifiers and the variables already accounted for, an E-value is hard to interpret on its own. It must always be judged in context.

 

👀 It will be very interesting to observe how E-values are used in the upcoming JCA procedures.