ML4H / AHLI Logo

OncoShift

ML4H 2026
Multi-Modal Auditing Harness (Pathology • Radiology • EHR)

Enterprise Robustness & Safety Auditing for Health AI

Quantifying latent-space representation collapse (W₁, GW₂), Brier calibration error, and clinical Net Benefit failure under distribution shift across 10 foundation backbones.

N = 1,000 Cohort Evaluated10 Active System ModelsSDK: pip install oncoshift
Multi-Modal Shift Sweeps

Simulates stain variation, defocus blur, motion noise, and Hawkes process EHR missingness cascades across 5 severity levels (ε ∈ [0.0, 1.0]).

Optimal Transport Geometry

Computes exact 1st Wasserstein Distance (W₁ via POT linear programming) and Gromov-Wasserstein alignment (GW₂) to track latent collapse.

Clinical Decision Audit

Decomposes Brier score calibration penalty (Murphy 1973) and tracks clinical Net Benefit loss via Decision Curve Analysis at threshold pt = 0.30.

Multi-Model Robustness Leaderboard (10 Models)

Rankings evaluated under maximal simulated distribution shift (ε = 1.0)

Foundation ModelModalityWasserstein Shift (W1) ↑Brier Reliability ↓Rel (TTA) ↓Net Benefit (pt=0.30) ↑Attribution (ρ attn)Status
Virchow2Pathology0.34600.25000.25000.28570.6864 Audited
UNIPathology0.26180.25000.25000.28570.7900 Audited
Prov-GigaPathPathology0.26920.25000.25000.28570.7913 Audited
CONCHPathology0.27250.25000.25000.28570.7942 Audited
UniMedVLRadiology0.47330.25000.25000.2857-0.0694 Audited
BiomedCLIPRadiology0.49960.25000.25000.2857-0.0799 Audited
MedSAMRadiology0.51020.25000.25000.2857-0.0837 Audited
LLaVA-MedRadiology0.49470.25000.25000.2857-0.0577 Audited
OdysseyEHR0.17020.01460.02600.49800.9665 Audited
CEHR-BERTEHR0.16000.00560.00830.50000.9664 Audited