Multimodal AI generates virtual population for tumor microenvironment modeling.

Publication Title

Cell

Authors

Jeya Maria Jose Valanarasu
Hanwen Xu
Naoto Usuyama
Chanwoo Kim
Cliff Wong
Peniel Argaw
Racheli Ben Shimol, Earle A. Chiles Research Institute, Providence Cancer Institute, Portland, OR, USA.Follow
Angela Crabtree, Earle A. Chiles Research Institute, Providence Cancer Institute, Portland, OR, USA.Follow
Kevin Matlock, Earle A. Chiles Research Institute, Providence Cancer Institute, Portland, OR, USA.Follow
Alexandra Q Bartlett, Earle A. Chiles Research Institute, Providence Cancer Institute, Portland, OR, USA.Follow
Jaspreet Bagga
Yu Gu
Sheng Zhang
Tristan Naumann
Bernard A Fox, Earle A. Chiles Research Institute, Providence Cancer Institute, Portland, OR, USA.Follow
Bill J. Wright, Providence Research Network, Renton, WA, USA.Follow
Ari Robicsek, Providence Research Network, Renton, WA, USA; Arclight Intelligence, Newton, MA, USA.Follow
Brian D. Piening, Providence Genomics, Portland, OR, USA; Earle A. Chiles Research Institute, Providence Cancer Institute, Portland, OR, USA.Follow
Carlo Bifulco, Providence Genomics, Portland, OR, USA; Earle A. Chiles Research Institute, Providence Cancer Institute, Portland, OR, USA.Follow
Sheng Wang
Hoifung Poon

Document Type

Article

Publication Date

12-9-2025

Keywords

digital pathology; multimodal AI; precision health; real-world data; real-world evidence; spatial proteomics; tumor microenvironment; virtual population.; oregon; genomics; portland; prn; renton; artificial intelligence

Abstract

The tumor immune microenvironment (TIME) critically impacts cancer progression and immunotherapy response. Multiplex immunofluorescence (mIF) is a powerful imaging modality for deciphering TIME, but its applicability is limited by high cost and low throughput. We propose GigaTIME, a multimodal AI framework for population-scale TIME modeling by bridging cell morphology and states. GigaTIME learns a cross-modal translator to generate virtual mIF images from hematoxylin and eosin (H&E) slides by training on 40 million cells with paired H&E and mIF data across 21 proteins. We applied GigaTIME to 14,256 patients from 51 hospitals and over 1,000 clinics across seven US states in Providence Health, generating 299,376 virtual mIF slides spanning 24 cancer types and 306 subtypes. This virtual population uncovered 1,234 statistically significant associations linking proteins, biomarkers, staging, and survival. Such analyses were previously infeasible due to the scarcity of mIF data. Independent validation on 10,200 TCGA patients further corroborated our findings.

Area of Special Interest

Cancer

Specialty/Research Institute

Oncology

DOI

10.1016/j.cell.2025.11.016