Efficient 3D reconstruction of Whole Slide Images in Melanoma

Cutaneous melanoma is an invasive cancer with a worldwide annual death toll of 57,000 (Arnold et al., JAMA Dermatol 2022). In a metastatic state, surgical interventions are not curative and must be coupled with targeted therapy, or immunotherapy. However, resistance appears almost systematically and late-stage prognosis can remain poor. The complexity to eradicate melanoma stems from its plasticity; these cancer cells continually adapt to the tumor microenvironment, which leads to treatment resistance. Our primary assumption is that therapeutic resistance relies in part on a series of non-genetic transitions including changes in the metabolic states of these cancer cells. The 3D spatial distribution of blood vessels that are sources of nutrition and oxygen that drive this metabolic status is an important variable for understanding zoning aspects of this adaptation process. Using Whole Slide Images (WSI) of melanoma tumors from Patient-Derived Xenograft (PDX) mouse models, we build 3D vascular models to help predict and understand the metabolic states of cancer cells within the tumor. Our 3D reconstruction pipeline was based on PDX tumor samples sectioned over 2mm depth and stained with Hematoxylin and Eosin (H&E). The pipeline involves three primary steps, including 2D vessel segmentation using Deep Learning, intensity- and affine-based image registration, and 3D reconstruction using interpolation and 3D rendering (allowing for better interaction with biologists, pathologists, and clinicians). The originality of our computer-assisted pipeline is its capability to (a) deal with sparse data (i.e., not all tissue sections were readily available), and (b) adapt to a multitude of WSI-related challenges (e.g., epistemic uncertainty, extended processing times due to WSI scale, etc.). We posit both our 3D reconstruction pipeline, quantitative results of the major stages of the process, and a detailed illustration of the challenges faced, presenting resolutions to improve the pipeline's efficiency.