High resolution alignment of 3D imaging with 2D imaging
Inventors
WINETRAUB, YONATAN • Yuan, Edwin • Terem, Itamar • Yu, Caroline • De La Zerda, Adam
Assignees
Leland Stanford Junior University
Publication Number
US-12361656-B2
Publication Date
2025-07-15
Expiration Date
2040-04-10
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
Alignment of a 2D image to a corresponding 3D image is provided by writing a pattern into a 3D sample. The pattern is at known positions in the 3D image, and provides visible reference features in the 2D image. This permits accurate determination of the plane in the 3D image that corresponds to the 2D image.
Core Innovation
The invention provides a method for aligning a 2D image with a corresponding 3D image using a predetermined pattern written into a 3D sample. The pattern, placed at known positions in the 3D sample through optical bleaching techniques, creates visible reference features in the 2D slice, enabling accurate determination of the matching plane in the 3D image that corresponds to the 2D image.
This method addresses the problem in clinical practice of correlating a single 2D histo-pathology section with the corresponding 3D imaging data, such as from Optical Coherence Tomography (OCT). Prior to this invention, accurate alignment at single-cell resolution was challenging due to distortions and lack of identifying markers in the sample.
The technique involves staining tissue with fluorescent dye (or embedding it in a fluorescent gel), photobleaching specific patterns using the OCT laser, followed by conventional histology sectioning. The photobleached barcode is visible in both modalities and supports alignment with accuracy better than 20 microns, facilitating pairing of histology with structural and temporal information from OCT imaging.
Claims Coverage
The patent claims include multiple independent claims focusing on methods for aligning 2D images to 3D images by writing and using predetermined fluorescence patterns, and on the use of this data for machine learning. The main inventive features are detailed as follows.
Method of aligning 2D and 3D images by fluorescent pattern bleaching
A method comprising providing a 3D sample, sensitizing it to fluorescence, optically writing a predetermined pattern via fluorescence bleaching, cutting a 2D slice, imaging the 3D sample with a first modality and the 2D slice with a second modality, and determining the matching 2D section using reference features from the predetermined pattern.
Pattern formed by intersecting lines creating planes
Forming the predetermined pattern as intersecting lines written on the top surface of the 3D sample, which correspond to intersecting planes in 3D, with the first and second set of lines perpendicular to each other and spaced to permit line identification through distance ratios in the 2D image.
Parametrization of 2D slice plane in 3D sample
Representing the 2D section by a vector origin and two vector directions, determining x and y components by least squares fitting, assuming uniform shrinkage and no shear for z components of plane directions, and determining the z component of the origin by independent measurement.
Use of particles to improve and quantify alignment accuracy
Adding particles visible in both imaging modalities to the 3D sample, using particle features in the 2D and 3D images to improve and quantify the alignment between the two images.
Extending method to align two 3D images
Applying the 2D-to-3D alignment method multiple times to align two or more planes of a second 3D image to a first 3D image, enabling multimodal 3D image registration.
Incorporation of various first and second imaging modalities
Using first imaging modalities including intravital microscopy, lipid-cleared 3D imaging, optical coherence tomography, photoacoustic imaging, two photon microscopy, confocal microscopy, diffusive optical imaging, and magnetic resonance imaging; and second imaging modalities including histological imaging, non-histological imaging, optical microscopy, electron microscopy, and mass spectroscopy imaging.
Application to various sample types
Applying the method to ex vivo biological samples, in vivo biological samples, and non-biological samples.
Creation of training datasets for machine learning
Performing the alignment method on multiple samples to create paired datasets relating the first imaging modality to histology, then training a machine learning system that can synthesize histology images from the first image modality.
Characterizing sample distortion
Characterizing sample distortion caused by cutting the 2D slice and/or performing the second imaging modality, including time-dependent characterization of dynamic distortion.
The claimed invention covers a comprehensive method for high-resolution alignment of 2D images to 3D images by sensitizing and photobleaching a fluorescent pattern in a 3D sample, enabling accurate plane matching and extension to various imaging modalities and sample types, while also supporting machine learning applications and characterization of sample distortions.
Stated Advantages
Provides unprecedented alignment accuracy of less than 20 microns between 2D histology images and corresponding 3D imaging.
Enables accurate alignment to facilitate combined structural, dynamic, and molecular information from different imaging modalities.
Supports creation of paired training datasets that enable machine learning to synthesize histology-like images from non-invasive imaging.
Improves understanding and compensation of sample distortions due to cutting and imaging processes.
Applicable to a wide range of biomedical and non-medical applications, including materials science and calibration of cutting devices.
Documented Applications
Using machine learning to directly predict histological images from OCT images for non-invasive histology-like imaging.
Combining structural and dynamic information from OCT with metabolic or protein information from histochemistry.
Imaging activities of particular cells in tumors longitudinally in OCT and correlating with molecular information from histology at a single time point.
Characterization of sample deformation and distortion during cutting and imaging processes.
Tracking deformation in soft-materials processing in material sciences for stress and shear analysis.
Reconstructing 3D volumes from a series of 2D sections using the alignment method.
Using the method as a calibration tool for precision slicing or cutting machines on realistic samples.
Interested in licensing this patent?