Techniques for determining tissue characteristics using multiplexed immunofluorescence imaging

Inventors

Svekolkin, Viktor • Galkin, Ilia • Postovalova, Ekaterina • Ataullakhanov, Ravshan • Bagaev, Alexander • Varlamova, Arina • Ovcharov, Pavel

Assignees

BostonGene Corp

Abstract

Techniques for processing multiplexed immunofluorescence (MxIF) images. The techniques include obtaining at least one MxIF image of a same tissue sample, obtaining information indicative of locations of cells in the at least one MxIF image, identifying multiple groups of cells in the at least one MxIF image at least in part by determining feature values for at least some of the cells using the at least one MxIF image and the information indicative of locations of the at least some cells in the at least one MxIF image and grouping the at least some of the cells into the multiple groups using the determined feature values, and determining at least one characteristic of the tissue sample using the multiple cell groups.

Core Innovation

A method uses at least one computer hardware processor to obtain at least one multiplexed immunofluorescence (MxIF) image of a tissue sample, where the at least one MxIF image comprises a plurality of channels associated with respective markers in a plurality of markers. The method obtains, using a machine learning technique, information indicative of locations of cells in the at least one MxIF image and identifies multiple groups of cells at least in part by identifying pixel values for at least some of the cells using the at least one MxIF image and the information indicative of locations of cells.

The method determines, using a neural network, marker expression signatures for the at least some of the cells at least in part by using the identified pixel values. Each marker expression signature for a particular cell includes, for each particular marker of one or more of the plurality of markers, a respective likelihood output by the neural network that the particular marker is expressed in the particular cell. The method groups the at least some of the cells into the multiple groups using the marker expression signatures, where the grouping uses the respective likelihood output by the neural network for each particular marker of the one or more of the plurality of markers.

In one claim, the information indicative of locations of cells includes information indicating cell boundaries, and the grouping is performed at least in part by clustering using a graph neural network different from the neural network. The clustering comprises grouping the at least some of the cells by identifying pixels within a particular cell's boundary, calculating at least one feature value using the identified pixel intensity values for the pixels identified within the particular cell's boundary, and determining similarities among the calculated feature values.

The method determines at least one characteristic of the tissue sample using the multiple groups, including determining information about cell types in the tissue sample, and/or determining cell masks, and/or determining spatial distribution of the cell types.

Claims Coverage

The partial content provides three independent claims (clm-00001, clm-00022, clm-00030). Across these claims, there are six inventive features centered on MxIF image acquisition, machine learning-based cell location information, neural-network marker expression signatures with likelihood outputs, cell grouping into multiple groups, graph-neural-network-based clustering using cell boundaries and pixel-defined features, and determining tissue characteristics from the resulting groups.

Across clm-00001 and clm-00022, the core claim chain is: obtain MxIF images, obtain cell locations via a machine learning technique, identify pixel values for cells, compute marker expression signatures via a neural network that outputs marker-specific likelihoods, group cells into multiple groups using those likelihood outputs, and determine tissue characteristics from the multiple groups. Across clm-00030, cell-location estimation uses a U-Net or region-based convolutional neural network with at least one million parameters, and clustering is performed using a graph neural network different from the neural network, using boundary-defined pixel intensity feature values and similarities, followed by tissue characteristic determination including cell types, cell masks, and/or spatial distribution.

Stated Advantages

Documented Applications