Synthesis singleplex from multiplex brightfield imaging using generative adversarial network

Inventors

Kim, Jungwon • Lorsakul, Auranuch • Nie, Yao • Wang, Xingwei • Zhao, Zuo

Assignees

Ventana Medical Systems Inc

Abstract

A multiplex image is accessed that depicts a particular slice of a particular sample stained with two or more dyes. Using a Generator network, a predicted singleplex image is generated that depicts the particular slice of the particular sample stained with each of the expressing biomarkers. The Generator network may have been trained by training a machine-learning model using a set of training multiplex images and a set of training singleplex images. Each of the set of training multiplex images depicted a slice of a sample stained with two or more dyes. Each of the set of training singleplex images depicted a slice of a sample stained with a single dye. The machine-learning model included a Discriminator network configured to discriminate whether a given image was generated by the Generator network or was a singleplex image of a real slide. The method further includes outputs the predicted singleplex image.

Core Innovation

The invention generates “synthesis/synthetic singleplex” images from multiplex brightfield IHC by accessing a multiplex image that depicts a particular slice of a particular sample stained with two or more dyes, then generating with a Generator network a predicted singleplex image that depicts the particular slice stained with only one of the dyes. The Generator network is trained using a set of training multiplex images and a set of training singleplex images, with each training multiplex image showing a slice stained with two or more dyes and each training singleplex image showing a slice stained with a single dye.

In one training multiplex set, an unmixed image associated with a type of stain represents the particular sample stained with only one of the dyes. The unmixed image is decomposed from the multiplex image using a color deconvolution algorithm selected based on a context in which the multiplex image was obtained, and the unmixed image is perturbated to generate a remixed image for the context.

The machine-learning model trained on the remixed image includes a Discriminator network configured to discriminate whether a given image was generated by the Generator network or was a singleplex image of a real slide. The approach is described as supporting GAN-based Generator/discriminator training using named architectures such as Pix2Pix and Pix2PixHD, and alternative architectures including CycleGAN and BicycleGAN with cycle-consistency and discriminator losses.

Training singleplex targets may be synthetic via context-selected color deconvolution followed by perturbation/remixing, or may be real adjacent-slide singleplex images not present in multiplex training. Robustness across scanners and sites is described as achieved through learned or fixed Generator parameter values used consistently during prediction.

Claims Coverage

Independent claim coverage comprises three independent claims: a computer-implemented method, a system, and a computer-program product. Across these independent claims, the inventive coverage centers on four inventive features: accessing multiplex stained slices, training a Generator network to predict single-dye singleplex images from multiplex images, context-selected color deconvolution with perturbated remixed images for unmixed training data, and a Discriminator network configured to discriminate generated images versus real singleplex images.

Across the independent claims, the core coverage is a computer-implemented method, system, and computer-program product that uses a trained Generator network with a Discriminator to generate a predicted singleplex image stained with only one dye from an input multiplex-stained slice, where unmixed training data are created via context-selected color deconvolution and perturbated remixed images.

Stated Advantages

Documented Applications