Artificial intelligence-based systems and methods for providing personalized skin product recommendations

Inventors

DISSANAYAKE, Dissanayake Mudiyanselage Mahathma Bandara

Assignees

Procter and Gamble Co

Abstract

Artificial intelligence-based systems and methods are described for providing personalized skin product recommendations. Natural language data of a user is received using a conversation engine, such as a generative pretrained transformer AI model. The natural language data is input into a synthetic image generation model for generating a digital twin image depicting skin condition(s). A product recommendation model inputs the digital twin image and/or phenotype and/or demographic classifications of the user, and outputs a skin care product recommendation for the user. An image simulation model inputs the product recommendation, the digital twin image and phenotype and/or demographic classifications of the user to output a simulated image of the user having one or more graphical enhancements. The simulated image may be displayed to the user on display screen along with natural language data describing the simulated image as depicted skin as predicted to appear following treatment with the skin care product.

Core Innovation

The invention describes an AI-based system and method for personalized skin product recommendations. The system receives user input as natural language data via a conversation engine that includes a natural language model, and it generates phenotype classifications and demographic classifications based on the user natural language data. The phenotype classifications can include skin concerns and skin characteristics, and the demographic classifications can include attributes such as gender, age, and ethnicity.

Using the phenotype classifications and demographic classifications, the system generates user-specific photorealistic digital twin images. The digital twin images are generated without requiring user photorealistic image transmission, and the system can generate images according to one or more image color formats, such as L*a*b* and RGB color formats. The system then uses product recommendation models to generate personalized skin product recommendations for the user based on the digital twin images and the phenotype and demographic classifications.

The invention further uses an image simulation model to output simulated and graphically enhanced images showing predicted treatment effects. The system can render the simulated images and display them on a display screen or in an app, and it can provide natural-language explanations corresponding to the simulated images. A model training pipeline is described with a training phase and a multi-model execution phase to generate the simulated enhanced images and the product recommendations, and the system includes security and performance advantages associated with not transmitting user images or PII.

Claims Coverage

Not explicitly described in patent.

Not explicitly described in patent.

Stated Advantages

Documented Applications