Machine learning extraction of clinical variable values for subjects from clinical record data

Inventors

Wittmershaus, Brett • Amster, Guy • Waskom, Michael • Roher, Natalie • Singh, Nisha • Phadke, Sharang • Shapiro, Will

Assignees

Flatiron Health Inc

Abstract

Described herein are techniques of using machine learning to automatically extract clinical variable values for subjects from clinical record data. The techniques designate certain clinical variables as hybrid variables that can be assigned values by machine learning model prediction. The techniques process, using a machine learning model trained to predict a value of a hybrid variable, clinical record data associated with a subject to obtain a predicted hybrid variable value and an associated confidence score. The techniques set the value of the hybrid variable for the subject to the predicted hybrid variable value when the model prediction is of sufficiently high confidence.

Core Innovation

The invention uses machine learning to automatically extract values of variables for a plurality of subjects from clinical record data. The approach generates a dataset storing values of the variables for the plurality of subjects, where the variables include a first variable designated as a hybrid variable configured for assignment by performance of machine learning prediction and a second variable configured for assignment by manual extraction without performance of machine learning prediction.

For each subject, the hybrid first variable is set at least in part by processing the clinical record data with an ML model trained to predict the value of the hybrid first variable. The ML model provides an ML predicted value of the first variable and an associated confidence score, and the system determines using the confidence score whether to set the value of the first variable for the subject to the ML predicted value. When the determination indicates setting, the dataset is updated with the ML predicted value; when the determination indicates not setting, a manually extracted value of the first variable is obtained and the dataset is updated with the manually extracted value.

Values of the second variable for the plurality of subjects are set in the dataset to manually extracted values without obtaining ML predicted values of the second variable. The system and method incorporate clinical record processing for ML inference on unstructured clinical textual data into features for ML prediction, and include user-facing graphical user interface behaviors that present predicted values and receive user input associated with manual extraction and acceptance.

Claims Coverage

The independent claims, including a method, a system, and a non-transitory computer-readable medium, cover using machine learning to extract variable values from clinical record data for a plurality of subjects by generating a dataset with a hybrid first variable and a manually extracted second variable. Across the independent claims, there are three main inventive features: a hybrid variable assigned by confidence-based ML prediction versus manual extraction, a confidence score used to decide whether the ML predicted value is set, and a second variable assigned only by manual extraction without ML prediction.

Across the independent claims, the inventive concept is implemented by generating a dataset for a plurality of subjects from clinical record data that includes a hybrid first variable and a second variable. The hybrid first variable is assigned using an ML predicted value accompanied by a confidence score, with a determination that selects either the ML predicted value or a manually extracted value. The second variable is always filled using manual extraction without ML prediction.

Stated Advantages

Documented Applications