Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-9996670-B2
Publication Date
2018-06-12
Expiration Date
Abstract
Clinical content analytics engines and associated processes are described. An engine receives a clinical decision support document, accesses corresponding reference content, identifies and extracts medical intervention content from the clinical decision support document, segments extracted medical intervention content into a first plurality of segments including at least a first segment comprising a first set of text, determines if the first segment corresponds to at least a first item included in the reference content, the first item comprising a second set of text comprising terminology different than that found in the first set of text, and in response to determining that the first segment corresponds to the first item included in the reference content, causing a report to include an indication that the first segment corresponds to the first item included in the reference content.
Core Innovation
A computer-implemented method and analytics system improve content analyzer system accuracy for identifying CDS document deficiencies and consistencies with respect to reference content, where the reference content comprises clinical guidelines. The system repeatedly trains a machine learning module based on new incoming data and uses the trained electronic model to determine whether content is relevant and matching or non-relevant with respect to the reference content.
The machine learning module automatically determines which content features to use and constructs or modifies an electronic model accordingly, including features comprising one or more of text length, presence of a medication term, medical intervention language, use of a negation, or context. The method repeatedly improves accuracy by collecting positive and negative cases from CDS documents and training the electronic model using the collected positive and negative cases.
The trained model takes as input a text segment extracted from a CDS document and returns a likelihood that the text segment matches a reference checklist item, where new incoming data indicates whether the likelihood is correct or incorrect. The system receives a clinical decision support document, accesses reference content corresponding at least in part to the clinical decision support document, and uses the electronic model to identify and extract medical intervention content from the clinical decision support document.
The system segments at least a portion of the extracted medical intervention content into a first plurality of segments, evaluates each given segment to identify a core concept, determines whether a negation is associated with the medical intervention, and determines whether a given segment corresponds to an item included in the reference content based on terminology differences. At least partly in response to the correspondence results, the system generates a version of the CDS document and/or a report including a visual indication of matched items and visual indication of missing items.
Claims Coverage
The document includes three independent claims. Across these claims, the core coverage includes repeatedly training a machine learning module using new incoming data to output likelihoods for matching to reference checklist items, automatically selecting content features for relevant versus non-relevant designation, extracting and segmenting medical intervention content with core concept and negation evaluation, and generating a report or a version of the CDS document with visual indications for matched and missing reference items.
Repeated machine learning training with positive and negative cases for likelihood-based checklist matching
repeatedly training a machine learning module based on new incoming data by collecting positive and negative cases from CDS documents, training an electronic model using the collected positive and negative cases, taking as input a text segment extracted from a CDS document and returning a likelihood that the text segment matches a reference checklist item, and where the new incoming data indicates whether the likelihood that the text segment matches the reference checklist item is correct or incorrect
Automatic content feature selection and electronic model construction for relevant versus non-relevant designation
automatically determining which content features are to be used to determine whether content is to be designated as relevant and matching to the reference content and which content is to be designated as non-relevant and to construct or modify an electronic model accordingly, wherein the features comprise one or more of text length, presence of a medication term, medical intervention language, use of a negation, or context
Electronic model extraction and segmentation of medical intervention content into segments with core concept evaluation
using the electronic model to identify and extract medical intervention content from the clinical decision support document, and segmenting at least a portion of the extracted medical intervention content into a first plurality of segments, wherein a given segment in the first plurality of segments is evaluated to identify a core concept, and identifying the core concept further comprises determining whether a given segment includes a plurality of medical interventions and determining which of the plurality of medical interventions are part of the core concept and which are not part of the core concept
Negation association with core concept and correspondence to reference items with terminology differences
if the core concept of the given segment comprises at least one medical intervention, determining whether a negation is associated with the at least one medical intervention, and determining if the first segment corresponds to at least a first item included in the reference content, the first item comprising terminology not present in the first and second sets of text
Visual indication reporting for matched and missing reference items
causing a version of the clinical decision support document to be generated to include a visual indication that the first segment corresponds to the first item included in the reference content, determining whether a second item included in the reference content corresponds to at least one of the first plurality of segments, and causing the version of the clinical decision support document to include a visual indication that the first plurality of segments fails to include at least one segment that corresponds to the second item included in the reference content, or causing the version of the clinical decision support document to include a visual indication that the second item corresponds to at least one segment in the first plurality of segments
Across the independent claims, the invention centers on repeatedly training a machine learning module to assess likelihood that extracted CDS text segments match reference checklist items, using automatically selected content features including medication terms, medical intervention language, negation, and context. The system extracts and segments medical intervention content, identifies a core concept and negation association, evaluates correspondence to reference items with terminology differences, and produces a report and/or a version of the CDS document with visual indications of matched and missing checklist items.
Stated Advantages
Improves content analyzer system accuracy in identifying CDS document deficiencies and consistencies with respect to reference content comprising clinical guidelines.
Generates a version of the clinical decision support document and/or a report including visual indication of matched reference items and missing reference items.
Documented Applications
Analyzing a clinical decision support document from a medical service provider system against reference content comprising clinical guidelines to identify deficiencies and consistencies, including correspondence to reference checklist items.
Generating a version of the clinical decision support document and/or report that visually indicates which segments correspond to reference items and which items are not included in the segments.
Interested in licensing this patent?