Computer-implemented method for performing hierarchical classification
Inventors
Beskales, George • Kraemer, John • Ilyas, Ihab F. • Cleary, Liam • Roome, Paul
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-11782966-B1
Publication Date
2023-10-10
Expiration Date
Abstract
Given a number of records and a number of target classes to which these records belong to, a (weakly) supervised machine learning classification method leverages known possibly dirty classification rules, efficiently and accurately learns a classification model from training data, and applies the learned model to the data records to predict their classes.
Core Innovation
The disclosed invention provides a computer-implemented hierarchical classification system for large-scale record-to-class labeling using a hierarchy of classes represented as a tree/DAG. It uses a collection of classifiers that predict, as represented by a predicted score, whether a data record is a member of one of the classes in the hierarchy of classes or any of its descendants, thereby producing a predicted score for a plurality of classes for each data record.
For hierarchical prediction, the document describes computing conditional probabilities using a chain rule and using a specificity-weighted scoring function. It further describes using A* search to find the top-scoring subclass in the hierarchy. The disclosed training approach uses positive examples from a class and its descendants and negatives from sibling-descendant regions to support learning across the hierarchy.
A key component of the invention is an entropy-driven active learning loop for training data expansion. For each data record, the system computes an entropy of the predicted scores, and selects a weighted random sample for labeling where the weight is a function of the computed entropy so that the selected records are high impact questions. Labeled high impact questions are then combined with the training data to build an expanded training set and retrain the hierarchical classification model using the collection of classifiers.
Claims Coverage
The partial content identifies one independent claim. The independent claim focuses on training a hierarchical classification model using per-class membership predicted scores, entropy computation per record, and entropy-weighted sampling of high impact questions for operator labeling, followed by retraining of the hierarchical classifiers.
Entropy-driven weighted sampling for high impact question labeling in hierarchical training
Predicts per-record predicted scores for membership in classes or descendants using a collection of classifiers, computes an entropy of the predicted scores for each data record, and selects a weighted random sample of data records for labeling where the weight is a function of the computed entropy, and the selected data records are high impact questions.
Operator labeling from the class hierarchy and training data expansion
Presents each data record selected for labeling to an operator for the operator to label with the correct class from the hierarchy of classes, and combines the labeled high impact questions with the training data thereby expanding the training data for model training.
Retraining hierarchical membership classifiers using expanded training data
Builds, using the expanded training data, the collection of classifiers each of which predicts, as represented by a predicted score, whether a data record is a member of one of the classes in the hierarchy of classes or any of its descendants, thereby training the hierarchical classification model.
Across the identified independent claim, training is driven by per-record class-membership predicted scores, entropy-based determination of labeling impact via a function of computed entropy, operator labeling of high impact questions using the hierarchy of classes, and retraining hierarchical classifiers on the expanded training data to predict membership in classes and descendants.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Not explicitly described in patent.
Interested in licensing this patent?