Unsupervised validation framework for large language model (LLM) outputs

Inventors

Subber, Waad • Singh, Ankit • HARTYE, Eric

Assignees

Honeywell International Inc

Abstract

The disclosure relates to a system and method for unsupervised validation of Large Language Model (LLM)-generated outputs. The method includes the steps of receiving input data, which may comprise structured or unstructured text; generating an LLM output based on the input data; extracting a first set of topics from the input data, with each topic represented by a set of keywords; converting the first set of topics into human-readable reference data; extracting a second set of topics from the LLM output; and converting the second set of topics into human-readable candidate data. The method further includes comparing the reference data with the candidate data using one or more performance metrics. Finally, the method determines a validation score based on the comparison results. The disclosed method allows for the objective validation of LLM-generated content, enhancing accuracy and efficiency in diverse applications such as customer service, technical documentation, and legal analysis.

Core Innovation

A method for unsupervised validation of a Large Language Model (LLM) generated output receives input data representing a set of information to be processed by a primary Large Language Model (LLM1) and generates LLM output by the LLM1 based on the input data. The method extracts a first set of topics from the input data, wherein each topic is represented by a set of keywords, and converts the first set of topics into human-readable data by a second Large Language Model (LLM2) to generate reference data representing key topics.

The method extracts a second set of topics from the generated LLM output and converts the second set of topics into human-readable data by the LLM2 to generate candidate data representing key topics. The reference data is compared with the candidate data using one or more performance metrics, and a validation score is determined based on the comparison of the reference data and the candidate data.

The method automatically performs a control action comprising releasing the generated LLM output when the validation score meets or exceeds a predefined threshold and routing the generated LLM output for manual review when the validation score is below the predefined threshold. The disclosed framework also includes determining an optimal number of topics based on computation of a coherence score, extracting topics using a topic modeling algorithm selected from Latent Dirichlet Allocation (LDA), Non-negative Matrix Factorization (NMF), and Latent Semantic Analysis (LSA), and generating a confidence score based on the validation score to represent probability that the LLM-generated output meets a predefined quality standard.

Claims Coverage

The independent claims cover an unsupervised validation workflow and corresponding system and computer-readable medium, each grounded in topic extraction from both the input and the LLM output, human-readable conversion of topics using a second LLM, and comparison using one or more performance metrics to compute a validation score that drives an automated control action. The claims include seven inventive features.

Overall, the independent claims center on unsupervised validation by deriving reference key topics from the input and candidate key topics from the generated LLM output via topic extraction and LLM2-based conversion to human-readable data, comparing them with one or more performance metrics to compute a validation score, and using that score to release the generated output or route it for manual review below a predefined threshold. Additional refinements include specific metric types, named topic modeling algorithms, coherence-score-based topic-count selection, confidence scoring from the validation score, and constraints on input/output forms.

Stated Advantages

Documented Applications