Systems and methods for detecting errors and hallucinations in generative model output data

Inventors

EMREY, Abigail • AMICI, Olivia • BARTLOW, Nicholas

Assignees

Noblis Inc

Abstract

Provided herein are systems and methods that can detect errors and hallucinations in output data produced by generative models. The detection systems described herein may compare information from the generative model output data with ground truth information to determine whether the generative model output data comprises errors and/or hallucinations. The systems and methods described herein may generate an output indicative of whether the generative model output data comprises errors and/or hallucinations. The systems and methods described herein can readily detect false information generated by NLG systems, thus potential harms of reliance on false information can be minimized.

Core Innovation

The invention detects errors and hallucinations in generative model output data by receiving output data from a generative model and generating a first data structure based on the generative model output data. The first data structure comprises a first graph including a first plurality of nodes and a first plurality of edges, thereby providing a structured representation of the generative model output data.

The invention compares the first data structure to a second data structure that represents fact data. The comparison determines whether the generative model output data comprises one or more of an error and a hallucination, linking the determination to the structured representation of fact data.

Based on the determination that the generative model output data comprises the one or more of an error and a hallucination, the invention generates an output indicating the determination. The document further describes generating structured representations for comparison, including graph-based representations of fact data.

Claims Coverage

The document includes three independent claims covering a system, a method, and a non-transitory computer-readable storage medium. Across the independent claims, the inventive features are the graph-structured representation of generative model output, the comparison to a fact-data structure, and the generation of an output indicating whether errors and/or hallucinations are present.

The independent claims cover detecting errors and hallucinations by generating a graph-based first data structure from generative model output data, comparing it to a second data structure representing fact data, and generating an output indicating whether an error and/or a hallucination is determined to be present.

Stated Advantages

Documented Applications

No documented applications found