Contextualized human machine systems and methods of use
Inventors
Assignees
Publication Number
US-12242981-B1
Publication Date
2025-03-04
Expiration Date
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
A contextualized human machine system is provided comprising a context engine and a user interface configured to communicate a recommended data to a user. In some embodiments, the context engine selects the recommended data based on an activity of the user. In some embodiments, the input of the user comprises a chat stream of the user. In some embodiments, the recommended data comprises one of a video product, a hyperlink to information or a suggestion for annotating a product. In some embodiments, the context engine is configured to represent user activity, content, mission and actor as nodes in a multi-layer knowledge graph.
Core Innovation
A contextualized human machine system comprising a context platform configured to receive input data and to define property values of nodes in a multi-layer knowledge graph, and a recommendation engine configured to execute a recommendation algorithm to automatically determine a context-aware recommendation of a third node based on a connection strength measure and a similarity measure. The system represents user activity, content, mission and actor as nodes in a multi-layer knowledge graph and can communicate recommended data to a user, where recommended data may include video products, hyperlinks to information, or suggestions for annotating a product. The context platform can automatically define first and second property values for node pairings and define relationship property values between nodes, and the recommendation engine uses graph traversal methods to identify and rank potential recommendations.
Data analysts face an increased need to access and analyze large volumes of disparate data in a timely and efficient manner, including text data, visual data, video data, prior work products, work products of others and mission/task data, creating an urgent need to redesign tools to balance automation and human-in-the-loop expertise for collaborative and interactive data analysis. The disclosed contextualized human machine systems address this problem by ingesting multimodal human-machine interface/interaction inputs, populating a multi-layer knowledge graph according to a pre-defined domain model (including a synonymy layer for translation), and producing context-aware automations and augmentations to improve information management, product generation, and interaction between analysts and the technologies they use.
Claims Coverage
Two independent claims are identified. The main inventive features from each independent claim are listed below.
Context platform receives input data from an analyst workstation
a context platform configured to receive an input data from an analyst workstation;
Multi-layer knowledge graph node property definition
the context platform configured to automatically define, from the input data, a first property value of a first node corresponding to a multi-layer knowledge graph; the context platform configured to define a second property value of a second node of the multi-layer knowledge graph;
Content node comprises a work product
the second node comprising a content node comprising a work product;
Recommendation engine determines context-aware recommendation
a recommendation engine configured to execute a recommendation algorithm to automatically determine a context-aware recommendation of a third node based on a connection strength measure and a similarity measure;
Communicating augmented work product to the analyst workstation
communicating the second node and the third node to the analyst workstation as an augmented work product;
Graph traversal algorithm configured with steps a through h
wherein the recommendation algorithm comprises a graph traversal algorithm configured to: (a) identify one or more additional node pairing of the first node connected by any relationship type to another node in a graph layer of the multi-layered knowledge graph; (b) calculate a connection strength measure of the relationship type for each additional node pairing and associate the connection strength measure to each of the nodes in the additional node pairing; (c) calculate a similarity measure of the nodes in each additional node pairing and associate the similarity measure to each of the nodes in the additional node pairing; (d) iterate steps (a)-(c) for a next step out of the graph layer for subsequent node pairings connected by any relationships type until a threshold traversal depth of steps; (e) define each of the nodes in the each of the additional node pairings and the subsequent node pairings as a plurality of related nodes; (f) filter the plurality of related nodes to define a plurality of filtered nodes as a plurality of potential recommendations; (g) determine a weighted value of each of the plurality of filtered nodes as a function of the connection strength measure and the similarity measure; and (h) select the filtered node with the greatest weighted value as the context-aware recommendation.
Processor-based receiving of input data from an analyst workstation
receiving, with a processor, an input data from an analyst workstation in the processor-based human machine system;
Automatic definition of activity and content property values
defining, with the processor automatically from the input data, an activity property value of an activity node corresponding to a multi-layer knowledge graph; defining, with the processor, a content property value of a content node of the multi-layer knowledge graph;
Defining relationship property value between content and activity nodes
defining, with the processor, a relationship property value of a relationship type between the content node and the activity node;
Processor executes recommendation algorithm and communicates augmented product
executing, with the processor, a recommendation algorithm to automatically determine a context-aware recommendation for a second activity node or a second content node based on a connection strength measure and a similarity measure; communicating, with the processor, the content node and one of the second activity node or the second content node to the analyst workstation as an augmented work product;
Processor-based graph traversal algorithm with steps a through h
the recommendation algorithm comprises a graph traversal algorithm configured to execute, with the processor, the method of: (a) identifying one or more additional node pairing of the activity node connected by any relationship type to another node in a graph layer of the multi-layered knowledge graph; (b) calculating a connection strength measure of the relationship type for each additional node pairing and associate the connection strength measure to each of the nodes in the additional node pairing; (c) calculating a similarity measure of the nodes in each additional node pairing and associate the similarity measure to each of the nodes in the additional node pairing; (d) iterating steps (a)-(c) for a next step out of the graph layer for subsequent node pairs of nodes connected by any relationships type until a threshold traversal depth of steps; (e) defining each of the nodes in the each of the additional node pairings and the subsequent node pairings as a plurality of related nodes; (f) filtering the plurality of related nodes to define a plurality of filtered nodes as a plurality of potential recommendations; (g) determining a weighted value of each of the plurality of filtered nodes as a function of the connection strength measure and the similarity measure; and (h) selecting the filtered node with the greatest weighted value as the context-aware recommendation.
The independent claims disclose a context platform that ingests analyst input to populate a multi-layer knowledge graph (including activity, content, actor, and mission nodes), and a recommendation engine that uses a graph-traversal recommendation algorithm—weighing connection strength and similarity measures—to select and communicate context-aware recommendations and augmented work products to an analyst workstation.
Stated Advantages
Improve information management and interaction between analysts and the technologies they use.
Enable analysts to more efficiently produce intelligence products tailored to the needs of their supported unit.
Provide proactive support with partially automated product generation and recommendations of relevant information.
Enable dynamic and interactive work products that maintain links between data from processing to dissemination and provide additional contextual information on-demand.
Reduce cognitive load and reduce the time and effort it takes to navigate between tasks, improving efficiency.
Increase responsiveness by using automation to assemble intelligence products and deliver a greater number of high-quality products in a given amount of time.
Documented Applications
Intelligence, Surveillance, and Reconnaissance (ISR) analysts and the PED (Processing, Exploitation, and Dissemination) process.
Analysts of data for financial, law enforcement, security, weather, logistics, monitoring and educational applications.
Full-motion video (FMV) analyst use cases, including creation of stills and dynamic stop intelligence products (DSIPs).
Distributed monitoring or command and control systems.
Automated product generation and augmentation workflows such as automatically generating video products, links to related information, and product annotation suggestions based on chat streams.
Interested in licensing this patent?