Conversational assistant for conversational engagement
Inventors
Kalns, Edgar T. • Vergyri, Dimitra • ACHARYA, GIRISH • Kathol, Andreas • Almada, Leonor • Kim, Hyong-Gyun • BASIOU, Nikoletta • Wessel, Michael • Spaulding, Aaron • Heusser, Roland • Carpenter, James F. • Yin, Min
Assignees
Publication Number
US-12367869-B2
Publication Date
2025-07-22
Expiration Date
2040-06-15
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
A conversational assistant for conversational engagement platform can contain various modules including a user-model augmentation module, a dialogue management module, and a user-state analysis input/output module. The dialogue management module receives metrics tied to a user from the other modules to understand a current topic and a user's emotions regarding the current topic from the user-state analysis input/output module and then adapts dialogue from the dialogue management module to the user based on dialogue rules factoring in these different metrics. The dialogue rules also factors in both i) a duration of a conversational engagement with the user and ii) an attempt to maintain a positive experience for the user with the conversational engagement. A flexible ontology relationship representation about the user is built and stores learned metrics about the user over time with each conversational engagement, and then in combination with the dialogue rules, drives the conversations with the user.
Core Innovation
The invention is a conversational assistant for conversational engagement implemented as a conversational engagement microservice platform composed of various modules including a user-model augmentation module, a dialogue management module, and a user-state analysis input/output module. The dialogue management module receives metrics tied to a user from other modules to understand the current topic and the user's emotions regarding that topic and adapts the dialogue based on dialogue rules factoring these metrics. These rules also consider the duration of the conversational engagement and aim to maintain a positive experience for the user during the conversation.
The user-model augmentation module builds and maintains a flexible ontology relationship representation about the user, capturing learned metrics over time from multiple engagements. This user model includes social relationships, emotional state, health indicators, and interests, among other topics. The dialogue management module cooperates with this user model and the dialogue rules, which are codified in a dialogue specification language, to dynamically adapt conversation flow between free-form casual dialogue and directed dialogue based on conversational context and user state analysis.
The problem being solved addresses the limitations of traditional virtual personal assistants that are goal-oriented, limited to short queries and commands, and lack the ability to maintain free-flowing conversational engagement or build a user model. Current assistants do not effectively engage users in general human-interest topics or track emotional and health-related cues, which is particularly vital for seniors facing social isolation and health concerns. There is an unmet need for a system that facilitates extended, natural conversational engagement, captures valuable insights from dialogue, and supports positive user experiences through adaptive conversational strategies.
Claims Coverage
The patent contains one independent device claim and one independent method claim, which encompass inventive features related to the conversational assistant's modular architecture, dynamic dialogue adaptation, user modeling, and health monitoring.
Conversational engagement platform architecture and dialogue adaptation
A conversational engagement microservice platform containing modules for user-model augmentation, dialogue management, and user-state analysis. The dialogue management module receives user-related metrics to understand conversation topics and user emotions, and adapts the dialogue using a set of dialogue rules considering interaction duration and attempts to maintain a positive user experience. The platform dynamically adapts between casual and directed dialogue without requiring advance notice of conversation topics.
User-model augmentation with flexible ontology
Detecting and tracking user-specific knowledge on multiple topics including family, concerns, emotional state, health indicators, and interests, storing this information in long-term memory within a flexible ontology relationship representation tailored to the user. Nodes in this ontology link to stored facts and are referenced by the dialogue management module to generate context-aware questions and responses.
Dialogue specification language for context-aware conversation modeling
Use of a domain-specific dialogue specification language that enables expressive context-aware modeling of conversational content, including rules and branching structures to decide when to maintain or transition topics while reducing code size and memory usage. This language supports dynamic adaptation of dialogue based on current conversational context and user state.
Contextual dialogue generation from conversation history and learned data
Once a topic is selected, the dialogue management module uses dialogue rules to review the entire current conversation, stored prior conversations on the topic, and linked topic information as weighted factors to guide generating the next question or response using dialogue templates.
User-state analysis integration for conversation steering
The user-state analysis input/output module cooperates with dialogue management to adapt conversation topics toward points the user is amenable to or positive about and steer away from topics where negative sentiment or emotion is detected, maintaining a positive engagement experience.
Topic understanding via hierarchical classifiers and co-clustering
An information-extraction and topic-understanding module cooperates with dialogue management, detecting and tracking topic IDs from hierarchical classifiers and co-clustering of related topics to correctly identify multiple discussed topics in free-form conversations.
Health monitoring and conversational adaptation
An audibly detectable health input/output module monitors dialogues to assess the user's current health state, referencing prior health history including coughs, and cooperates with dialogue management to decide when to shift conversation topics to address health concerns.
Text-to-speech and speech recognition integration with short-term memory
A text-to-speech input/output module interfacing with speech-to-text and automatic speech recognition to process user's audio input and generate audio responses. The system uses short-term memory storing entire conversations to analyze utterances in context for decisions on dialogue progression, topic changes, and response generation.
The independent claims collectively cover a conversational assistant platform integrating modular user modeling, dynamic dialogue management driven by context-aware rules, user emotion and health state analysis, and advanced topic understanding to facilitate extended, positive conversational engagement adaptable to user state and interaction history.
Stated Advantages
Supports almost human-like dialogue leveraging multi-modal inputs such as emotion and polarity.
Systematically captures and stores detailed user information over time enabling personalized and contextually relevant conversations.
Dynamically adapts conversations to maintain user engagement by considering duration and positive experience factors.
Reduces dialogue specification code size and memory consumption through a domain-specific dialogue specification language.
Enables unobtrusive health monitoring through audio cues and adjusts dialogues based on detected health conditions.
Facilitates rapid and expressive context-aware conversation modeling accessible to end users and developers.
Documented Applications
Engaging users, particularly seniors, in free-form conversational interaction about general human-interest topics such as family, friends, and health concerns.
Providing caregivers with valuable health insights derived from unobtrusive monitoring during conversations.
Addressing social isolation issues among seniors through prolonged, natural conversational engagements.
Unobtrusive auditory health monitoring focusing on respiratory conditions like coughs during conversation sessions.
Interested in licensing this patent?