Situational recommendations and control
Inventors
Sethu, Ramesh • ADITHTHAN, ARUN • Islam, Md Mhafuzul • PERANANDAM, PRAKASH M. • Zolfagharian, Amirhossein • Aghababaeyan, Zohreh
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12459359-B2
Publication Date
2025-11-04
Expiration Date
Abstract
A system for providing situational recommendations within a vehicle includes a system controller in communication with a plurality of onboard sensors, the plurality of onboard sensors adapted to collect real-time data related to a location of the vehicle and operating conditions of the vehicle, a database in communication with the system controller adapted to store data related to past actions and data related to a location of the vehicle and operating conditions of the vehicle when such past actions occurred, the system controller including a driver specific machine learning model adapted to predict a desired action based on the real-time data related to the location and operating conditions of the vehicle and data from the database, the system controller further adapted to initiate the predicted desired action, receive input from an occupant within the vehicle, and update the driver specific machine learning model.
Core Innovation
The invention provides situational recommendations within a vehicle by collecting real-time data related to a location of the vehicle and operating conditions of the vehicle using a plurality of onboard sensors in communication with a system controller. The system controller accesses a database of stored data related to past actions and to the location of the vehicle and the operating conditions of the vehicle when those past actions occurred. Based on the real-time data and the stored past-action/location/condition data, the system predicts a desired action using a driver specific machine learning model within the system controller.
The driver specific machine learning model identifies patterns of behavior indicating that the occupant takes a specific action each time the occupant arrives at a specific location and/or each time a specific condition exists. The machine learning model probabilistically predicts the desired action based on the identified pattern of behavior. The system controller initiates the predicted desired action by prompting the occupant within the vehicle with a recommendation when the probability exceeds fifty percent, and by automatically initiating the predicted desired action when the probability exceeds ninety percent.
After initiating or prompting, the system receives input from an occupant within the vehicle and updates the driver specific machine learning model. The system controller receives, via communication between the system controller and onboard systems within the vehicle, data related to an action being taken by the occupant within the vehicle, and compares the action being taken to the predicted desired action. If the action does not match the predicted desired action, the system prompts the occupant to verify proceeding, and if the action is identified as an anomaly or as an inherently unsafe action, the system prompts verification or prompts a warning message. The system supports anomaly detection and inherently unsafe action handling as part of the comparison and prompting logic.
Claims Coverage
The independent claims are method claim 1, system claim 7, and vehicle claim 13. Across these independent claims, the inventive features focus on driver-specific probabilistic prediction from real-time onboard sensor data and stored past actions, probability-threshold prompting versus automatic initiation, occupant input receipt and model updating with mismatch/anomaly/unsafe-action handling, and, in claim 13, specific HMI modalities for recommendation and input.
Real-time sensor data and past-action/location/condition database
Collecting, with a plurality of onboard sensors in communication with a system controller, real-time data related to a location of the vehicle and operating conditions of the vehicle; accessing, with the system controller, a database of stored data related to past actions and data related to the location of the vehicle and operating conditions of the vehicle when such past actions occurred.
Driver-specific machine learning model for probabilistic desired-action prediction
Predicting, with a driver specific machine learning model within the system controller, a desired action based on the real-time data related to the location of the vehicle and the operating conditions of the vehicle and data from the database, including identifying a pattern of behavior indicating that the occupant takes a specific action each time the occupant arrives at a specific location and/or identifying a pattern of behavior indicating that the occupant takes a specific action each time a specific condition exists; and probabilistically predicting, with the machine learning model, the desired action based on the identified pattern of behavior.
Probability-threshold prompting and automatic initiation of predicted action
Initiating the predicted desired action, including when a probability of the desired action exceeds fifty percent, prompting the occupant within the vehicle with a recommendation for the predicted desired action; and when the probability of the desired action exceeds ninety percent, automatically initiating the predicted desired action.
Occupant input, action comparison, and driver-specific model updating with verification and safety handling
Receiving input from an occupant within the vehicle; updating the driver specific machine learning model; receiving, via communication between the system controller and onboard systems within the vehicle, data related to an action being taken by the occupant within the vehicle; comparing the action being taken by the occupant within the vehicle to the predicted desired action, and prompting the occupant within the vehicle to verify when the action does not match the predicted desired action, prompting verification when the action is identified by the machine learning model as an anomaly, and prompting with a warning message when the action is identified by the system controller as an inherently unsafe action.
HMI audible/visual recommendation and verbal/touch occupant input
Initiating the predicted desired action, including when a probability of the desired action exceeds fifty percent, prompting the occupant within the vehicle with a recommendation for the predicted desired action by at least one of providing the recommendation audibly via a speaker connected to a human machine interface (HMI) and displaying the recommendation on a touch screen display of the HMI; and receiving input from an occupant within the vehicle by at least one of receiving verbal input via a microphone connected to the HMI and receiving input via the touch screen display; and updating the driver specific machine learning model by receiving new data from other vehicles, selecting training data from the new data, and updating the driver specific machine learning model with the selected training data, and comparing input received to the predicted desired action such that when the input does not match the predicted desired action, updating the driver specific machine learning model.
Across claims 1, 7, and 13, the independent inventive concept is a vehicle controller that uses a driver specific machine learning model to identify behavior patterns and probabilistically predict a desired action from real-time location/operating-condition sensor data and a database of past actions. The controller then uses probability thresholds to prompt versus automatically initiate the action, and it incorporates occupant input, comparison of predicted versus actual actions, and updating of the driver-specific model, including anomaly and inherently unsafe action warning/verification behavior. Claim 13 additionally specifies HMI modalities for audible and visual recommendations and for verbal and touch input.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Providing situational recommendations within a vehicle for an occupant based on real-time vehicle location and operating conditions, using patterns of occupant behavior and stored past actions.
Prompting an occupant within the vehicle with a recommendation for a predicted desired action when probability exceeds fifty percent.
Automatically initiating the predicted desired action when probability exceeds ninety percent.
Prompting the occupant to verify proceeding when the occupant’s action does not match the predicted desired action.
Prompting the occupant to verify proceeding when the occupant’s action is identified as an anomaly.
Prompting the occupant with a warning message when the occupant’s action is identified as an inherently unsafe action.
[procedural detail omitted for safety] Example use cases mentioned include opening a glove box at a mall and behaviors after the occupant leaves in contexts such as post-leave rain or locking-door behaviors.
Interested in licensing this patent?