Vehicle operator workload estimation system and method
Inventors
Park, Jong Hoon • Chen, Lawrence • Higgins, Ian • Zheng, Zhaobo • MEHROTRA, Shashank Kumar • Salubre, Kevin • Mousaei, Mohammadreza • Willits, Steven • Levedahl, Blaine • Buker, Timothy • Xing, Eliot • Misu, Teruhisa • Scherer, Sebastian • Oh, Jean • Akash, Kumar
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Assignees
Member
Carnegie Mellon UniversityCarnegie Mellon University is a global research institution based in Pittsburgh, Pennsylvania, recognized for interdisciplinary education, research, and innovation in science, engineering, arts, technology, and social sciences. The university leads advancements in artificial intelligence, robotics, digital health, and performing arts. Located in a technology-driven and culturally rich city, CMU powers real-world impact through research centers, industry engagement, workforce training, and initiatives that shape regional and global communities.
Carnegie Mellon University is a global research institution based in Pittsburgh, Pennsylvania, recognized for interdisciplinary education, research, and innovation in science, engineering, arts, technology, and social sciences. The university leads advancements in artificial intelligence, robotics, digital health, and performing arts. Located in a technology-driven and culturally rich city, CMU powers real-world impact through research centers, industry engagement, workforce training, and initiatives that shape regional and global communities.
Publication Number
US-12504820-B1
Publication Date
2025-12-23
Expiration Date
Abstract
An estimation system includes a plurality of sensors that generate a multimodal signal, where the multimodal signal indicates a state of a user. The estimation system also includes at least one processor that receives the multimodal signal from the plurality of sensors, and determines a workload experienced by the user based on the multimodal signal and a workload model, wherein the workload model relates multimodal signal data to an experienced workload.
Core Innovation
The invention relates to an estimation system for determining workload experienced by a user. A plurality of sensors generate a multimodal signal indicating a state of the user, and at least one processor receives the multimodal signal. The at least one processor determines a workload experienced by the user based on the multimodal signal and a workload model that relates multimodal signal data to an experienced workload.
The multimodal signal further includes travel data indicating a predetermined travel route to be performed by a vehicle. The at least one processor receives predetermined operation information indicating a plurality of different predetermined operations to be performed by the user in the vehicle along the predetermined travel route, and receives actual operation information indicating a plurality of different actual operations performed by the user in the vehicle along the predetermined travel route.
For each operation along the predetermined travel route, the at least one processor determines a conformity between the predetermined operation information and the actual operation information. The at least one processor determines an effect on the workload experienced by the user along the predetermined travel route based on the determined conformity as part of the multimodal signal.
Claims Coverage
Two independent claims are identified: an estimation system and a computer-implemented method. Each independent claim is centered on a multimodal-signal workload model and extends it with vehicle travel-route context and conformity between predetermined and actual operations to determine an effect on experienced workload.
Multimodal-signal workload estimation based on a workload model
A plurality of sensors generate a multimodal signal indicating a state of a user, and at least one processor receives the multimodal signal and determines a workload experienced by the user based on the multimodal signal and a workload model, wherein the workload model relates multimodal signal data to an experienced workload.
Travel-route context with predetermined vs actual operation conformity driving workload effect
The multimodal signal includes travel data indicating a predetermined travel route to be performed by a vehicle; predetermined operation information and actual operation information are received for a plurality of different predetermined and actual operations along the predetermined travel route; conformity is determined between the predetermined operation information and the actual operation information for each operation; and an effect on the workload experienced by the user along the predetermined travel route is determined based on the determined conformity as part of the multimodal signal.
Computer-implemented determination of experienced workload using machine-learning workload model
The method generates a multimodal signal using a plurality of sensors indicating a state of a user; transmits the multimodal signal to at least one processor; and determines a workload experienced by the user based on the multimodal signal and a workload model using a machine learning algorithm executed by the at least one processor, wherein the workload model relates multimodal signal data to an experienced workload.
Method incorporating travel data and per-operation conformity to determine workload effect along the travel route
The method generates travel data with a vehicle operated by the user indicating a predetermined travel route and incorporates the travel data into the multimodal signal; receives predetermined operation information and actual operation information along the predetermined travel route; determines conformity between the predetermined operation information and the actual operation information for each operation; and determines an effect on the workload experienced by the user along the predetermined travel route based on the determined conformity as part of the multimodal signal.
Across both independent claims, the core coverage is determining experienced workload from a multimodal signal using a workload model, and linking that workload estimation to a predetermined travel route by computing per-operation conformity between predetermined and actual operations and using that conformity to determine an effect on the experienced workload.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Not explicitly described in patent.
Interested in licensing this patent?