Augmented reality systems for time critical biomedical applications
Inventors
Kaul, Robert • Kohli, Sandeep S. • Widman, David • Ross-Howe, Sara
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
An augmented reality system and method configured to automatically provide a user, such as a physician, with a real-time heads-up view of a patient's real-time medical status using an augmented reality headset. The system can automatically identify patients, pull up relevant medical records, obtain real-time biomedical sensor data from the patient, and display this to the user while, at the same time, allowing the user to directly view the patient through the headset's transparent lenses, and leaving the user's hands free to manipulate the patient or perform other functions. The system and method are particularly useful for intensive care units and other emergency medical situations where the user needs to get an almost instant understanding of the patient's status.
Core Innovation
The disclosed invention provides a method and system for automatically displaying patient medical information on an augmented reality headset worn by a user. Using at least one computer processor in communication with the augmented reality headset and at least one sensor, the method automatically determines when the user is in proximity of the patient, determines an identity of the patient, and obtains medical history data pertaining to the patient. It further acquires real-time biomedical sensor data pertaining to the patient from at least one patient wearable biomedical sensor currently worn by the patient and displays at least some of the medical history data and at least some of the real-time biomedical sensor data on the augmented reality headset while the headset is in proximity to the patient.
The disclosure also includes privacy gating based on whether the augmented reality headset is oriented toward the patient’s position, such that at least some of the medical history data and real-time biomedical sensor data are suppressed when the viewing direction is not oriented toward the patient. Determination of proximity, patient identity, and obtaining of medical history data can be performed using speech recognition input from the user. In addition, the medical history data can include medical images, and the system uses the augmented reality headset and a patient location sensor to automatically scale and transform the medical images to create a medical image overlay aligned with a portion of the patient’s anatomy and nearby the patient.
For visualization of physiological information, the invention defines real-time biomedical sensor data as cardiovascular sensor data and animates a computer representation of the patient’s heart and lungs to correspond with the cardiovascular sensor data, producing a sensor responsive animation displayed on the augmented reality headset. The invention further uses patient identity, medical history data, and real-time biomedical sensor data to retrieve from computer memory and display clinical decision algorithms, alternative potential diagnoses, and alternative medical treatments.
Claims Coverage
The independent claims are directed to two complementary aspects: a method and a system for automatically displaying patient medical information on an augmented reality headset, with core inventive features totaling at least three areas (proximity/identity/history determination, acquisition of real-time wearable biomedical sensor data, and conditional AR display). Additional inventive features cover privacy gating, medical image overlay alignment, cardiovascular animation, medical-history correction of the representation, and optional clinical decision support retrieval and display.
Proximity-based patient identification and medical history retrieval for AR display
Automatically determines when the augmented reality headset and user are in proximity of the patient, determines an identity of the patient, and obtains medical history data pertaining to the patient using at least one computer processor and at least one sensor comprising at least a patient proximity sensor.
Real-time biomedical sensor data acquisition from patient wearable sensors
Acquires real-time biomedical sensor data pertaining to the patient from at least one patient wearable biomedical sensor currently worn by the patient and uses the acquired data for display while the augmented reality headset is in proximity to the patient.
Conditional augmented reality display of medical history and real-time sensor data
Displays, at least while the augmented reality headset is in proximity to the patient, at least some of the medical history data pertaining to the patient and at least some of the real-time biomedical sensor data pertaining to the patient.
Viewing-direction privacy gating for AR medical display
Suppresses display of at least some of the medical history data and real-time biomedical sensor data when a headset viewing direction is not oriented toward the patient’s position.
Speech recognition input for proximity, identity, and history determination
Determines when the user is in proximity of the patient, determines an identity of the patient, and obtains medical history data using speech recognition input from the user.
Medical image overlay creation aligned to patient anatomy
Uses the augmented reality headset and a patient location sensor to automatically scale and transform medical images included in the medical history data to create an overlay aligned with a portion of the patient’s anatomy and nearby the patient for the user.
Cardiovascular-responsive animation of heart and lungs
Animates a computer representation of the patient’s heart and lungs to correspond with cardiovascular sensor data included in the real-time biomedical sensor data, producing a sensor responsive animation displayed on the augmented reality headset.
Medical history corrected computer representation
Modifies the computer representation based on medical history data, producing a medical history corrected computer representation, and displays the sensor responsive animation on the augmented reality headset.
Clinical decision support retrieval and display from patient data
Uses any of the patient identity, medical history data, and real-time biomedical sensor data to retrieve and display clinical decision algorithms, alternative potential diagnoses, and alternative medical treatments from computer memory.
Unitized ambulatory vital sign monitoring sensor source for AR
Obtains at least some real-time biomedical sensor data from a unitized patient wearable vital sign monitoring system configured for an ambulatory patient, including ambulatory wearable sensors that transmit sensor data directly or indirectly to the augmented reality headset.
Across the independent claims, the coverage centers on automatically determining patient proximity and identity, obtaining medical history, acquiring real-time biomedical sensor data from patient wearable sensors, and displaying both on an augmented reality headset when proximate. Dependent claim refinements further cover privacy gating, speech recognition-based determination, anatomy-aligned medical image overlays, cardiovascular animation of heart and lungs, medical-history-based correction of the displayed representation, and optional retrieval and display of clinical decision algorithms, alternative potential diagnoses, and alternative medical treatments, including use of a unitized ambulatory vital-sign monitoring system as the sensor source.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Not explicitly described in patent.
Interested in licensing this patent?