Anatomical feature extraction and presentation using augmented reality
Inventors
Assignees
Smith and Nephew Orthopaedics AG • Smith and Nephew Asia Pacific Pte Ltd • Smith and Nephew Inc
Publication Number
US-12364568-B2
Publication Date
2025-07-22
Expiration Date
2040-12-08
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
An ultrasound probe captures real-time images of patient anatomy, which are analyzed by a processor to extract salient features pertaining to an anatomical structure. By tracking the location and orientation of the ultrasound probe, a model of that anatomical structure can be created. A visual indication of the position of segments of the anatomical structure can be presented holographically to a user of an augmented reality headset to provide information extracted from the ultrasound imaging, such as holographic display of a model of the anatomical structure at the approximate location of the visual field of the headset corresponding to the physical location of the actual anatomy being viewed by a user, without presenting the entirety of the ultrasound image to the user.
Core Innovation
The invention relates to a system and method for extracting anatomical features from real-time two-dimensional ultrasound images captured by an ultrasound probe during a surgical procedure, and presenting these features to a user via an augmented reality head-mounted display (HMD). The system tracks the pose of the ultrasound probe and updates a three-dimensional feature model of the anatomical structure as the probe is moved relative to the surgical site. This model is holographically displayed in the approximate spatial location corresponding to the patient's actual anatomy within the user's field of view, enabling the user to visualize relevant anatomical information without viewing the full ultrasound image.
The problem addressed is the inefficiency and distraction in surgical workflows caused by conventional intraoperative imaging methods. Surgeons must maintain focus on the patient and avoid distractions, but existing imaging systems require the surgeon to look away or mentally map displayed images, which also may present irrelevant data obscuring the field of view. Furthermore, reading medical imaging such as ultrasound requires extensive training, and full images often contain extraneous information. The invention solves this by extracting salient features—such as bone surfaces or tumor locations—from ultrasound images and displaying them in a spatially accurate manner through augmented reality, minimizing distraction and enhancing anatomical understanding during surgery.
Claims Coverage
The patent includes two independent claims covering both a surgical system and a computer-implemented method with several inventive features.
Ultrasound-based three-dimensional anatomical feature modeling and display
The system and method receive sequential two-dimensional ultrasound images representing slices of an anatomical structure as the ultrasound probe is moved, extract features associated with the anatomical structure from these images, and update a three-dimensional feature model based on the probe's pose determined by a tracking system. The features within the field of view of a head-mounted display are displayed holographically to the user, enabling spatial correlation between the model and the patient's actual anatomy.
Pose determination and relative pose computation
The tracking system determines the pose of the ultrasound probe and the head-mounted display; the system computes the relative pose of the ultrasound probe with respect to the head-mounted display. This relative pose is used to correlate and display anatomical features within the user's field of view.
Integration of tracking and processing components within the head-mounted display
Embodiments cover configurations where the computer system and/or the tracking system are incorporated into the head-mounted display, supporting a self-contained augmented reality platform.
Use of fiducial markers for tracking
The tracking system employs a camera system configured to identify fiducial markers positioned on the ultrasound probe to determine its pose.
Temporal aging and removal of outdated features
The computer system determines whether features in the three-dimensional feature model are older than a threshold time and removes such features from the displayed field of view to maintain accuracy and reduce user distraction.
Together, these inventive features provide a surgical system and method that extract pertinent anatomical features from ultrasound images, track probe and display positions, update a dynamic three-dimensional anatomical model, and present the model holographically in real time to a surgical user through a head-mounted display, while managing feature relevance over time.
Stated Advantages
Reduces surgeon distraction by displaying only salient anatomical features rather than entire medical images.
Enhances surgical workflow efficiency by allowing surgeons to maintain visual focus on patient anatomy without shifting attention to separate imaging displays.
Improves correlation between intraoperative image data and actual patient anatomy by spatially aligning holographic models with the physical surgical site.
Facilitates shared visualization among multiple surgical staff via multiple head-mounted displays tracked relative to patient and ultrasound probe.
Documented Applications
Intraoperative surgical visualization of anatomical structures such as bone surfaces during procedures like knee and hip surgeries using ultrasound imaging.
Identification and holographic display of tumors, blood vessels (using Doppler ultrasound), and implanted instruments such as catheters during surgery to assist navigation and avoid damage.
Guidance for surgical planning and execution in computer-assisted surgical systems, including robotic-assisted operations, utilizing real-time augmented reality visualization.
Interested in licensing this patent?