Multi-dimensional visualization in computer-assisted tele-operated surgery
Inventors
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-11918306-B2
Publication Date
2024-03-05
Expiration Date
Abstract
The technology described in this document can be embodied in a method of displaying images of portions of a human body on a display device. The method includes receiving a representation of a plurality of images that includes images of at least two different modalities, and location information corresponding to at least a subset of the plurality of images. A first image of a first modality is displayed on the display device in accordance with the corresponding location information. A second image of a second modality is overlaid on the first image in accordance with corresponding location information. At least a third image is overlaid on the first image in accordance with corresponding location information, the third image being of the second modality, and the second and third images being displayed concurrently for at least a period of time.
Core Innovation
The invention provides a multimodal image overlay for minimally invasive tele-operated robotic surgery. It receives a representation of a plurality of two-dimensional (2D) images of portions of a human body, where the plurality of 2D images includes images of at least two different modalities and the representation includes location information corresponding to at least a subset of the plurality of 2D images.
The system displays a first image of a first modality and displays a second image of a second modality overlaid on and registered with respect to the first image in accordance with corresponding location information. It also displays at least a third image of the plurality of 2D images, where the third image is of the second modality and is overlaid on and registered with respect to the first image using corresponding location information.
The second and third images are displayed together, each at a corresponding location as registered with respect to the first image, for at least a period of time, thereby providing a three-dimensional (3D) visualization of the portions of the human body on the display device. Implementations described include endoscopic and ultrasound images, including combining ultrasound B-mode and Doppler, where temporal visualization can be supported by sequential image trails with concurrent persistence.
Claims Coverage
The independent claims include three claim sets, as a method, a surgical system, and machine-readable instructions. Across the independent claims, the inventive features number in at least five recurring elements: receiving multimodal 2D images with location information, displaying a first modality image, overlaying and registering second-modality images at corresponding locations, displaying at least a third image of the second modality together for a period of time to provide a 3D visualization, and acquiring the second and third images using a probe held by a robotic arm while applying at least one motion constraint.
Multimodal 2D image representation with location information
Receiving a representation of a plurality of two-dimensional (2D) images of portions of the human body, wherein the plurality of 2D images includes images of at least two different modalities, and the representation includes location information corresponding to at least a subset of the plurality of 2D images.
Overlay and register second-modality image relative to first modality using location information
Displaying a first image of the plurality of 2D images on the display device in accordance with corresponding location information associated with the first image, the first image being of a first modality; and displaying a second image of the plurality of 2D images, the second image overlaid on and registered with respect to the first image in accordance with corresponding location information associated with the second image, the second image being of a second modality.
Overlay and register third image of second modality and display together for 3D visualization
Displaying at least a third image of the plurality of 2D images, the third image overlaid on and registered with respect to the first image in accordance with corresponding location information associated with the third image, the third image being of the second modality; wherein the second and third images are displayed together, each at a corresponding location as registered with respect to the first image, for at least a period of time, to provide a three-dimensional (3D) visualization of the portions of the human body on the display device.
Probe-held acquisition on a robotic arm with motion constraint
Wherein the second image and the at least third image of the plurality of 2D images are acquired by a probe of an imaging device held by a robotic arm and at least one motion constraint is applied to the robotic arm.
Surgical system implementation with display device and processing devices
A surgical system comprising a display device and one or more processing devices configured to operate the surgical system to perform a surgical process, receive a representation of a plurality of two-dimensional (2D) images, display a first image, display a second image, and display at least a third image, wherein the second and third images provide a three-dimensional (3D) visualization and are acquired by a probe held by a robotic arm with at least one motion constraint.
Across the independent claims, the coverage is centered on multimodal 2D images represented with location information, where a first modality image is displayed and second-modality images, including at least a third image, are overlaid and registered at corresponding locations relative to the first image and displayed together for at least a period of time to provide a 3D visualization, with acquisition of the second and third images by a probe held by a robotic arm under at least one motion constraint. The independent claims further define these functions as a method, a surgical system, and machine-readable instructions.
Stated Advantages
Reduced latency.
Improved visualization of anatomy and tumors.
Improved visualization of temporal events such as tissue perfusion and blood flow.
Documented Applications
Minimally invasive tele-operated robotic surgery using multimodal image overlay, including endoscope images and ultrasound images such as B-mode plus Doppler, to provide 3D visualization for at least a period of time.
Interested in licensing this patent?