Roboticized surgery system with improved control
Inventors
Brasset, Damien • Invernizzi, Paolo • Ruiz Morales, Emilio
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-11857278-B2
Publication Date
2024-01-02
Expiration Date
Abstract
A robotized surgery system comprises at least one robot arm which acts under the control of a control console intended for the surgeon. The console comprises an eye tracking system for detecting the direction of the surgeon's gaze and for entering commands depending on the directions of the gaze detected. The console comprises advantageously a screen with at least one zone for viewing the operating field and, among the commands which can be performed depending on the gaze directions.
Core Innovation
The invention is directed to a roboticized minimally invasive surgery system that includes a first robotic arm carrying a surgical instrument and a second robotic arm carrying an endoscopic camera, together with a surgeon console having a display displaying an image from the endoscopic camera. The system further includes an eye tracking system and a manipulator moveable by a user’s hands to generate movement command signals for the first robotic arm. The system is operable in an eye tracking mode in which movement of the second robotic arm is commanded based on signals from the eye tracking system corresponding to an estimated direction of the user’s gaze towards the image.
A manually-actuatable auxiliary input on the manipulator is used to enter the eye tracking mode. In response to user actuation of the auxiliary input, the system commands movement of the second robotic arm based on the gaze estimate, so that the endoscopic camera can be repositioned based on the eye tracking system signals. In a coordinated operation, movement of the manipulator while actuating the auxiliary input causes movement of the first robotic arm carrying the surgical instrument, and simultaneously causes the second robotic arm to reposition the endoscopic camera based on the eye tracking system input.
The invention also includes eye-tracking-based safety behavior tied to viewing zones or selection areas on the display. In this context, the system disables/deactivates robot-arm movement when the estimated gaze direction is outside an area of the display, thereby preventing continued commanded movement when the gaze is not within a viewing region. Example implementations described in the partial content also include a display HMI configured for screen-zone-based selection and optional gaze-driven telecamera centering/centering with confirmation, and a multi-computer console architecture managing real-time control and HMI generation.
Claims Coverage
The partial content identifies two independent claims (clm-00001 and clm-00007) covering both a method and a surgical system, respectively. Both independent claims share a common inventive concept: entering an eye tracking mode based on a manually-actuatable auxiliary input and commanding movement of a second robotic arm based on estimated gaze direction toward an endoscopic image, while the first robotic arm is moved by a hand manipulator.
Auxiliary-input-triggered eye tracking mode for gaze-estimated camera control
The system enters an eye tracking mode in response to actuation of a manually-actuatable auxiliary input on the manipulator, wherein in the eye tracking mode movement of the second robotic arm is commanded based on signals from the eye tracking system corresponding to an estimated direction of the user’s gaze towards the image.
Coordinated instrument and endoscopic camera movement during auxiliary-input actuation
Causing movement of the first robotic arm in response to movement of the manipulator, and commanding movement of the second robotic arm based on signals from the eye tracking system during the eye tracking mode entered in response to the auxiliary input actuation.
Hand-manipulator-driven first arm control with auxiliary-input entry into eye tracking mode
A manipulator moveable by a user’s hands to generate movement command signals for the first robotic arm, the manipulator further including a manually-actuatable auxiliary input, wherein the system operable in an eye tracking mode commands movement of the second robotic arm based on estimated direction of the user’s gaze towards the image in response to user actuation of the auxiliary input.
Eye-tracking-based deactivation when gaze is outside display area
Deactivating first and second robotic arms when signals from an eye tracking system indicate the user’s estimated gaze direction is outside an area of the display.
Across the independent claims, the inventive coverage centers on triggering an eye tracking mode using a manually-actuatable auxiliary input on a hand manipulator, and using eye-tracking signals corresponding to an estimated gaze direction toward an endoscopic image to command movement of a second robotic arm (endoscopic camera). Dependent refinements in the partial content further add coordinated instrument/camera movement during auxiliary-input activation and add deactivation when the gaze estimate is outside the display area.
Stated Advantages
Enters an eye tracking mode that commands movement of the endoscopic camera based on the estimated direction of the user’s gaze towards the image.
Provides an eye-tracking-based safety behavior by deactivating first and second robotic arms when the estimated gaze direction is outside an area of the display.
Documented Applications
Roboticized minimally invasive surgery using a surgical robotic system with an endoscopic camera and an eye tracking system to control camera movement based on surgeon gaze direction toward an image.
Operation in an eye tracking mode entered via a manually-actuatable auxiliary input on the manipulator, including coordinated movement of an instrument-carrying robotic arm and repositioning of an endoscopic camera.
Interested in licensing this patent?