Virtual reality surgical device
Inventors
Sachs, Adam • KHALIFA, Sammy • Greene, Barry Stuart
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-10285765-B2
Publication Date
2019-05-14
Expiration Date
Abstract
A system for use in surgery includes a central body, a visualization system operably connected to the central body, a video rendering system, a head-mounted display for displaying images from the video rendering system, a sensor system, and a robotic device operably connected to the central body. The visualization system includes at least one camera and a pan system and/or a tilt system. The sensor system tracks the position and/or orientation in space of the head-mounted display relative to a reference point. The pan system and/or the tilt system are configured to adjust the field of view of the camera in response to information from the sensor system about changes in at least one of position and orientation in space of the head-mounted display relative to the reference point.
Core Innovation
The invention relates to a system for use in surgery that uses a central body, a visualization system, a head-mounted display, and a camera assembly with at least a first camera and at least one of a pan system or a tilt system. A first processor generates images based on information from the at least one camera and displays the generated images on the head-mounted display. A first sensor tracks at least one of the position or orientation of the head-mounted display relative to a first reference point, and the pan system or tilt system adjusts the field of view of the camera in response to changes in the tracked head-mounted display position or orientation.
The visualization system supports reality augmentation through magnifying glass or loupe animation, virtual monitor presentation, and force-based color changes. The imaging is extended with optional stereoscopic images and image interlacing for panoramic or interlaced visualization, and reality augmentation elements are coordinated with the displayed imagery. VR head tracking updates the camera views based on the head-mounted display pose, including latency compensation effects described for the head tracking.
The system further includes a robotic device operably connected to the central body with a first rotational actuator and first hinged actuator, followed by a second rotational actuator and second hinged actuator, and a third rotational actuator and third hinged actuator. A surgical end-effector is operably coupled to the third hinged actuator, and a second sensor tracks a series of changes in a position of a portion of an arm of a user.
A second processor determines a series of positions and orientations of at least two of twelve portions of the robotic device in response to the series of arm changes based on the degrees of freedom of a human arm, so that the robotic device replicate a human-like motion achievable by the human arm. Servomotors adjust positions or orientations of the rotational actuators, hinged actuators, and/or the surgical end-effector to cause the robotic device to follow the determined series of positions and orientations so that at least one of the twelve portions mimics the motion of the arm of the user.
Claims Coverage
The independent claim provides three main inventive groupings: head-tracked visualization with pan/tilt field-of-view adjustment, a multi-actuator robotic device with rotational and hinged actuators and a surgical end-effector, and arm-motion tracking mapped to human-arm degrees of freedom to replicate human-like motion, driven by servomotors.
Head-tracked camera field-of-view adjustment
A visualization system operably connected to the central body includes at least a first camera and at least one of a pan system or a tilt system, and a first processor generates images based on information from the at least one camera for display on a head-mounted display, while a first sensor tracks position and/or orientation of the head-mounted display relative to a first reference point and the pan system or tilt system adjusts the field of view of the camera in response to changes in at least one of position or orientation.
Human-arm motion replication using degrees of freedom mapping
A second sensor tracks a series of changes in a position of a portion of an arm of a user, and a second processor determines a series of positions and orientations of at least two of the twelve portions of the robotic device in response to the series of changes based on the degrees of freedom of a human arm, whereby the determined series of positions and orientations replicate a human-like motion achievable by the human arm.
Servomotor control to mimic user arm motion with a surgical end-effector
At least one servomotor adjusts at least one of positions or orientations of at least one of the first rotational actuator, the second rotational actuator, the third rotational actuator, the first hinged actuator, the second hinged actuator, the third hinged actuator, or the surgical end-effector to cause the robotic device to follow the determined series of positions and orientations so that at least one of the twelve portions mimics the motion of the arm of the user.
Overall, the claim coverage centers on a head-mounted-display-driven visualization system that adjusts camera field of view via a pan/tilt system, combined with an articulated robotic device whose twelve portions are commanded using mapped user arm motion based on human-arm degrees of freedom so that a surgical end-effector and robotic portions mimic human-like arm motion.
Stated Advantages
Documented Applications
No documented applications found
Interested in licensing this patent?