Remote presence system including a cart that supports a robot face and an overhead camera
Inventors
Stuart, David • Sanchez, Daniel Steven • Lai, Fuji • Hanrahan, Kevin • Jordan, Charles S. • Roe, David • Rosenthal, James • Mangaser, Amante • Whitney, Blair • Walters, Derek
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-10887545-B2
Publication Date
2021-01-05
Expiration Date
Abstract
A tele-presence system that includes a cart. The cart includes a robot face that has a robot monitor, a robot camera, a robot speaker, a robot microphone, and an overhead camera. The system also includes a remote station that is coupled to the robot face and the overhead camera. The remote station includes a station monitor, a station camera, a station speaker and a station microphone. The remote station can display video images captured by the robot camera and/or overhead camera. By way of example, the cart can be used in an operating room, wherein the overhead camera can be placed in a sterile field and the robot face can be used in a non-sterile field. The user at the remote station can conduct a teleconference through the robot face and also obtain a view of a medical procedure through the overhead camera.
Core Innovation
The invention relates to a tele-presence system for surgical mentoring that uses a cart carrying a robot face and an overhead camera mounted on a boom. The cart includes a first camera and a second camera, a speaker, a microphone, and an input device, and the second camera is coupled via a boom including a plurality of joints configured to allow a distance between first and second ends of the boom to be varied. The system is configured for coupling to a remote station via a network so that remote participants can control the orientation of the first camera and the orientation of the second camera from the remote station.
The remote station includes a camera, a monitor, a microphone, and a speaker, and provides a user interface on the station monitor with a remote view field, a station view field, and a plurality of graphical controls. The graphical controls include controls to selectively display video from the first camera or the second camera in the remote view field, to display a still image captured by the first camera or the second camera in the remote view field, to transfer an image to the cart monitor for display, and to annotate an image displayed on the cart monitor from the remote station.
The station view field displays an image captured by the station camera, while the station speaker reproduces sound captured by the cart microphone and the station microphone captures sound transmitted to and reproduced by the cart speaker. The disclosed system addresses improved procedure vantage for surgical mentoring by positioning the overhead camera in or over a sterile field while keeping the robot face in a non-sterile field, and supports two-way teleconferencing and mentoring using the network-based session.
Claims Coverage
The partial content includes three independent claims: a cart apparatus for surgical mentoring, a method for surgical mentoring, and a non-transitory computer-readable medium for a method that includes automated health condition scoring. Across these independent claims, the core inventive features include a jointed boom with variable distance for a second camera, a network-coupled remote station user interface with remote and station view fields and graphical controls for selectively displaying video/stills, transferring and annotating images, two-way audio reproduction, and remote control of independently oriented cameras.
Cart with dual cameras on a jointed boom and remote station UI for mentoring
A cart comprising a first camera, a second camera, a speaker, a microphone, and an input device, wherein the second camera is coupled to the cart via a boom that includes a plurality of joints configured to allow a distance between a first end of the boom and a second end of the boom to be varied; wherein the cart is configured to be coupled to a remote station via a network, the remote station including a camera, a monitor, a microphone, and a speaker, and configured to display a user interface on the station monitor with a remote view field, a station view field, and a plurality of graphical controls; and wherein the controls selectively display video from the first camera or the second camera, display a still image captured by the first camera or the second camera, transfer an image to the cart monitor for display, and annotate an image displayed on the cart monitor from the remote station; the station view field displays an image captured by the station camera, the station speaker reproduces a sound captured by the cart microphone, the station microphone captures a sound transmitted to and reproduced by the cart speaker, and the orientation of the first camera and the orientation of the second camera can be independently controlled from the remote station.
Remote-station controlled surgical mentoring method with dual-camera viewing, image transfer, annotation, and two-way audio
A method for surgical mentoring comprising displaying a user interface on a monitor of a remote station with a remote view field, a station view field, and graphical controls; establishing a communication session with a cart in proximity to a surgical field, the cart including a first camera, a second camera, a monitor, a microphone, a speaker, and an input device, wherein the second camera is coupled to the cart via a boom with joints configured to vary a distance between ends; selectively displaying video from the first camera and the second camera in the remote view field; displaying a still image captured by one of the first camera or the second camera in the remote view field; transferring an image to the cart monitor for display; annotating an image displayed on the cart monitor; displaying, in the station view field, an image captured by the station camera on the station monitor; reproducing, by the station speaker, a sound captured by the cart microphone; capturing, by the station microphone, a sound transmitted to and reproduced by the cart speaker; and controlling the orientation of the first camera and the orientation of the second camera from the remote station.
Computer-readable medium causing automated health condition scoring with remote-station camera control and audio
A non-transitory computer-readable medium comprising program code that causes processors to perform a method for automated health condition scoring, with the same remote station user interface, communication session with the cart, dual-camera selective display, still image display, image transfer and annotation, station view field display, two-way audio reproduction, and remote control of camera orientations.
Across the independent claims, the apparatus and methods center on a cart with first and second cameras where the second camera is coupled via a boom with joints configured to vary boom distance, and a network-coupled remote station that uses a user interface with remote view field and station view field plus graphical controls. The claims further require selective video and still display from either camera, image transfer to the cart monitor for display, annotation of images displayed on the cart monitor, two-way audio reproduction between cart and station, and independent remote control of camera orientations.
Stated Advantages
Improved procedure vantage for surgical mentoring by positioning the overhead camera in or over a sterile field while keeping the robot face in a non-sterile field.
Supports two-way teleconferencing and mentoring using camera view switching, still capture, image transfer, and annotation.
Documented Applications
Surgical mentoring in a tele-presence/teleconferencing system using a cart in proximity to a surgical field.
Automated health condition scoring using a remote-station interface, camera selection/display, image transfer/annotation, and two-way audio reproduction.
Interested in licensing this patent?