Systems and methods of remote teleoperation of robotic vehicles
Inventors
Summer, Matthew D. • Bowman, William S. • Falendysz, Andrew D. • Makovy, Kevin M • Hedman, Daniel R • Truesdell, Bradley D.
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12124256-B2
Publication Date
2024-10-22
Expiration Date
Abstract
Systems and methods of manipulating/controlling robots. In many scenarios, data collected by a sensor (connected to a robot) may not have very high precision (e.g., a regular commercial/inexpensive sensor) or may be subjected to dynamic environmental changes. Thus, the data collected by the sensor may not indicate the parameter captured by the sensor with high accuracy. The present robotic control system is directed at such scenarios. In some embodiments, the disclosed embodiments can be used for computing a sliding velocity limit boundary for a spatial controller. In some embodiments, the disclosed embodiments can be used for teleoperation of a vehicle located in the field of view of a camera.
Core Innovation
A method of teleoperation of a vehicle in a field of view of a camera includes receiving, from an input device coupled to a user interface device, a desired direction of motion of a ground vehicle in a user interface reference frame. The method also includes receiving, via a first wireless network connection associated with an aerial vehicle coupled to a remote camera, a first position of the remote camera in a global reference frame, and the remote camera provides visual data of the ground vehicle for projection on the user interface reference frame.
The method further includes receiving, via a second network connection coupled to the ground vehicle, a second position of the ground vehicle in the global reference frame. The method generates, based on the first position of the remote camera in the global reference frame, the desired direction of motion in the user interface reference frame, and the second position of the ground vehicle in the global reference frame, a direction of motion of the ground vehicle in a vehicle reference frame that corresponds to the desired direction of movement in the user interface reference frame.
The method transmits the direction of motion to the ground vehicle and controls the ground vehicle to move in the direction of motion in the vehicle reference frame. Corresponding instructions on non-transitory, computer-readable media and a system with one or more processors perform similar operations for teleoperation using the user-indicated desired direction of motion and projecting visual data on the user interface reference frame.
Claims Coverage
The independent claims are clm-00001, clm-00007, and clm-00013. Each independent claim centers on generating a vehicle-reference-frame motion direction that corresponds to a user-indicated desired direction expressed in a user-interface reference frame, using remote-camera visual data and global-reference-frame positions received via wireless network connections.
User-indicated motion direction mapped to vehicle reference frame using remote camera and global positions
Receiving, from an input device coupled to a user interface device, a desired direction of motion of a ground vehicle in a user interface reference frame; receiving, via a first wireless network connection associated with an aerial vehicle coupled to a remote camera, a first position of the remote camera in a global reference frame where the remote camera provides visual data of the ground vehicle for projection on the user interface reference frame; receiving, via a second network connection coupled to the ground vehicle, a second position of the ground vehicle in the global reference frame; generating, based on the first position of the remote camera in the global reference frame, the desired direction of motion in the user interface reference frame, and the second position of the ground vehicle in the global reference frame, a direction of motion of the ground vehicle in a vehicle reference frame that corresponds to the desired direction of movement in the user interface reference frame; transmitting the direction of motion; controlling the ground vehicle to move in the direction of motion in the vehicle reference frame.
Non-transitory media executing teleoperation direction generation using remote camera projection
One or more non-transitory, computer readable media storing instructions that cause one or more processors to perform operations comprising: receiving, from an input device coupled to a user interface device, a desired direction of motion of a first unmanned vehicle in a user interface reference frame; receiving, via a first wireless network connection associated with a second unmanned vehicle coupled to a remote camera, a first position of the remote camera in a global reference frame where the first unmanned vehicle is within a field of view of the remote camera and the remote camera provides visual data for projection on the user interface reference frame; receiving, via a second network connection coupled to the first unmanned vehicle, a second position of the first unmanned vehicle in the global reference frame; generating, based on the first position of the remote camera in the global reference frame, the desired direction of motion in the user interface reference frame, and the second position of the first unmanned vehicle in the global reference frame, a direction of motion of the first unmanned vehicle in a vehicle reference frame that corresponds to the desired direction of movement in the user interface reference frame; transmitting the direction of motion; controlling the first unmanned vehicle to move in the direction of motion in the vehicle reference frame.
System generating and transmitting vehicle-reference-frame direction using remote camera global pose and projection
A system comprising one or more processors and one or more non-transitory, computer-readable storage media storing instructions which when executed cause the one or more processors to perform operations comprising: receiving, from an input device coupled to a user interface device, a desired direction of motion of a first vehicle in a user interface reference frame; receiving, via a first wireless network connection associated with a second vehicle coupled to a remote camera, a first position of the remote camera in a global reference frame where the first vehicle is within a field of view of the remote camera and the remote camera provides visual data for projection on the user interface reference frame; receiving, via a second network connection coupled to the first vehicle, a second position of the first vehicle in the global reference frame; generating, based on the first position of the remote camera in the global reference frame, the desired direction of motion in the user interface reference frame, and the second position of the first vehicle in the global reference frame, a direction of the first vehicle in a vehicle reference frame that corresponds to the desired direction of movement in the user interface reference frame; transmitting the direction of motion to the first vehicle; controlling the first vehicle to move in the direction of motion in the vehicle reference frame.
Across clm-00001, clm-00007, and clm-00013, the claim coverage focuses on computing a motion direction in a vehicle reference frame that corresponds to a desired direction expressed in a user interface reference frame, using remote-camera visual data for projection and camera/vehicle positions in a global reference frame received via wireless network connections.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Teleoperation of a vehicle located in a field of view of a camera, where a ground vehicle is controlled to move based on a user-indicated desired direction of motion expressed in a user interface reference frame using remote camera visual data projection and global reference frame positions.
Interested in licensing this patent?