Compact haptic mixed reality simulator
Inventors
HENSCHEL, Christopher • Leibacher, Ivo • CHOFEN, Daniel • TUSCHSCHMID, Stefan • Oelhafen, Markus • MATTES-O'BRIEN, Carolyn
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12159550-B2
Publication Date
2024-12-03
Expiration Date
Abstract
A compact and versatile mixed reality simulator system and methods may comprise a mixed reality processing engine and a haptic control engine to control a linear electromagnetic motor (LEM) of 1 degree of freedom made of a handheld instrument replicate and a trocar-like instrument hosting duct. The compact and versatile mixed reality simulator system may comprise multiple interchangeable instrument hosting ducts and handheld instrument replicates and the mixed reality processing engine may automatically detect, from sensors, which instrument replicate has been inserted in which duct. The mixed reality processing engine may detect a virtual contact between the handheld instrument replicate and a virtual object in a mixed reality scenario. The mixed reality processing engine may calculate with a real-time solver the kinesthetic force feedback signal to power the LEM as a function of the position and orientation of the instrument tracked from sensors, and of the position, orientation and material property of the virtual object as it deforms due to the virtual contact according to the mixed reality scenario. The mixed reality processing engine may adapt in real¬ time the haptic feedback by resealing its magnitude, and/or combining it with a vibration signal, possibly with an additional vibrotactile actuator arrangement. The mixed reality processing engine may jointly adapt the haptic feedback signal and the virtual reality scene rendering for a more realistic mixed reality experience with a haptic retargeting method such as space warping.
Core Innovation
The invention relates to simulating a medical procedure scenario in which an operator interacts with a medical simulator anatomy model mounted on a rotational element allowing a change in orientation. The method detects the orientation of the anatomy model and detects a position of a virtual object into the medical procedure scenario, where the virtual object has at least one material and/or physical property parameter and a virtual deformation model.
According to the detected orientation, the method updates the position, deformation model, and material and/or physical property of the virtual object. The method acquires a position and orientation from at least one instrument replicate manipulated by the operator, and detects a virtual contact between the acquired instrument replicate and the updated virtual object, the virtual contact causing a virtual deformation of the virtual object according to the updated deformation model.
The invention also provides a medical simulation system including an anatomy model with at least one portal entry mounted on a rotational element, an instrument hosting duct mounted on the portal entry, and an instrument replicate adapted to slide into the instrument hosting duct. The system includes at least one instrument position tracking system, a mixed reality processor engine, and a haptic controller, with the instrument hosting duct and instrument replicate arranged to form together a linear electromagnetic motor.
The mixed reality processor engine is configured to acquire instrument pose, acquire anatomy model orientation, update the virtual object position, deformation model, and material and/or physical property according to the anatomy orientation, detect virtual contact and deformation, calculate an axial force feedback signal based on virtual object state and instrument pose, and transmit the axial force feedback signal to the haptic controller for producing the axial force along the instrument hosting duct axis.
Claims Coverage
The document includes two independent claims, a method claim and a system claim. Across these claims, the coverage is built around rotational anatomy orientation driving virtual object updates, virtual contact-based deformation, and axial force haptic rendering using a linear electromagnetic motor formed by a duct and a sliding instrument replicate, with instrument pose tracking and controller transmission of a calculated axial force feedback signal.
Orientation-driven virtual object update for deformable material and physical property parameters
Detect an orientation of the anatomy model; detect a position of a virtual object having at least one material and/or physical property parameter and a virtual deformation model; update the position, deformation model, and material and/or physical property of the at least one virtual object according to the orientation of the anatomy model.
Linear electromagnetic motor formed by an instrument hosting duct and a sliding instrument replicate
Acquire a position and orientation from at least one instrument replicate manipulated by the operator, the instrument replicate being adapted to slide into an instrument hosting duct mounted on a portal entry of the anatomy model, said instrument hosting duct and said instrument replicate forming a linear electromagnetic motor; transmit the calculated variable axial force feedback signal to a controller of the linear electromagnetic motor so as to produce an axial force along the instrument hosting duct axis.
Virtual contact detection causing virtual deformation and axial force feedback rendering
Detect a virtual contact between the acquired position of the instrument replicate and the updated virtual object position in the medical procedure scenario, the virtual contact causing a virtual deformation of the virtual object according to the updated virtual deformation model; calculate an axial force feedback signal to haptically render the interaction between the virtual object and the instrument replicate, the magnitude of the axial force feedback signal being a function of the virtual object position, the updated virtual object deformation model, the updated virtual object material and/or physical property parameters, as well as of the position and orientation of the instrument replicate.
Medical simulation system with rotational anatomy, instrument tracking, mixed reality processor engine, and haptic controller for axial force generation
Provide an anatomy model with at least one portal entry mounted on a rotational element allowing a change in an orientation of the anatomy model, an instrument hosting duct mounted on the portal entry, and an instrument replicate adapted to slide into the instrument hosting duct, with at least one instrument position tracking system, a mixed reality processor engine, and a haptic controller, wherein the instrument hosting duct and the instrument replicate are arranged to form together a linear electromagnetic motor; configure the mixed reality processor engine to acquire instrument position and orientation, acquire anatomy model orientation, update a position, deformation model and material and/or physical property of at least one virtual object according to the orientation of the anatomy model, detect a virtual contact causing virtual deformation, calculate an axial force feedback signal as a function of updated virtual object state and instrument position/orientation, and transmit the calculated variable axial force feedback signal to the haptic controller for controlling the linear electromagnetic motor to produce an axial force along the instrument hosting duct axis.
Across the independent claims, the core coverage is the combination of rotationally oriented anatomy with mixed reality virtual object updating, virtual contact detection between tracked instrument replicate pose and the updated virtual object, and haptic rendering using a linear electromagnetic motor arranged as an instrument hosting duct plus sliding instrument replicate to produce axial force along the duct axis based on a calculated axial force feedback signal.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Not explicitly described in patent.
Interested in licensing this patent?