System and method for tracking non-visible structure of a body with multi-element fiducial
Inventors
Daon, Ehud (Udi) • Beckett, Martin Gregory
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-9844413-B2
Publication Date
2017-12-19
Expiration Date
Abstract
A monitoring system tracks the non-visible structure of a body in three dimensions. A tracker obtains image information of the object and instruments in its vicinity, all bearing 3D tracking markers with at least one pattern segment. A controller spatially relates image information with previously obtained scan data of the object revealing non-visible structure of the object. For the scan a fiducial reference detectable in the scan is removably attached to a location on the object. Scan data and image information is used by the controller to provide the user with real time information on relative 3D locations and orientations of instruments and the non-visible structure of the body. The monitoring system may be used to model and track changes in the body itself. A model of the body and instruments is used to track contemplated actions and warn about possibly inappropriate instrument procedures.
Core Innovation
The invention provides a 3D monitoring system that determines the relative position and orientation of non-visible structure of a body by relating image information of a tracking marker to previously obtained scan data that includes internal structure of the body. A fiducial reference is used to establish correspondence between the body scan and the current tracked image region, enabling the system to determine where the external marker and fiducial reference are in relation to internal structure. A display system presents the determined relationship to a user.
The approach includes embodiments where the fiducial reference is fixed to the body with at least one pattern segment, and a three-dimensional tracking marker is rigidly attached at a predetermined location and predetermined orientation relative to the fiducial reference or to a fixed relative position and orientation. In other embodiments, the fiducial reference is capable of being removably attached to the body and is observable by a tracker, and a controller spatially relates the image information to the previously obtained scan data with the fiducial reference attached. A computer system or controller determines three-dimensional location and orientation by relating the image information to the scan data.
The system further supports identifying and modeling the fiducial reference in scan data and image information, including the identity of the fiducial reference and a model of a region based on scan data, fiducial identity, and image information. Multi-element fiducial patterns with individually locatable pattern segments are used to determine segment-level three-dimensional location and orientation, enabling monitoring of subcutaneous motion and spatial distortion and calculation of spatial transformation for expressing real-time location and orientation of the fiducial reference in relation to internal structure. Real-time image information and a derived spatial transformation matrix are used to relate the pose of internal structure in the scan to the pose during real-time monitoring.
Claims Coverage
The partial set includes three independent system claims and one independent method claim. Collectively, they describe determining real-time 3D position and orientation of internal non-visible structure by correlating tracker-derived image information with previously obtained scan data through a fiducial reference and a tracking marker.
Fiducial reference with rigid 3D tracking marker pose determination using image information and scan data
A system includes a fiducial reference adapted to be rigidly attached to the body, a three-dimensional tracking marker rigidly attached at a predetermined location in a predetermined orientation on an implement proximate the body, a tracker obtaining image information about an area encompassing the fiducial reference and the marker, and a computer system with previously obtained scan data of the body including internal structure with the fiducial reference fixed to the body, the computer system determining the relative position and orientation of the marker and the fiducial reference relative to the internal structure of the body based on the image information and the scan data.
Recognizing fiducial reference in scan data and images and calculating a region model
A system includes a fiducial reference capable of being attached to the body with at least one pattern segment perceptible in scan data, a three-dimensional tracking marker having a fixed connection with the fiducial reference, a tracker obtaining image information of a region encompassing the tracking marker, and a computing device having software capable of recognizing the fiducial reference in the scan data and in the image information and calculating a model of the region based on the scan data, the identity of the fiducial reference, and the image information.
Removably attached fiducial reference spatially relating image information to scan data
A system includes a tracker for obtaining image information of an area encompassing the body, a fiducial reference configured for removably attaching to the body to be observable by the tracker, a controller configured to spatially relate the image information to previously obtained scan data of the internal structure of the body with the fiducial reference attached, and software executable by the controller to determine a three-dimensional location and orientation of the fiducial reference in relation to the internal structure of the body by relating the image information to the scan data.
Real-time spatial transformation matrix relating internal structure in scan to internal structure in monitoring
A method performs relating in real time the three-dimensional position and orientation of non-visible internal structure of a body to the location and orientation of the structure in a scan of the body by removably attaching a fiducial reference having at least one pattern segment, performing a scan with the fiducial reference attached to obtain scan data including the internal structure, determining the three-dimensional location and orientation of the fiducial reference from the scan data, obtaining real time image information of the body, determining in real time the three-dimensional location and orientation of the fiducial reference from the image information, and deriving a spatial transformation matrix for expressing in real time the three-dimensional location and orientation of the fiducial reference in relation to the internal structure of the body as determined from the image information in terms of the three-dimensional location and orientation of the fiducial reference as determined from the scan data.
Across the independent claims, the core claim coverage centers on a fiducial reference with pattern segments, a three-dimensional tracking marker and tracker providing image information, and a computer system/controller using previously obtained scan data to determine three-dimensional location and orientation and to derive a real-time spatial transformation matrix relating internal non-visible structure in the scan to internal structure during monitoring. Additional coverage includes recognizing fiducial identity in scan data and image information and calculating a region model.
Stated Advantages
Enables real-time monitoring of the position and orientation of non-visible internal structure by relating image information to previously obtained scan data.
Determines relative position and orientation of a tracking marker and a fiducial reference relative to internal structure.
Uses fiducial reference identity and scan data to calculate a model of a region.
Provides a spatial transformation matrix for expressing real-time location and orientation of the fiducial reference in relation to internal structure.
Documented Applications
Dental surgery and endoscopic surgical site monitoring.
Monitoring tracking of non-visible structure within archaeological/forensic artifacts.
Applying the approach to tracking non-visible structure in artifacts including an Antikythera device.
Interested in licensing this patent?