Three-dimensional shape data production method and three-dimensional shape data production system
Inventors
Nakagawa, Takeshi • Barada, Daisuke
Assignees
Utsunomiya University • Rion Co Ltd
Publication Number
US-10878563-B2
Publication Date
2020-12-29
Expiration Date
2039-02-28
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
A three-dimensional shape data production method and a system for the same for realizing contactless receiving of data regarding the inner shape of a tubular body which includes a step of generating multiple pieces of two-dimensional image data based on a signal from an image capturing device configured to be movable inside a tubular body and configured to capture an image of the inside of the tubular body; a step of receiving space information on the image capturing device upon image capturing by the image capturing device based on a signal from a motion sensor placed at the image capturing device; and a step of associating the two-dimensional image data and the space information with each other and generating three-dimensional shape data of the inside of the tubular body based on the two-dimensional image data and the space information.
Core Innovation
The invention relates to a three-dimensional shape data production method and a system that realizes contactless acquisition of shape data regarding the inner shape of a tubular body, such as the external ear canal. It involves generating multiple two-dimensional image data from a movable image capturing device inside the tubular body, receiving spatial information of this device via a motion sensor, and associating the image data with the spatial information to generate three-dimensional shape data of the tubular body's interior.
The problem addressed is the difficulty in capturing accurate inner shape data of tubular bodies like the external ear canal without using impression materials. Traditional methods relying on impression materials have issues such as deformation due to sharp curvatures, skin movement in elderly subjects, and dependence on the skill of the person making the impression. Contactless techniques mainly focus on outer shapes and have not adequately addressed inner tubular shapes. The invention aims to provide a method and system that can solve such problems by reproducing three-dimensional data contactlessly.
The method comprises generating two-dimensional imagery from an image capturing device with lighting, objective optical system, and image capturing element inside the tubular body, and receiving space information from a motion sensor attached to the device. The three-dimensional shape data is generated by calculating the movement locus of the objective optical system's principal point, defining cylindrical coordinates around this locus, discretizing the surrounding space into infinitesimal surfaces, and analyzing straight lines connecting light receiving elements to this principal point to determine reflected light intensities. These intensities allow estimation of the tubular body's inner surface, enabling three-dimensional reconstruction without contact.
Claims Coverage
The patent includes two independent claims covering a method and a system for producing three-dimensional shape data from inside a tubular body.
Generating three-dimensional shape data using two-dimensional image data and motion sensor data
The method involves generating multiple two-dimensional image data pieces via an image capturing device movable inside a tubular body with lighting, an objective optical system, and an image capturing element having multiple light receiving elements. It receives space information from a motion sensor placed at the device and relates the two-dimensional image data and space information to generate three-dimensional shape data of the tubular body's interior.
Matrix-based calculation of reflected light intensities from discretized infinitesimal surfaces
The method calculates a movement locus of the objective optical system's principal point and sets cylindrical coordinates about this locus. It discretizes the surrounding space into infinitesimal surfaces and determines straight lines through the principal point and each light receiving element. Using matrix formula u=(AT A)−1ATv, it calculates reflected light intensities for these surfaces and estimates those forming the tubular body's inside to build three-dimensional shape data. Matrix A's elements have specific nonzero or zero values based on whether reflected light is receivable from surfaces on these straight lines.
Correction of space information using additional motion sensor data
The method further includes a step of correcting the space information on the image capturing device obtained by the motion sensor, based on space information on the tubular body from other motion sensors.
Four-part system for producing three-dimensional shape data
The system includes a two-dimensional image data generation device, a space information receiving device for signals from a motion sensor on the image capturing device, and a three-dimensional shape data generation device. The system performs the same cylindrical coordinate-based discretizing, matrix-based calculation of reflected light intensities, and three-dimensional shape data building as the method.
Configuration of image capturing device with tip end and bendable portion
The image capturing device in the system includes a tip end portion with the objective optical system and a bent portion supporting this tip end flexibly, with the motion sensor provided at the tip end portion.
Inclusion of holding device to hold the tubular body
The system further includes a holding device configured to directly or indirectly hold the tubular body during image capturing and processing.
The claims cover a contactless three-dimensional shape data production method and system that employ an image capturing device inside a tubular body and a motion sensor to generate two-dimensional image data and space information. A key inventive aspect is employing cylindrical coordinates to discretize space, associating light receiving elements with infinitesimal surfaces on straight lines to calculate reflected light intensities via a matrix formula and thereby reconstructing three-dimensional shape data. Additional features include correcting spatial data using other sensors, system configuration aspects, and holding devices.
Stated Advantages
Enables contactless receiving and reproduction of inner shape data of tubular bodies, avoiding problems caused by impression material deformation and skill dependency.
Solves issues such as difficulty in removing hardened impression materials and inaccuracies caused by skin movement or wrinkles in the external ear canal.
Allows generation of accurate three-dimensional shape data based on two-dimensional images combined with precise spatial information from motion sensors.
Documented Applications
Producing three-dimensional shape data of an external ear canal for customized hearing aids and ear molds.
Reproducing internal shapes of other tubular bodies such as the digestive tract (intestines), respiratory tract, or water pipes.
Automatic identification methods for clinical conditions of the human body and detecting damaged or defective portions of tubular bodies.
Interested in licensing this patent?