Method for determining whether medication has been administered and server using same
Inventors
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-11457862-B2
Publication Date
2022-10-04
Expiration Date
Abstract
Provided is a server for determining whether medication has been administered, the server including: a transceiver receiving a video recorded by a wearable device; a memory storing a detection model and a confirmation model, wherein the detection model is trained to output whether each of preset targets appears in an image, and the confirmation model is trained to output whether medication has been administered, wherein the preset targets include an object related to a medicine or a medicine container and a posture related to medication administration; and one or more processors configured to detect the preset targets by inputting image frames of the video to the detection model and to determine whether medication has been administered by inputting confirmation model input data to the confirmation model, the confirmation model input data generated based on a detection result of the detection model.
Core Innovation
A server determines whether medication has been administered by receiving a plurality of image frames obtained by a wearable device. The server uses a detection model to detect an object related to medication administration and a posture related to medication administration, and the detection model generates detection results for each image frame in the form of sub data related to a plurality of labels. The server then generates confirmation model input data including a probability value related to each label at a plurality of time points based on the sub data across the image frames.
The server inputs the confirmation model input data to a confirmation model to obtain information indicating whether a user has administered medication. The confirmation model is a model for identifying medication administration based on a detection result of the detection model. The plurality of labels are set in a training process of the detection model, and the labels include a first object label corresponding to a medicine container, a first posture label corresponding to a first posture of holding the medicine container, and a second posture label corresponding to a second posture of taking a medicine.
Additional embodiments include video-category classification that conditionally selects between different type-specific detection and confirmation pipelines for different medication administration modalities, and embodiments that distinguish main-hand versus sub-hand medication administration. The disclosed processing synthesizes probability outputs across time points into confirmation model inputs, and the description includes handling omitted portions of the medication process by training with removed frames and sub data.
Claims Coverage
The document includes three independent claims, covering a server architecture for detection-plus-confirmation across time points, a processor-implemented method counterpart of the same pipeline, and training of both the detection model and the confirmation model using predetermined label sets and time-point probability inputs. Across the independent claims, the inventive features focus on label-based object and posture detection from wearable-device frames, synthesis into confirmation inputs at multiple time points, and outputting whether medication has been administered; additional refinements include category-conditional selection of model pipelines.
Time-series detection-to-confirmation medication determination
A server that receives a plurality of image frames from a wearable device, obtains sub data for a plurality of labels using a detection model for an object related to medication administration and a posture related to medication administration, constructs confirmation model input data including probability values for each label at a plurality of time points based on the sub data, and inputs the confirmation model input data to a confirmation model to obtain information indicating whether a user has administered medication.
Object and posture labels tied to medicine container holding and taking
Labels for training and inference that include a first object label corresponding to a medicine container, a first posture label corresponding to a first posture of holding the medicine container, and a second posture label corresponding to a second posture of taking a medicine, wherein the first posture label includes at least a portion of the medicine container and a portion of a finger or a palm, and wherein the second posture label includes at least a portion of a face and at least a portion of a finger or a palm.
Conditional model selection based on video category
Classifying a category of a video using a classification model and selecting between a first and a second detection model to generate sub data corresponding to whether the video category corresponds to a first or second medicine container type, and determining whether medication has been administered using category-dependent confirmation models based on the video category.
Training detection and confirmation models using label sets and time-point probabilities
A training method that obtains detection model training data including training images and detection model labels determined based on a predetermined label set, updates the detection model by comparing detection model labels and detection model output data, obtains confirmation model training data including confirmation model input data with probabilities for each label at a plurality of time points and a confirmation model label for determining whether medication has been administered, and updates the confirmation model by comparing the confirmation model label and confirmation model output data.
Overall claim coverage centers on wearable-device image frames processed by a detection model to produce probability-related label sub data across multiple frames, followed by a confirmation model that consumes probability-related inputs at multiple time points to output whether medication was administered; label definitions tie medicine-container objects to specific holding and taking postures, with optional video-category-driven selection and aligned category-dependent detection and confirmation pipelines. A separate independent claim covers training of both models using predetermined label sets for detection and time-point probability inputs for confirmation.
Stated Advantages
Improved determination accuracy when combining detection and confirmation versus monitoring-only.
Improved determination accuracy when using category-specific models.
Documented Applications
Telemedicine reporting, including a telemedicine secondary opinion report and prescription report, based on determination of whether medication has been administered.
Interested in licensing this patent?