Method and system of enforcing privacy policies for mobile sensory devices
Inventors
Kapadia, Apu • Templeman, Robert E. • Crandall, David • KORAYEM, Mohammed
Assignees
Indiana University Research and Technology Corp • US Department of Navy
Publication Number
US-10592687-B2
Publication Date
2020-03-17
Expiration Date
2034-10-09
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
A method and device for classifying collected images. The method and device include instructions to compare a captured image to a known set of images to determine the location depicted therein; and applying a classification upon the image based upon the determined location depicted therein and whether the determined location indicates that the image has the potential to depict privacy sensitive information.
Core Innovation
The invention provides a method and device for classifying collected images to enforce privacy policies by determining the location depicted in the captured images. This is done by comparing a captured image to a known set of images associated with a geospatial location and applying a classification based on whether the location suggests that the image may depict privacy sensitive information.
The system includes a system layer between the image capturing device and applications that requires images to pass through the layer before use. This layer includes an image classifier, policy rules, and a policy enforcer to decide whether images should be freely available, quarantined pending user approval, or prevented from use or deleted, based on the classification of images as potentially private.
The problem addressed is that mobile, portable devices with sensors capable of capturing audio and video might invade privacy either inadvertently or through malicious exploitation. Because users may forget these devices are capturing media, and because some areas or professions require protection of sensitive information, there is a need for enforcing privacy policies that restrict or quarantine media collected in privacy sensitive locations to prevent unauthorized dissemination.
Claims Coverage
The patent includes three independent claims focusing on methods, devices, and computer-readable media for classifying images captured by mobile devices and enforcing privacy policies based on location and image content.
Image classification based on location and privacy sensitivity
Obtaining known images based on geospatial location; comparing captured images to these known images to determine specific locations depicted; and applying a classification to images based on the determined location and whether that location indicates potential privacy-sensitive information. Classification uses a scoring function evaluating similarity and a probabilistic model based on the chronological sequence of captured images and locations.
System layer enforcing image privacy policies on mobile devices
An image handling device including an image capturing device, applications, and memory storing instructions that instantiate a system layer between the image capture and applications. This system layer includes an image classifier that determines classifications based on comparisons to known images and probabilistic models using chronological image sequences. An image policy enforcer applies image policy rules that specify actions (make freely available, make unavailable, or hold for user approval) based on classification and location sensitivity.
Computer-readable medium with instructions for privacy-aware image classification
Non-transitory computer-readable media storing instructions that cause a processor to obtain known images based on geospatial location, compare captured images to these known images to determine locations depicted, and apply classification based on whether these locations potentially depict private information. The classification uses similarity scoring and probabilistic models based on chronological image sequences and associated locations.
Overall, the claims cover methods, devices, and media implementing classification of captured images using location-based known images, combined with probabilistic models on sequences of images, to enforce privacy policies including quarantining or deleting images that depict privacy-sensitive locations or information.
Stated Advantages
Prevents unauthorized capture and dissemination of privacy-sensitive media content by quarantining or deleting images taken in sensitive locations.
Distinguishes between active and passive image capturing to improve classification accuracy and potential invasiveness detection.
Utilizes local invariant features and global scene-level features to improve robustness of image classification, including handling blurry or occluded images.
Incorporates temporal sequencing models (Hidden Markov Model) to improve classification accuracy over sequences of images.
Allows user interaction to label ambiguous images, improving classification performance with minimal user input.
Documented Applications
Use in handheld or portable computing/sensory devices such as smartphones, life-logging devices (e.g., Memoto, Autographer, Google Glass) to enforce privacy policies on captured images.
Protect workplace privacy in environments such as hospitals or offices where personally identifiable medical or sensitive information is present.
Filtering images captured in blacklisted spatial locations to prevent privacy violations by quarantining or deleting images.
Interested in licensing this patent?