Methods and systems for detecting objects by non-visible radio frequencies and displaying associated augmented reality effects
Inventors
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-10318811-B1
Publication Date
2019-06-11
Expiration Date
Abstract
Methods and systems for detecting and identifying one or more objects in the vicinity using radio frequency signals, excluding visible spectrum frequencies, (the “non-visible RF signals”) and displaying an AR image with an AR element associated with each of the one or more identified objects are disclosed. An exemplary method includes the steps of: detecting and registering one or more objects in the vicinity of a mobile device by non-visible RF signals; tracking the registered objects; rendering the AR elements associated with the registered objects; and displaying an AR image with the AR elements anchored to the associated objects in the visual scan area of the mobile device. This embodiment may further include a step of rendering a special kind of AR elements, called LARI AR elements, that includes a boundary and a cut-out area, and the associated AR effect rendered is limited to the area in the boundary but excluding the cut-out area. As such, in this embodiment, the object with a LARI AR element would appear entirely or partially framed by the AR element in the AR image.
Core Innovation
The invention renders an Augmented Reality (AR) image on a display of a mobile device for one or more objects detected by radio frequency signals excluding visible spectrum frequencies (non-visible RF signals). The method repeatedly detects and registers the objects located within a vicinity of the mobile device by one or more sensors of the mobile device communicating with the objects by the non-visible RF signals, and repeatedly tracks a relative location of each object relative to the mobile device. The invention then generates in real time an AR element of each object with an AR type as Large-AR-Image (LARI) and located inside a visual scan area, and displays the AR image with the AR element anchored to each object inside the visual scan area.
The LARI AR element is generated by determining a boundary configured to surround a real-world image of the object, determining a cut-out area within the boundary configured to show a predetermined portion of the real-world image of the object, and rendering an AR effect within the boundary excluding the cut-out area. The AR effect is scaled and oriented based on the relative location and an orientation of the object. The cut-out area is determined using distances derived from multiple transmission broadcast units (TBUs) located along a peripheral of a projected cut-out area, based on response time for each TBU to respond to the mobile device via the non-visible RF signals and in a sequence of the TBUs, and also using a distance between each pair of consecutive TBUs.
The visual scan area is determined based on the camera field of view and non-visible RF signals. The invention also supports generating AR elements using object attributes and, in some embodiments, one or more associated social media profiles or combinations thereof, and determining whether to generate or display the AR element based on one or more object attributes meeting one or more user input criteria. Registered objects may be removed from the storage component when the object’s relative location stays outside the visual scan area for at least a predetermined period of time, and objects can be registered in batch using unique identification (ID) via non-visible RF signals.
Claims Coverage
The independent claims cover at least five core inventive groupings that define rendering LARI AR elements from objects detected using non-visible RF signals, defining the AR element with a boundary and a cut-out area while excluding the cut-out region from the rendered AR effect, anchoring/scaling/orienting the rendered AR effect based on relative location and object orientation, determining the cut-out area using distances derived from multiple TBUs including response-time sequencing and inter-TBU distances, and refinements concerning visual scan area determination and optional gating using retrieved attributes and user input criteria.
LARI AR rendering from non-visible RF detected objects
Repeatedly detect and register one or more objects located within a vicinity of a mobile device using sensors communicating with the objects by non-visible RF signals excluding visible spectrum frequencies; repeatedly track a relative location of each object relative to the mobile device; generate in real time an AR element of each object with an AR type as Large-AR-Image (LARI) located inside a visual scan area; display the AR image with the AR element anchored to each object located inside the visual scan area.
Boundary and cut-out defining a Large-AR-Image
Determine a boundary of the AR element configured to surround a real-world image of the object; determine a cut-out area within the boundary configured to show a predetermined portion of the real-world image of the object; render an AR effect within the boundary excluding the cut-out area.
AR effect scaling and orientation from relative location and object orientation
Render the AR effect scaled and oriented based on the relative location and an orientation of the object, while the AR effect is rendered within the boundary excluding the cut-out area, and display the AR image rendered with the AR element anchored to each object inside the visual scan area.
Cut-out area determined from multiple TBUs response-time distances
Determine the cut-out area based on a distance between the mobile device and each of multiple transmission broadcast units (TBUs) located along a peripheral of a projected cut-out area, where the distance is determined based on a response time for each TBU to respond to the mobile device via the non-visible RF signals and in a sequence of the TBUs, and a distance between each pair of consecutive TBUs.
Visual scan area based on camera field of view and non-visible RF signals
Determine a visual scan area based on the camera field of view of a mobile device and non-visible RF signals.
Conditional AR generation/display based on attributes meeting user input criteria
Generate and display an AR element of each object only when one or more object attributes retrieved via the non-visible RF signals satisfy one or more user input criteria.
Object attribute sources include social media profiles
Generate each AR element in the visual scan area based on one or more object attributes retrieved from the objects and/or from social media profiles associated with the objects.
Batch object detection and registration using unique ID requests
Simultaneously scan and register a batch of one or more objects near a mobile device by broadcasting a single unique identification (ID) request via non-visible RF signals, receiving each object’s ID via the non-visible RF signals, collecting unregistered objects into the batch until a predetermined batch size is reached, and then registering each batch member object by saving its ID and relative location in a storage component.
Relative location determined via triangulation using response-time-derived distances
Repeatedly determine a member object’s relative location via triangulation using real-time distances derived from response times to non-visible RF signals between the mobile device and the member object, between the mobile device and a reference object, and between the reference object and the member object.
Lifecycle cleanup when objects remain outside the visual scan area
Remove a registered object from the storage component when the object’s relative location stays outside the visual scan area for at least a predetermined period of time.
Across the independent claims, the invention consistently covers rendering LARI AR elements for nearby objects detected via non-visible RF signals, using a boundary plus a cut-out area that excludes the cut-out region from the rendered AR effect, and anchoring the AR image to tracked objects with scaling and orientation based on relative location and object orientation. The cut-out area is determined from distances derived from multiple TBUs using response-time sequencing and inter-TBU distances, with additional refinements regarding visual scan area determination, conditional gating based on retrieved attributes and user input criteria, optional social-media-based AR content sourcing, batch ID-based registration, triangulation for relative location, and cleanup when objects leave the scan area.
Stated Advantages
Not explicitly described in patent.
Documented Applications
social networking, lost asset recovery, property recovery trackers (active RFID/GPS trackers), rescue scenario victim tracking, and patient monitoring/surveillance [procedural detail omitted for safety]
Interested in licensing this patent?