System and methods for data transmission and rendering of virtual objects for display
Inventors
Pavloff, Alexander George • Redd, Bryan Lafayette
Assignees
Publication Number
US-11403822-B2
Publication Date
2022-08-02
Expiration Date
2039-09-23
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
System and methods are disclosed for displaying real-time, geo-registered data in an augmented reality or other virtual environment. The system and methods may be configured to supply missing information from the real-time, geo-registered data set in order to facilitate the display in three dimensions.
Core Innovation
The invention provides systems and methods for displaying real-time, geo-registered data in augmented reality (AR) or virtual reality (VR) environments. It enables a "geo-capable" device functioning in a global reference frame, understanding the user's position and direction, and allowing real-time display of georegistered information in three dimensions based on Latitude, Longitude, and Altitude coordinates. This capability addresses the limitation of current AR devices that lack positioning information or relate only to local reference frames.
The system resolves the challenge of insufficient and inconsistent data for accurate 3D rendering, for example, cases where elevation data is missing or incomplete in geo-registered data sets. The invention supplies missing information by transforming, combining, or retrieving supplemental data, such as elevation meshes, to enable proper positioning and rendering of virtual objects. It also optimizes data processing by simplifying and transforming data sets to reduce processing demands on wearable devices, maintaining appropriate fidelity while managing large and complex data streams in real time.
The system also addresses challenges in combining multiple data sources with different coordinate systems, datums, and resolutions. It converts and aligns data into a uniform structure and reference frame for use in AR/VR rendering, enabling correlated and appropriately scaled displays of virtual objects relative to physical environments. The invention supports local caching and on-the-fly transformations to enable use in environments with limited connectivity and device resources.
Claims Coverage
The patent includes two independent claims covering a method and a system for displaying real-time geo-registered virtual objects in augmented reality environments, focusing on handling missing data and establishing common reference frames.
Supplying missing information for 3D rendering using a mesh reference data set
The method receives a data set missing information needed for 3D rendering, stores a mesh-based reference data set at different densities covering the missing information, and uses this to supply the missing data to enable virtual object rendering in an AR system related to the real world.
Defining and converting to a common reference frame related to the local physical environment
The method defines a common reference frame based on a local reference frame tied to the physical environment and converts the received data set to this frame for accurate positioning of virtual objects relative to the real world.
Generating and rendering geo-registered virtual objects in real time
The system processes real-time geo-registered information that may have missing data, transforms it according to the common reference frame, supplements missing data from stored reference meshes, and displays virtual objects geo-registered and correlated to the real environment through an AR display system.
Mesh generation and simplification techniques to optimize data
The method tessellates raw reference data into triangular mesh representations, simplifies this mesh by combining or averaging triangles within thresholds, and employs variable mesh densities based on data variability and viewer position to optimize processing while maintaining rendering fidelity.
The inventive features collectively provide a method and system for accurately rendering real-time, geo-registered virtual objects in AR by addressing missing data through mesh-based references, unifying data into a common spatial reference, and optimizing data processing to efficiently render virtual objects correlated with the real world environment.
Stated Advantages
Allows real-time display of geo-registered information in 3D, enhancing user situational understanding by correlating virtual and physical environments.
Manages and supplies missing data in received datasets to facilitate accurate virtual object rendering in augmented or virtual reality.
Optimizes device resources by transforming and simplifying data sets while maintaining appropriate fidelity to reduce processing power, time, and power consumption.
Provides hardware-agnostic software solutions compatible with AR and VR systems and various data sources and coordinate systems.
Supports caching and data management to enable operation with limited or intermittent network connectivity.
Enables novel user interactions, including varying display scales (overview and 1:1), user-generated virtual objects, and exporting augmented views for collaboration and record keeping.
Documented Applications
Displaying real-time, geo-registered aircraft positions and information overlay in augmented reality using ADS-B data for aviation situational awareness.
Monitoring infrastructure such as utility poles, roads, and vehicles with geo-registered virtual object overlays for asset management and maintenance support.
Augmented reality visualization of property parcels and destroyed buildings to provide situational understanding in disaster recovery scenarios.
Use in military mission planning and battlefield awareness through real-time, geo-registered displays of troop and asset positions.
Integration with airborne video and images to add geo-referenced virtual overlays for inspection applications, such as utility inspections and infrastructure monitoring.
Providing overview and detailed scale viewing modes for situational awareness, asset placement, and planning in various operational contexts.
Interested in licensing this patent?