Conflation of geospatial points of interest and ground-level imagery
Inventors
De, Debraj • Gurav, Rutuja • Fan, Junchuan • Thakur, Gautam
Assignees
Publication Number
US-12008800-B2
Publication Date
2024-06-11
Expiration Date
2043-10-25
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
A prediction system harvests geo-tagged ground-level images through one or more algorithms. The system receives point of interest data representing structures or events and tags the geo-tagged ground-level images with a probability describing a classification. The system tags point of interest data with a hierarchical genre classification and encodes the tagged geo-tagged ground-level images as vectors to form nodes and edges in a proximity graph. The system encodes tagged points of interest data as similarity vectors to render more nodes and more edges on the proximity graph associated with the tagged geo-tagged ground-level images nodes by calculated semantic distances. The system splits the proximity graph into a training subgraph and a testing subgraph and trains a neural network by aggregating and sampling information from neighboring nodes within the training subgraph graph and validates through the testing subgraph. Training ends when a loss measurement is below a threshold.
Core Innovation
The invention introduces a predictive system that harvests geo-tagged ground-level images using scene detection algorithms and receives point of interest data representing structures or events. Each geo-tagged image is tagged with a probability to describe a classification, while each point of interest is assigned a hierarchical genre classification. These elements are encoded into numerical vectors that form nodes and edges within a proximity graph, using measured semantic distances to define their relationships.
The system employs an entity encoding process to convert images and point of interest data into semantic representations, constructing a graph-based model where images and points of interest become nodes connected by edges based on their semantic proximity. This approach allows for the integration of diverse datasets in various formats and provides a mechanism to align and compare disparate data such as hierarchical categories for points of interest and probabilistic tags for images within a unified framework.
A key process involves splitting the proximity graph into training and testing subgraphs. A graph neural network is trained on the training subgraph by aggregating information from neighboring nodes, with training continuing until a binary cross-entropy loss falls below a predetermined threshold. The system is then evaluated using the testing subgraph, resulting in a predictive model capable of making accurate semantic relationships and label predictions across multimodal geospatial datasets.
Claims Coverage
There are three independent claims in this patent, each disclosing a major inventive feature relating to the conflation of geospatial points of interest and ground-level imagery in a predictive system.
Non-transitory machine-readable medium for predictive geospatial conflation and edge prediction
A machine-readable medium encoded with instructions for: - Harvesting geo-tagged ground-level images via a scene detection algorithm executed by an image classifier. - Receiving point of interest data representing structures or events through a transceiver. - Tagging each ground-level image with a probability object describing a tag classification. - Tagging each point of interest data with a hierarchical level genre classification object. - Encoding tagged ground-level images as hot encoded vectors to render nodes and edges within a proximity graph, and encoding tagged point of interest data as similarity vectors for semantic association in the same graph using a calculated semantic distance measurement. - Splitting the proximity graph into a training subgraph and a testing subgraph. - Initiating and conducting a neural network training session by aggregating and sampling information from neighboring nodes within the training subgraph, terminating the session when a binary cross-entropy loss measurement falls below a predetermined threshold, and initiating evaluation via the testing subgraph.
Process for predictive geospatial conflation and edge prediction
A process comprising steps of: - Harvesting geo-tagged ground-level images through a scene detection algorithm executed by an image classifier. - Receiving point of interest data representing structures or events via a transceiver. - Tagging each geo-tagged ground-level image and each point of interest data with probability and hierarchical genre classifications, respectively. - Encoding geo-tagged images as encoded vectors, and point of interest data as similarity vectors, both forming nodes and edges on a proximity graph according to calculated semantic distance measurements. - Splitting the proximity graph into training and testing subgraphs. - Initiating a neural network training session that aggregates sampled neighbor node information within the training subgraph, and terminating upon binary cross-entropy loss falling below a predetermined threshold.
Edge prediction system using semantic embedding and proximity graph
A system comprising: - An image classifier that harvests geo-tagged ground-level images via scene detection algorithms. - A transceiver receiving point of interest data representing structures or events. - A processor tagging geo-tagged images with probability objects and point of interest data with hierarchical genre classification objects. - An encoder converting tags into hot encoded vectors and similarity vectors, building nodes and edges within a proximity graph based on calculated semantic distance measurements. - A partitioning engine splitting the proximity graph into training and testing subgraphs. - A training engine processing the training subgraph through a neural network, aggregating sampled neighbor information and terminating training based on binary cross-entropy loss threshold. - A testing engine evaluating the neural network with the testing subgraph.
The inventive features center around systems, processes, and machine-readable media for harvesting, tagging, encoding, and integrating geo-tagged images with points of interest into a proximity graph, training neural networks for semantic edge prediction, and evaluating the predictions using structured semantic relationships.
Stated Advantages
The system provides accurate and comprehensive predictive analytics by leveraging diverse geospatial data sources and multimodal information.
It enables proactive predictive analysis and makes predictions that increase computer efficiency, streamline processes, and reduce required human interaction.
The approach enhances computer system performance and functionality, delivering fine-grained, information-rich, and configurable point-of-interest services.
The use of semantic embedding and shared vector space allows for improved alignment, integration, and direct comparison of image and point of interest data from heterogeneous sources.
Dense proximity graph modeling and neighbor sampling enable scalability and efficiency for large graph datasets, with robust prediction performance even as input edge information is reduced.
Documented Applications
Automatic quality assurance devices and processes.
Point of interest and location services providers and processes.
Automatic data cleaning processes without human intervention.
Fine-grained and information-rich point of interest configurable devices and processes.
Downstream applications and devices including geographical information systems, location-based services, geospatial intelligence systems, and assessment of critical infrastructure for disaster management.
Interested in licensing this patent?