Systems and methods for fast and repeatable embedding of high-dimensional data objects using deep learning with power efficient GPU and FPGA-based processing platforms

Inventors

Kaufhold, John Patrick • Trammell, Michael Jeremy

Assignees

General Dynamics Mission Systems Inc

Abstract

Embodiments of the present invention are directed to providing new systems and methods for using deep learning techniques to generate embeddings for high dimensional data objects that can both simulate prior art embedding algorithms and also provide superior performance compared to the prior art methods. Deep learning techniques used by embodiments of the present invention to embed high dimensional data objects may comprise the following steps: (1) generating an initial formal embedding of selected high-dimensional data objects using any of the traditional formal embedding techniques; (2a) designing a deep embedding architecture, which includes choosing the types and numbers of inputs and outputs, types and number of layers, types of units/nonlinearities, and types of pooling, for example, among other design choices, typically in a convolutional neural network; (2b) designing a training strategy; (2c) tuning the parameters of a deep embedding architecture to reproduce, as reliably as possible, the generated embedding for each training sample; (3) optionally deploying the trained deep embedding architecture to convert new high dimensional data objects into approximately the same embedded space as found in step (1); and optionally (4) feeding the computed embeddings of high dimensional objects to an application in a deployed embodiment.

Core Innovation

A system and method generate a deterministic deep embedding that substantially replicates an embedding produced by a selected embedding algorithm operating within an object embedding module. The embedding is created from a plurality of high-dimensional training data objects, where each training object has a different representation of an actual object, and the embedding comprises a set of ordered pairs of a high-dimensional training data object and a corresponding low-dimensional training vector.

A deep architecture training module trains a neural network with the set of ordered pairs to produce a deterministic deep architecture function that can substantially replicate the embedding. After training, a deep architecture deployment module receives a high-dimensional input data object from an external data source obtained from an observation of a physical object and invokes the deterministic deep architecture function to generate a low-dimensional summary vector representation of the received high-dimensional input data object.

The invention emphasizes separation between creating paired training data using a selected embedding algorithm and deploying a trained deep architecture function for independent out-of-sample embedding without recomputing the original embedding algorithm. By decoupling embedding discovery from deploy-time embedding, the deterministic deep architecture function supports repeatability for new received high-dimensional input data objects while not perturbing existing embeddings.

Claims Coverage

Independent claim clm-00001 covers a multi-processor system architecture with three functional modules: embedding ordered pairs using a selected embedding algorithm, training a neural network to learn a deterministic deep architecture function that replicates the embedding, and deploying the trained function to generate a low-dimensional summary vector for a received high-dimensional input data object obtained from an observation of a physical object. The claim recites 3 main inventive features across these modules.

The independent claim combines ordered-pair embedding generation from high-dimensional training data, neural network training to yield a deterministic function that substantially replicates that embedding, and deployment to map a new observed high-dimensional input data object to a low-dimensional summary vector.

Stated Advantages

Documented Applications