Generating and reusing transformations for evolving schema mapping

Inventors

Roth, Sharon • Ilyas, Ihab F. • Bruckner, Daniel Meir • Goldin, Gideon

Assignees

Tamr Inc

Abstract

A system and method of use resolves the frustration of repeated manual work during schema mapping. The system utilizes a transformation graph—a collection of nodes (unified attributes) and edges (transformations) in which source attributes are mapped and transformed. The system further leverages existing mappings and transformations for the purpose of suggesting to a user the optimal paths (i.e., the lowest cost paths) for mapping new sources, which is particularly useful when new sources share similarity with previously mapped sources and require the same transformations. As such, the system also promotes an evolving schema by allowing users to select which unified attributes they want to include in a target schema at any time. The system addresses the technical challenge of finding optimal transformation paths and how to present these to the user for evaluation.

Core Innovation

The invention provides a computer assisted mapping of source attributes of an unmapped data source into an existing transformation graph. The transformation graph is an evolving graph of nodes, where nodes represent unified attributes and edges represent transformations between the nodes, and the transformation graph allows for building reusable transformation paths. The system maps source attributes from at least one data source into the nodes within the transformation graph by creating and connecting nodes and edges as the graph evolves.

For a second data source distinct from the at least one data source, the system selects one or more source attributes and, for each selected source attribute, checks for an exact match between the selected source attribute and existing nodes within the transformation graph. When an exact match is found, the matched source attribute is automatically mapped to the matched node; when an exact match is not found, a new node is created in the transformation graph and the unmatched source attribute is mapped to the new node. The system then identifies existing nodes similar to the new node and allows a selected similar node to be chosen.

The core innovation further includes receiving from the user a path from the new node to the selected node, and creating a new edge from the new node to a first existing node sequentially in the path received from the user. This user-supplied path enables building reusable transformation paths by connecting the newly created node into the existing transformation graph through transformation edges. The resulting source attributes of the second data source are mapped into the transformation graph according to the reusable transformation paths.

Claims Coverage

The independent claims are clm-00001 and clm-00009. Each independent claim includes the same core inventive functionality: mapping source attributes from a second distinct data source into an evolving transformation graph by exact-match node checking, creating new nodes when no exact match exists, selecting a similar existing node, and receiving a user-provided path to create new transformation edges sequentially to build reusable transformation paths (reusable mapping paths via new edge connections).

Across clm-00001 and clm-00009, the claims cover assisted mapping of source attributes into an existing evolving transformation graph by combining exact-match checking (creating new nodes when unmatched), similarity-based selection of existing nodes, and user-provided paths used to create new edges sequentially, thereby building reusable transformation paths in the transformation graph.

Stated Advantages

Documented Applications