Data tracing identifiers for tracking data flow through a data model and computing services
Inventors
MARTURANA, LUCA • Yadav, Nikunj
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-11861568-B2
Publication Date
2024-01-02
Expiration Date
Abstract
There are provided systems and methods for data tracing identifiers for tracking data flow through a data model and computing services. An entity, such as company or business, may provide computing services. When providing these services, one or more computing services, processors, or the like of the service provider's computing architecture may be used. This may include processing data through a data processing flow that uses multiple computing services. To automate data tracing through a data processing flow and corresponding computing services, a data library may be used to create and inject data trace identifiers to a message header associated with the data to be processed. This may be propagated with the data to other and/or centralized computing services, which may then persist the identifier with the data in a data table column for a standardized data model. These mappings may be stored by the centralized computing services.
Core Innovation
The invention provides a data tracing system for a service provider having a plurality of computing services that share a unified data model for data storage by corresponding databases. For a data processing flow, the system receives data, generates a data trace identifier (ID) for the data based at least on the unified data model, and writes the data trace ID to a data library in a data table accessible by the plurality of computing services during the computing service. The data trace ID comprises information particular to the data.
During use of the data processing flow, the system propagates the data trace ID with the data through the plurality of computing services. The propagating includes injecting the data trace ID, using a remote procedure call (RPC) interceptor, to a message header of a message associated with the data, and individual records for the data trace ID are created in the corresponding databases of the plurality of computing services based on the propagating.
After propagation, the system queries the corresponding databases for the data trace ID and determines ID tracing data based on the data storage by corresponding databases. The system standardizes the ID tracing data for the data trace ID in a data format for the unified data model, writes the standardized ID tracing data to the data library in association with the data trace ID, and includes additional ID tracing data for the data from at least one additional data processing pathway. The system then determines that the data is processed by the plurality of computing service using a plurality of data processing flows, connects a data processing pathway associated with the data processed by a subset of the plurality of computing services with the at least one additional data processing pathway, and identifies based on the data processing pathway one of the subset of the plurality of computing services that causes a data processing issue with the data.
Claims Coverage
The patent contains three independent claims: clm-00001 (system), clm-00008 (method), and clm-00015 (non-transitory machine-readable medium). Across these independent claims, there are 11 main inventive features supporting centralized ID-based tracing using a unified data model, shared data library storage, RPC interceptor propagation into message headers, per-service database record creation and querying, standardized ID tracing data, pathway connection using additional data processing pathway information, and identification of a subset of computing services that causes a data processing issue.
Unified data model across corresponding databases
receiving data for a data processing flow using at least one of a plurality of computing services associated with a service provider, wherein the plurality of computing services share a unified data model for data storage by corresponding databases of the plurality of computing services
Data trace identifier generation based on unified data model
generating a data trace identifier (ID) for the data based at least on the unified data model for the data storage, wherein the data trace ID comprises information particular to the data
Writing data trace ID to shared accessible data library table
writing the data trace ID to a data library for the plurality of computing services of the service provider, wherein the data trace ID is written to a data table of the data library that is accessible by the plurality of computing services during the computing service
RPC interceptor message header propagation
propagating the data trace ID with the data through the plurality of computing services during a use of the data processing flow with the service provider, wherein the propagating comprises injecting, using a remote procedure call (RPC) interceptor, the data trace ID to a message header of a message associated with the data during the use of the data processing flow
Creating per-service database records for the trace ID
creating individual records for the data trace ID in the corresponding databases of the plurality of computing services based on the propagating
Querying corresponding databases for data trace ID
querying the corresponding databases for the data trace ID
Determining ID tracing data and standardizing in unified data model format
determining ID tracing data based on the data storage by corresponding databases of the plurality of computing services; standardizing, in a data format for the unified data model, the ID tracing data for the data trace ID in the data library
Writing standardized and additional pathway ID tracing data
writing the standardized ID tracing data to the data library in association with the data trace ID, wherein the data library comprises additional ID tracing data for the data trace ID from at least one additional data processing pathway
Determining multiple data processing flows using the traced library data
determining that the data is processed by the plurality of computing service of the service provider using a plurality of data processing flows based on the writing and the data library
Connecting data processing pathway using standardized and additional ID tracing data
connecting, based on the standardized ID tracing data and the additional ID tracing data, the data processing pathway associated with the data that is processed by a subset of the plurality of computing services with the at least one additional data processing pathway
Identifying subset computing service that causes data processing issue
identifying, based on the data processing pathway, one of the subset of the plurality of computing services that causes a data processing issue with the data for the computing services
Across the independent claims, the core coverage is centralized tracing using a data trace identifier generated from a unified data model, stored in an accessible data library, propagated through computing services by injecting the trace ID into message headers (using an RPC interceptor in the system claim), recorded as per-service database records, then queried, standardized, and used to connect data processing pathways and identify a subset of computing services that causes a data processing issue.
Stated Advantages
Identifies one of the subset of the plurality of computing services that causes a data processing issue with the data for the computing services.
Enables data pathway reconstruction by connecting data processing pathways associated with traced data.
Standardizes ID tracing data in a data format for the unified data model to support standardized tracing across multiple computing services.
Documented Applications
Financial-service data processing flows, including payment card transaction and underwriting/payment deposit transactions, for tracing data pathways and identifying faulty services.
Tracing to determine a computing service causing a data processing issue for traced data processed by multiple computing services.
Interested in licensing this patent?