Methods and apparatuses for automated performance tuning of a data modeling platform

Inventors

Brown, Marion A. • Hoyle, Keith

Assignees

McKesson Corp

Interested in licensing this patent?

MTEC can help explore whether this patent might be available for licensing for your application.

Publication Number

US-10956386-B2

Patent

Publication Date

2021-03-23

Expiration Date


Abstract

Embodiments are disclosed for automated performance tuning of a data modeling platform. An example method includes ingesting, using a tuning template, a set of records from one or more data sources into one or more stage tables in a staging layer of the data modeling platform. This example method further includes generating, using the one or more stage tables, a data integration layer of the data modeling platform for modeling data represented by the set of records ingested into the one or more stage tables. This example method further includes estimating an anticipated performance of the data modeling platform and generating an optimized tuning template based on the estimated anticipated performance. Finally, this example method further includes automatically executing a performance tuning operation on the data modeling platform. Corresponding apparatuses and computer program products are also provided.

Core Innovation

The invention automates performance tuning of a data modeling platform by ingesting a set of records from one or more data sources into one or more stage tables in a staging layer using a tuning template. The stage tables comprise a set of standardized plumbing fields, where at least one standardized plumbing field includes a source primary key having a value that is a hash of an arrangement of related source data fields.

After ingestion, the invention generates a data integration layer of the data modeling platform using the one or more stage tables to model the data represented by the ingested records. Tuning circuitry estimates an anticipated performance of the data modeling platform based on the generated integration layer and generates a modified tuning template that reflects a change to the instructions for identifying the arrangement of the related source data fields included in the source primary key.

The invention automatically executes a performance tuning operation on the data modeling platform according to the modified tuning template. The execution results in a change in the arrangement of the related source data fields included in the hash and results in a change of the value of the source primary key, so the resulting integration layer is performance-tuned through updating the standardized plumbing fields that are used to model and represent the ingested data.

Claims Coverage

The independent claims cover an automated performance tuning method, an automated performance tuning apparatus, and a computer program product. Across these independent claims, the core coverage centers on ingesting records into stage tables using a tuning template, generating a data integration layer, estimating anticipated performance, producing a modified tuning template that changes source-primary-key hash-arrangement instructions, and automatically executing a performance tuning operation that changes the hash arrangement and the source primary key value.

Automated performance tuning using tuning template and standardized plumbing fields

Ingest a set of records from one or more data sources into one or more stage tables in a staging layer using a tuning template, wherein the stage tables comprise standardized plumbing fields and at least one standardized plumbing field comprises a source primary key having a value that is a hash of an arrangement of related source data fields, the arrangement identified according to instructions of the tuning template.

Generate a data integration layer from stage tables

Generate a data integration layer of the data modeling platform using the one or more stage tables for modeling data represented by the set of records ingested into the one or more stage tables.

Estimate anticipated performance and create a modified tuning template

Estimate an anticipated performance of the data modeling platform and generate a modified tuning template based on the estimated anticipated performance, the modified tuning template reflecting a change to the instructions for identifying the arrangement of the related source data fields included in the source primary key.

Automatically execute performance tuning to update source-primary-key hash arrangement

Automatically execute a performance tuning operation on the data modeling platform according to the modified tuning template, wherein execution results in a change in the arrangement of the related source data fields included in the hash and results in a change of the value of the source primary key.

The independent claims collectively define a feedback-driven tuning process in which anticipated performance is estimated from a generated data integration layer, and a modified tuning template changes the instructions used to form the source primary key hash arrangement. Automatic execution of the performance tuning operation updates the standardized plumbing fields so the source primary key value changes in accordance with the modified instructions.

Stated Advantages

Improves auditability/recoverability and query performance.

Reduces manual, error-prone tuning by applying tuning across stage tables.

Documented Applications

Automated performance tuning of a data modeling platform for query performance, including performance-tuned integration-layer modeling after ingestion and tuning.

JOIN OUR MAILING LIST

Stay Connected with MTEC

Keep up with active and upcoming solicitations, MTEC news and other valuable information.