Efficient memory management for heterogeneous computing
Inventors
MINISKAR, Narasinga Rao • Monil, Mohammad Alaul Haque • Lara, Pedro Valero • Vetter, Jeffrey S. • Liu, Frank Y.
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12411721-B2
Publication Date
2025-09-09
Expiration Date
Abstract
In a heterogeneous computing environment, a validity flag indicates whether a given memory has a valid copy of a given data object. When the data object is required at a target memory for a scheduled task, another (source) memory having a valid copy of the data object is identified. A protocol guides selection of the source memory based on efficiency of data transfer to the target memory. A data transfer command is generated at runtime, when tasks have been scheduled among respective processors. Execution of the command copies the valid data object from the source memory to the target memory. Superior performance is achieved even in extremely heterogeneous environments. Performance results, variations, and applications are presented.
Core Innovation
The invention addresses heterogeneous computing memory-management by using metadata associated with a first data object that comprises a plurality of validity flags corresponding to respective memories, including a first memory and a second memory. Each validity flag indicates whether the respective memory has a copy of the first data object that is valid. A received request requiring a valid copy in the first memory is handled by determining, based on the validity flag of the first memory, that the first memory does not have the valid copy, and then identifying the second memory that has a second copy of the first data object that is valid.
The second memory is identified based on a policy that selects the second memory to implement a most efficient data transfer to provide the valid copy of the first data object to the first memory. Executing the policy includes iterating over data transfer types, from most efficient to least efficient, until the second memory is identified, and at each iteration searching for a memory coupled to transfer data to the first memory according to the respective data transfer type and having a valid copy of the first data object. Based on this identification, the invention determines at run-time a command to perform a data transfer of the second copy from the second memory to the first memory and executes the command.
The invention extends the validity-flag logic to hierarchical parent and child data objects. A parent data object is hierarchically organized and comprises two or more child data objects, where the parent and child each have respective metadata comprising validity flags. For a given memory of the memories, the parent validity flag is set to valid when the validity flags of all the child data objects indicate valid, and is set to invalid when the validity flags of any one or more child data objects indicate invalid.
The described approach is evaluated using an Intelligent Runtime System (IRIS) with heterogeneous GPU systems, and performance improvements are reported for tiled matrix multiplication and tiled LU factorization. The runtime selects data transfer behavior dynamically in a way that provides benefits over baseline and manual transfers, including fallback behavior when higher-performance link options are unavailable.
Claims Coverage
The independent claims cover a computer-implemented method and computer-readable media that use validity flags across memories to locate a valid copy, select a source memory according to an efficiency-based policy over transfer types, and issue and execute a run-time transfer command; the coverage includes hierarchical parent/child validity handling of metadata validity flags.
Valid copy selection using plurality of validity flags across memories
Determining that a received request requires a valid copy of a first data object in a first memory, where metadata associated with the first data object comprises a plurality of validity flags corresponding to respective memories including the first memory and a second memory, each validity flag indicating whether the respective memory has a copy of the first data object that is valid; determining, based on the validity flag of the first memory, that the first memory does not have the valid copy; and identifying, based on one or more additional flags among the validity flags, that the second memory has a second copy of the first data object that is valid.
Policy-driven most efficient data transfer source identification and run-time transfer command
Selecting the second memory based on a policy that selects the second memory so as to implement a most efficient data transfer to provide the valid copy of the first data object to the first memory, wherein executing the policy comprises iterating over data transfer types, from most efficient to least efficient, until the second memory is identified, and at each iteration searching for a memory of the memories coupled to transfer data to the first memory according to the respective data transfer type and having a valid copy of the first data object; determining, at run-time, a command to perform a data transfer of the second copy of the first data object from the second memory to the first memory; and executing the command.
Hierarchical parent/child validity flag propagation
A parent data object is hierarchically organized and comprises two or more child data objects, the parent and child data objects each having respective metadata comprising validity flags; setting the validity flag of the parent data object to indicate valid when the validity flags of all the child data objects indicate valid, and setting the validity flag of the parent data object to indicate invalid when the validity flags of any one or more of the child data objects indicate invalid.
Coprocessor-local memory and task I/O with first data object as input
Receiving a request that specifies a task scheduled to execute on a coprocessor that hosts the first memory, wherein the first data object serves as input to the task.
Across the independent claims, the core inventive coverage is the use of validity flags in metadata to determine whether a first memory holds a valid copy of a first data object, identify a second memory holding a valid copy, and select that second memory using an efficiency-based policy over data transfer types to determine and execute a run-time transfer command. The coverage further includes hierarchical parent/child metadata validity flag propagation and a coprocessor/task context where the first data object serves as task input.
Stated Advantages
Performance improvements are reported for tiled matrix multiplication and tiled LU factorization using IRIS with heterogeneous GPU systems.
Dynamic transfer selection provides benefits over baseline and manual transfers.
Documented Applications
Tiled matrix multiplication and tiled LU factorization performed using an Intelligent Runtime System (IRIS) with heterogeneous GPU systems.
Run-time data transfer selection in heterogeneous computing environments with fallback behavior when higher-performance link options such as OpenMP move command are unavailable.
Interested in licensing this patent?