Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12373411-B2
Publication Date
2025-07-29
Expiration Date
Abstract
Columnar storage provides many performance and space saving benefits for analytic workloads, but previous mechanisms for handling single row update transactions in column stores suffer from poor performance. A columnar data layout facilitates both low-latency random access capabilities together with high-throughput analytical access capabilities, simplifying Hadoop architectures for use cases involving real-time data. In disclosed embodiments, systems may perform read and scan features through which users may have consistent reads of the columnar storage at any given point in time. These scans (e.g., snapshots) allow for analytics to be performed on the data, consistent backups of the data, for incremental backups, perform cross-cluster synchronization, and/or for offline audit analysis.
Core Innovation
The disclosed invention provides a method for implementing read or scan in a distributed database system that supports multi-version concurrency control (MVCC) by instantiating a scanner at a particular timepoint. The scanner is configured to provide a point-in-time snapshot of a dataset associated with the particular timepoint. The dataset includes a row stored in an on-disk store via a base data module that stores column data for the row and a delta module that stores REDO delta records indicating later states and UNDO delta records indicating earlier states, each associated with a unique transaction identifier.
The method updates the stored column data to a particular later state by performing a delta compaction. Delta compaction applies one or more REDO delta records to update the column data stored in the base data module, removes the applied one or more REDO delta records from the delta module to create a decreased set of REDO delta records, and adds one or more new UNDO delta records that correspond to the removed REDO delta records to create an increased set of UNDO delta records.
The scanner provides the point-in-time snapshot by selecting a subset of the increased set of UNDO delta records or the decreased set of REDO delta records based on a state of an MVCC manager instance captured when the scanner is instantiated. The MVCC manager instance associates a particular transaction identifier with the particular timepoint. The scanner copies the updated column data for the row from the base data module into an output buffer, iteratively applies mutations indicated by the selected subset to change the updated column data to a particular committed state associated with the particular timepoint, and provides the point-in-time snapshot via the output buffer.
Claims Coverage
The partial content includes three independent claims: a method claim, a computing system claim, and a non-transitory computer-readable medium claim. Across these independent claims, the core coverage comprises instantiating a timepoint-configured scanner over a base data module plus REDO/UNDO delta modules, performing delta compaction that updates base data and adjusts REDO/UNDO sets, and selecting which UNDO/REDO mutations to iteratively apply based on an MVCC manager instance state associated with transaction identifiers.
Timepoint scanner using base and REDO/UNDO delta modules
Instantiating a scanner at a particular timepoint configured to provide a point-in-time snapshot of a dataset that includes a row stored via a base data module storing column data and a delta module storing REDO delta records indicating later states and UNDO delta records indicating earlier states, where each set is associated with a unique transaction identifier.
Delta compaction updating base data and adjusting UNDO/REDO sets
Performing a delta compaction that updates the column data stored in the base data module to a particular later state by applying one or more REDO delta records, removing the applied REDO delta records from the delta module to create a decreased set of REDO delta records, and adding corresponding new UNDO delta records to create an increased set of UNDO delta records.
MVCC-manager-driven selection and iterative mutation application to scanner output
Using the scanner to provide the point-in-time snapshot of the dataset associated with the particular timepoint by selecting a subset of the increased set of UNDO delta records or the decreased set of REDO delta records according to a state of an MVCC manager instance captured by the scanner when instantiated, where the state associates a particular transaction identifier with the particular timepoint; copying updated column data from the base data module into an output buffer; iteratively applying mutations indicated by the selected subset to change the updated column data to a particular committed state associated with the particular timepoint; and providing the point-in-time snapshot via the output buffer.
The independent claims consistently claim a point-in-time scan framework in which a scanner selects UNDO/REDO delta records based on an MVCC manager instance state tied to transaction identifiers, after delta compaction updates base data and restructures the REDO/UNDO record sets.
Stated Advantages
Documented Applications
No documented applications found
Interested in licensing this patent?