Method for mathematical language processing via tree embeddings
Inventors
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Assignees
William Marsh Rice University • University of Massachusetts Amherst
Member
Rice UniversityRice University is a leading research university in Houston, Texas, recognized for its emphasis on scientific discovery, innovation, and interdisciplinary collaboration. The institution is committed to academic excellence, impactful research, and community engagement, offering robust undergraduate and graduate programs in engineering, natural sciences, social sciences, humanities, business, and the arts. Rice is distinguished by its history of collaboration with organizations such as NASA, fostering advances in space science, biotechnology, energy research, and artificial intelligence.
Rice University is a leading research university in Houston, Texas, recognized for its emphasis on scientific discovery, innovation, and interdisciplinary collaboration. The institution is committed to academic excellence, impactful research, and community engagement, offering robust undergraduate and graduate programs in engineering, natural sciences, social sciences, humanities, business, and the arts. Rice is distinguished by its history of collaboration with organizations such as NASA, fostering advances in space science, biotechnology, energy research, and artificial intelligence.
Publication Number
US-12147407-B2
Publication Date
2024-11-19
Expiration Date
Abstract
A method for processing formulae includes encoding a formula by: training, with a server, a model by using a machine learning algorithm with a data set that includes a plurality of formulae; transforming, with a processor, a first formula into a tree format using the trained model; converting, with the processor, the tree format of the first formula into a plurality of lists; and encoding, with the processor, the plurality of lists into a fixed dimension vector by leveraging a stacked attention module; and generating one or more formula candidates by: obtaining, with the processor, input information; and generating, with the processor, one or more second formula candidates based on input information by using the stacked attention module with a tree beam search algorithm.
Core Innovation
The claimed invention relates to a method for processing formulae that includes encoding a formula and generating one or more second formula candidates. Encoding the formula includes training, with a server, a model by using a machine learning algorithm with a data set that includes a plurality of formulae; transforming, with a processor, a first formula into a tree format using the trained model; converting, with the processor, the tree format of the first formula into a plurality of lists; and encoding, with the processor, the plurality of lists into a fixed dimension vector by leveraging a stacked attention module. Generating the one or more formula candidates includes obtaining, with the processor, input information; and generating, with the processor, one or more second formula candidates based on input information by using the stacked attention module in reverse order with a tree beam search algorithm.
The background identifies a problem that the large quantity of scientific formulae poses challenges for effectively organizing and synthesizing scientific formulae to derive new knowledge and insights, with an important real-world use case being formula retrieval. Existing techniques typically treat symbols as “words” and use sequential models that ignore the inherent tree structure of formulas, resulting in a loss of important structural information, and prior tree-based models are limited to binary trees and cannot handle formulas with an arbitrary number of children. Therefore, there is a need for a more efficient and accurate model that can process the tree structure of mathematical expressions and formulas while maintaining high speed, low cost, and easy implementation.
The disclosure proposes a framework that represents each symbolic formula as an operator tree to explicitly capture structural and semantic properties, includes an encoder that converts the operator tree into lists and leverages a stacked attention module to produce a fixed dimensional vector, and includes a tree decoder that generates a formula from a vector in operator tree format using a novel tree beam search algorithm. The method is validated on a formula reconstruction task and a similar formula retrieval task on a dataset of over 770k formulae, and experimental results show that the method significantly outperforms various baselines.
Claims Coverage
The independent claim includes five main inventive features.
Training a model on a dataset of formulae
Training, with a server, a model by using a machine learning algorithm with a data set, wherein the data set includes a plurality of formulae.
Transforming a formula into a tree format using the trained model
Transforming, with a processor, a first formula into a tree format using the trained model.
Converting the tree format into a plurality of lists
Converting, with the processor, the tree format of the first formula into a plurality of lists.
Encoding the plurality of lists into a fixed dimension vector using a stacked attention module
Encoding, with the processor, the plurality of lists into a fixed dimension vector by leveraging a stacked attention module.
Generating formula candidates using the stacked attention module in reverse order with a tree beam search algorithm
Obtaining, with the processor, input information; and generating, with the processor, one or more second formula candidates based on input information by using the stacked attention module in reverse order with a tree beam search algorithm.
The independent claim covers training a machine learned model on a dataset of formulae; transforming input formulae into a tree format; converting the tree into lists; encoding those lists into a fixed-dimension vector via a stacked attention module; and generating formula candidates by applying the stacked attention module in reverse with a tree beam search algorithm.
Stated Advantages
Provides an efficient and accurate way to retrieve relevant formulae similar to a query formula.
Encodes a formula into an effective and succinct numeric summarization (fixed dimension vector) suitable for downstream tasks such as retrieval.
Tree beam search enables generation of higher-quality formula trees compared to greedy search.
The method significantly outperforms various baselines on evaluated tasks.
Capable of processing trees in which nodes can have an arbitrary number of children, addressing limitations of prior binary-only tree models.
Documented Applications
Math content retrieval, including finding similar questions on academic question and answer sites, finding similar formula and content in scientific/mathematical articles, and finding similar questions or contents in STEM textbooks.
Clustering students' math answers to assist teachers for more efficient grading and feedback forwarding.
Math content generation to automatically generate relevant formulae based on a guide input such as a paragraph or text.
Automatic grading of students' mathematical responses to STEM questions involving formulae.
Automated derivation and verification of simple mathematical steps or proofs.
Automatic detection of cheating on answers to math questions involving formulae.
Automatic generation of formulae from text and vice versa.
Automatic generation of math practice problems in STEM disciplines with different contexts and numeric values.
Improving math knowledge tracing with models such as item response theory by explicitly taking into account the mathematical content in the question and students' answers.
Enabling next generation search engines for mathematical content and machine-assisted scientific/mathematical content writing/generation where formulae are involved.
Interested in licensing this patent?