Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-8949111-B2
Publication Date
2015-02-03
Expiration Date
Abstract
A method includes accessing text that includes a plurality of words, tagging each of the plurality of words with one of a plurality of parts of speech (POS) tags, and creating a plurality of tokens, each token comprising one of the plurality of words and its associated POS tag. The method further includes clustering one or more of the created tokens into a chunk of tokens, the one or more tokens clustered into the chunk of tokens based on the POS tags of the one or more tokens, and forming a phrase based on the chunk of tokens, the phrase comprising the words of the one or more tokens clustered into the chunk of tokens.
Core Innovation
The invention addresses discovering latent relationships in data by applying Latent Semantic Analysis (LSA) to sub-phrases derived from an accessed phrase. The method accesses a phrase comprising a plurality of nouns, creates a plurality of tokens in which each token includes one of the nouns together with an associated noun tag, and clusters one or more of the created tokens into a chunk of tokens based on the noun tags of the tokens in the chunk.
Based on the clustered tokens, the method forms one or more sub-phrases of the accessed phrase, where the sub-phrases comprise the nouns of the clustered tokens, and then performs LSA using the sub-phrases. Noun tagging is used to differentiate proper noun usage for geographic names and names of persons, including proper nouns that can be both geographic and personal names.
The disclosure also includes stored rules for clustering and rule-based formation of token-to-chunk phrase chunks and sub-phrases, including grammar-based chunking rules expressed as binary/trinary productions and type-conversion productions such as NNPG->NNP and NNPP->NNP.
Claims Coverage
The independent claims are directed to a system, a computer-implemented method, and a non-transitory computer-readable medium, each covering the same core pipeline. Across the independent claims, four main inventive features are consistently present: noun-tag tokenization, noun-tag-based clustering into token chunks, sub-phrase formation from clustered tokens, and performing LSA using the sub-phrases, with additional tag and rule refinements specified in dependents.
Noun-tagged token creation for an accessed phrase
access a phrase comprising a plurality of nouns; create a plurality of tokens, each token comprising one of the plurality of nouns and an associated noun tag of the plurality of noun tags;
Noun-tag-based token chunk clustering
cluster one or more of the created tokens into a chunk of tokens, the one or more tokens clustered into the chunk of tokens based on the noun tags of the one or more tokens;
Sub-phrase formation from clustered token nouns
form one or more sub-phrases of the accessed phrase based on the chunk of tokens, the one or more sub-phrases comprising the nouns of the one or more tokens clustered into the chunk of tokens;
LSA performed using the sub-phrases
perform Latent Semantic Analysis (LSA) using the one or more sub-phrases;
The independent claims define a process that discovers latent relationships by converting noun phrases into noun-tagged tokens, clustering tokens into chunks based on noun tags, generating sub-phrases from clustered token nouns, and then performing LSA using those sub-phrases. Dependent claims further specify noun-tag coverage for proper nouns, including geographic and person-name overlap, and stored rule-based clustering and noun replacement rules.
Stated Advantages
Improves phrase identification compared with n-gram approaches.
Reduces the candidate phrase count compared with n-gram approaches.
Improves downstream LSA results.
Documented Applications
Latent Semantic Analysis (LSA) for discovering latent relationships in data using sub-phrases derived from an accessed phrase.
Interested in licensing this patent?