Lecture
The vector space model (VSM), or term vector model, is an algebraic model for representing text documents (or, more generally, items) as vectors, where the distance between vectors represents the relevance between documents. It is used in information filtering, information retrieval, indexing, and relevance ranking. Its first application was in the SMART Information Retrieval System.
In this section we consider a specific vector space model based on the “bag of words” representation. Documents and queries are represented as vectors.
Each dimension corresponds to a separate term. If a term occurs in a document, its value in the vector is non-zero. Several different ways of computing these values, also known as (term) weights, have been devised. One of the best-known schemes is tf–idf weighting (see the example below).
The definition of a term depends on the application. Usually terms are single words, keywords, or longer phrases. If words are chosen as terms, the dimensionality of the vector equals the number of words in the vocabulary (the number of distinct words occurring in the corpus).
Vector operations can be used to compare documents with queries.
The search and ranking of candidate documents from a corpus can be carried out by various methods. The relevance ranking of documents in keyword search can be calculated using the assumptions of document similarity theory by comparing the deviation of the angle between each document's vector and the original query vector, where the query is represented as a vector of the same dimensionality as the vectors representing the other documents.
In practice it is easier to compute the cosine of the angle between the vectors than the angle itself:
Where
represents the intersection (i.e. the dot product) of the document (d 2 in the figure on the right) and query (q in the figure) vectors.
is the norm of vector d² , and
— is the norm of vector q. The norm of a vector is calculated as follows:
Using the cosine, the similarity between document dj and query q can be calculated as follows:
Since all vectors considered in this model are element-wise non-negative, a cosine value of zero means that the query vector and the document vector are orthogonal and have no matches (i.e. the query term is absent from the document under consideration). See the “Cosine similarity” section for more information.
In the classical vector space model proposed by Salton, Wong, and Yang, term-specific weights in document vectors are products of local and global parameters. This model is known as the term frequency–inverse document frequency model (tf–idf). The weight vector for document d equals
, where
and
— the frequency of occurrence of term t in document d (local parameter).
is the inverse document frequency (global parameter).
— the total number of documents in the document set;
— the number of documents containing term t.The vector space model has the following advantages over the standard Boolean model:
Most of these advantages follow from the difference in representation density of a document collection between the Boolean approach and the approach based on term frequency and inverse document frequency. When Boolean weights are used, any document lies at a vertex of an n-dimensional hypercube. Consequently, the possible document representations are as follows:
and the maximum Euclidean distance between pairs equals
As documents are added to the collection, the region defined by the vertices of the hypercube becomes more filled and, consequently, denser. Unlike Boolean operators, when a document is added using weights based on term frequency and inverse document frequency, the inverse frequencies of terms in the new document decrease, while the frequencies of the remaining terms increase. On average, as documents are added, the region in which the documents lie expands, regulating the density of the entire collection representation. This behavior models the original motivation of Salton and his colleagues, who believed that a document collection represented in a low-density region could yield better search results.
The vector space model has the following limitations:
However, many of these difficulties can be overcome by integrating various tools, including mathematical methods such as singular value decomposition, and lexical databases such as WordNet.
Models based on and extending the vector space model include:
The following software packages may be of interest to those who want to experiment with vector models and implement search services based on them.
The generalized vector space model is a generalization of the vector space model used in information retrieval. Wong et al. presented an analysis of the problems created by the pairwise orthogonality assumption of the vector space model. From this, they extended the vector space model to the generalized vector space model (GVSM).
Recently, Tsatsaronis focused on the first approach. They measure semantic relatedness (SR) using a thesaurus (O) similar to WordNet. It takes into account path length, determined by path compactness (SCM), and path depth, determined by semantic path specificity (SPE).
Also building on the first approach, Waitelonis et al. computed semantic relatedness based on Linked Open Data resources, including DBpedia, as well as the YAGO taxonomy. In doing so, they use taxonomic relations between semantic entities in documents and queries after named entity linking.
Comments