Lecture
Vector space model (VSM) — in information retrieval, a representation of a document collection as vectors from a single vector space shared by the whole collection.
The vector space model is the foundation for solving many information retrieval tasks, such as: document retrieval by query, document classification, and document clustering.
In the vector space model, a document is treated as an unordered set of terms. In information retrieval, terms are the words that make up the text, as well as such text elements as, for example, 2010, II-5, or Tian Shan.
The weight of a term in a document — the «importance» of a word for identifying the given text — can be defined in various ways. For example, one can simply count the number of occurrences of a term in a document, the so-called term frequency — the more often a word occurs in a document, the greater its weight will be. If a term does not occur in a document, its weight in that document equals zero.
All the terms occurring in the documents of the collection being processed can be ordered. If, for some document, we now write out in order the weights of all terms, including those absent from that document, we get a vector, which is the representation of that document in the vector space. The dimensionality of this vector, like the dimensionality of the space, equals the number of distinct terms in the whole collection, and is the same for all documents.
More formally
dj = (w1j, w2j, …, wnj)
where dj — is the vector representation of the j-th document, wij — is the weight of the i-th term in the j-th document, n — is the total number of distinct terms across all documents in the collection.
Having such a representation for all documents, one can, for example, find the distance between points of the space and thereby solve the problem of document similarity — the closer the points are located, the more similar the corresponding documents are. In the case of retrieving a document by query, the query is likewise represented as a vector of the same space — and the relevance of documents to the query can be computed.
To fully define the vector space model, it is necessary to specify exactly how the weight of a term in a document will be found. There are several standard ways of defining the weighting function:
Cosine similarity — is a measure of similarity between two vectors of a pre-Hilbert space, used to measure the cosine of the angle between them.
If two feature vectors, A and B, are given, then the cosine similarity, cos(θ), can be represented using the dot product and the norm:
In the case of information retrieval, the cosine similarity of two documents ranges from 0 to 1, since the term frequency (tf-idf weights) cannot be negative. The angle between two term-frequency vectors cannot be greater than 90°.
One of the reasons for the popularity of cosine similarity is that it is efficient as an evaluation measure, especially for sparse vectors, since only the nonzero dimensions need to be taken into account.
the theory of document similarity, by comparing the deviation of the angles between each document vector and the original query vector, where the query is represented as a vector of the same dimensionality as the vectors representing the other documents.
In practice it is easier to compute the cosine of the angle between vectors than the angle itself:
where 
is the intersection (i.e., the dot product ) of the document vector (d 2 in the figure on the right) and the query vector (q in the figure),
- is the norm of vector d 2 , and
- is the norm of vector q. The norm of a vector is computed as follows:
Using the cosine, the similarity between document d j and query q can be calculated as:
Since all vectors considered in this model are non-negative in every element, a cosine value of zero means that the query vector and the document vector are orthogonal and have no overlap (i.e., the query term does not exist in the document under consideration). See cosine similarity for more information.
«Soft» cosine measure — is a «soft» measure of similarity between two vectors, that is, a measure that takes into account the similarities between pairs of features. Traditional cosine similarity treats the features of the vector model as independent, or completely separate, whereas the «soft» cosine measure takes into account the similarities of features in the vector model. This makes it possible to generalize the idea of the cosine measure, as well as the idea of similarity between objects in a vector space («soft» similarity).
For example, in the field of natural language processing, the similarity between objects is quite intuitive. Features such as words, N-grams, or syntactic N-grams can be quite similar, although formally they are considered distinct features in the vector model. For example, the words «играть» (to play) and «игра» (game) are distinct and are thus mapped to different dimensions in the vector model, even though they are obviously related semantically. In the case of N-grams or syntactic N-grams, the Levenshtein distance can be applied (moreover, the Levenshtein distance can also be applied to words).
To compute the «soft» cosine measure, a similarity matrix s between features is introduced. It can be calculated using the Levenshtein distance or other similarity measures, for example various similarity measures in Wordnet. Multiplication is then performed using this matrix.
Given two N-dimensional vectors a and b, the soft cosine measure is calculated as follows:
where sij = similarity(featurei, featurej).
In the absence of similarity between features (sii = 1, sij = 0 for i ≠ j)), this equation is equivalent to the standard cosine similarity formula.
The complexity of this measure is quadratic, which makes it quite applicable to real-world problems. The complexity can also be transformed into linear.
In the classic vector space model proposed by Salton , Wong , and Yang , the term weights in document vectors are products of local and global parameters. This model is known as the term frequency–inverse document frequency model . The weight vector for document d equals
, where
and
the frequency of term t in document d (local parameter)
- inverse document frequency (global parameter). 
- the total number of documents in the document set; 
- the number of documents containing the term t .The vector space model has the following advantages over the standard Boolean model :
Most of these advantages result from the difference in representation density of the document collection between the Boolean approach and the inverse-document-frequency approach. When using Boolean weights, any document lies at a vertex of an n-dimensional hypercube . Hence, the possible document representations are:
and the maximum Euclidean distance between pairs equals 
. As documents are added to the document collection, the region defined by the vertices of the hypercube becomes more populated and, consequently, denser. Unlike the Boolean value, when a document is added using inverse-document-frequency weights, the inverse term frequencies of the new document decrease, while the frequency of the remaining terms increases. On average, as documents are added, the region of document placement expands, regulating the representation density of the whole collection. This behavior models the original motivation of Salton and his colleagues, according to which a document collection represented in a low-density region can yield better retrieval results.
The vector space model has the following limitations:
However, many of these difficulties can be overcome by integrating various tools, including mathematical methods such as singular value decomposition, and lexical databases such as WordNet .
Models based on and extending the vector space model include:
The following software packages may be of interest to those who want to experiment with vector models and build search services based on them.
Comments