Indata analysis,cosine similarity is ameasure of similarity between two non-zero vectors defined in aninner product space. Cosine similarity is thecosine of the angle between the vectors; that is, it is thedot product of the vectors divided by the product of their lengths. It follows that the cosine similarity does not depend on the magnitudes of the vectors, but only on their angle. The cosine similarity always belongs to the interval${\displaystyle [-1,+1].}$ For example, twoproportional vectors have a cosine similarity of +1, twoorthogonal vectors have a similarity of 0, and twoopposite vectors have a similarity of −1. In some contexts, the component values of the vectors cannot be negative, in which case the cosine similarity is bounded in${\displaystyle [0,1]}$.

For example, ininformation retrieval andtext mining, each word is assigned a different coordinate and a document is represented by the vector of the numbers of occurrences of each word in the document. Cosine similarity then gives a useful measure of how similar two documents are likely to be, in terms of their subject matter, and independently of the length of the documents.^[1]

The technique is also used to measure cohesion within clusters in the field ofdata mining.^[2]

One advantage of cosine similarity is itslow complexity, especially forsparse vectors: only the non-zero coordinates need to be considered.

Other names for cosine similarity includeOrchini similarity andTucker coefficient of congruence; theOtsuka–Ochiai similarity (see below) is cosine similarity applied tobinary data.^[3]

The cosine of two non-zero vectors can be derived by using theEuclidean dot product formula:

${\displaystyle \mathbf{A}\cdot \mathbf{B}=\Vert \mathbf{A}\Vert \Vert \mathbf{B}\Vert \cos \theta }$

Given twon-dimensionalvectors of attributes,A andB, the cosine similarity,cos(θ), is represented using adot product andmagnitude as

${\displaystyle \text{cosine similarity}=S_C(A,B):=\cos (\theta )=\frac{\mathbf{A}\cdot \mathbf{B}}{\Vert \mathbf{A}\Vert \Vert \mathbf{B}\Vert }=\frac{\sum _{i=1}^n{A}_i{B}_i}{\sqrt{\sum _{i=1}^n{A}_i^2}\cdot \sqrt{\sum _{i=1}^n{B}_i^2}},}$

where${\displaystyle A_i}$ and${\displaystyle B_i}$ are the${\displaystyle i}$thcomponents of vectors${\displaystyle \mathbf{A}}$ and${\displaystyle \mathbf{B}}$, respectively.

The resulting similarity ranges from −1 meaning exactly opposite, to +1 meaning exactly the same, with 0 indicatingorthogonality ordecorrelation, while in-between values indicate intermediate similarity or dissimilarity.

Fortext matching, the attribute vectorsA andB are usually theterm frequency vectors of the documents. Cosine similarity can be seen as a method ofnormalizing document length during comparison. In the case ofinformation retrieval, the cosine similarity of two documents will range from${\displaystyle 0\to 1}$, since the term frequencies cannot be negative. This remains true when usingTF-IDF weights. The angle between two term frequency vectors cannot be greater than 90°.

If the attribute vectors are normalized by subtracting the vector means (e.g.,${\displaystyle A-\overline{A}}$), the measure is called the centered cosine similarity and is equivalent to thePearson correlation coefficient. For an example of centering,

${\displaystyle \text{if}A=[A_1,A_2{]}^T,\text{ then }\overline{A}={\left[\frac{(A_1+A_2)}{2},\frac{(A_1+A_2)}{2}\right]}^T,}$

${\displaystyle \text{ so }A-\overline{A}={\left[\frac{(A_1-A_2)}{2},\frac{(-A_1+A_2)}{2}\right]}^T.}$

When the distance between two unit-length vectors is defined to be the length of their vector difference then$${\displaystyle \mathrm{dist}(\mathbf{A},\mathbf{B})=\sqrt{(\mathbf{A}-\mathbf{B})\cdot (\mathbf{A}-\mathbf{B})}=\sqrt{\mathbf{A}\cdot \mathbf{A}-2(\mathbf{A}\cdot \mathbf{B})+\mathbf{B}\cdot \mathbf{B}}=\sqrt{2(1-S_C(\mathbf{A},\mathbf{B}))}.}$$

Nonetheless thecosine distance^[4] is often defined without the square root or factor of 2:

${\displaystyle \text{cosine distance}=D_C(A,B):=1-S_C(A,B).}$

It is important to note that, by virtue of being proportional to squared Euclidean distance, the cosine distance is not a truedistance metric; it does not exhibit thetriangle inequality property — or, more formally, theSchwarz inequality — and it violates the coincidence axiom. To repair the triangle inequality property while maintaining the same ordering, one can convert toEuclidean distance$\sqrt{2(1-S_C(A,B))}$ or angular distanceθ = arccos(S_C(A,B)). Alternatively, the triangular inequality that does work for angular distances can be expressed directly in terms of the cosines; seebelow.

The normalized angle, referred to asangular distance, between any two vectors${\displaystyle A}$ and${\displaystyle B}$ is a formaldistance metric and can be calculated from the cosine similarity.^[5] The complement of the angular distance metric can then be used to defineangular similarity function bounded between 0 and 1, inclusive.

When the vector elements may be positive or negative:

${\displaystyle \text{angular distance}=D_{\theta }:=\frac{\arccos (\text{cosine similarity})}{\pi }=\frac{\theta }{\pi }}$

${\displaystyle \text{angular similarity}=S_{\theta }:=1-\text{angular distance}=1-\frac{\theta }{\pi }}$

Or, if the vector elements are always positive:

${\displaystyle \text{angular distance}=D_{\theta }:=\frac{2\cdot \arccos (\text{cosine similarity})}{\pi }=\frac{2\theta }{\pi }}$

${\displaystyle \text{angular similarity}=S_{\theta }:=1-\text{angular distance}=1-\frac{2\theta }{\pi }}$

Unfortunately, computing the inverse cosine (arccos) function is slow, making the use of the angular distance more computationally expensive than using the more common (but not metric) cosine distance above.

Another effective proxy for cosine distance can be obtained by${\displaystyle L_2}$ normalisation of the vectors, followed by the application of normalEuclidean distance. Using this technique each term in each vector is first divided by the magnitude of the vector, yielding a vector of unit length. Then the Euclidean distance over the end-points of any two vectors is a proper metric which gives the same ordering as the cosine distance (amonotonic transformation of Euclidean distance; seebelow) for any comparison of vectors, and furthermore avoids the potentially expensive trigonometric operations required to yield a proper metric. Once the normalisation has occurred, the vector space can be used with the full range of techniques available to any Euclidean space, notably standarddimensionality reduction techniques. This normalised form distance is often used within manydeep learning algorithms.

In biology, there is a similar concept known as the Otsuka–Ochiai coefficient named afterYanosuke Otsuka (also spelled as Ōtsuka, Ootsuka or Otuka,^[6]Japanese:大塚 弥之助)^[7] and Akira Ochiai (Japanese:落合 明),^[8] also known as the Ochiai–Barkman^[9] or Ochiai coefficient,^[10] which can be represented as:

${\displaystyle K=\frac{|A\cap B|}{\sqrt{|A|\times |B|}}}$

Here,${\displaystyle A}$ and${\displaystyle B}$ aresets, and${\displaystyle |A|}$ is the number of elements in${\displaystyle A}$. If sets are represented as bit vectors, the Otsuka–Ochiai coefficient can be seen to be the same as the cosine similarity. It is identical to the score introduced byGodfrey Thomson.^[11]

In a recent book,^[12] the coefficient is tentatively misattributed to another Japanese researcher with the family name Otsuka. The confusion arises because in 1957 Akira Ochiai attributes the coefficient only to Otsuka (no first name mentioned)^[8] by citing an article by Ikuso Hamai (Japanese:浜井 生三),^[13] who in turn cites the original 1936 article by Yanosuke Otsuka.^[7]

The most noteworthy property of cosine similarity is that it reflects a relative, rather than absolute, comparison of the individual vector dimensions. For any positive constant${\displaystyle a}$ and vector${\displaystyle V}$, the vectors${\displaystyle V}$ and${\displaystyle aV}$ are maximally similar. The measure is thus most appropriate for data where frequency is more important than absolute values; notably, term frequency in documents.However more recent metrics with a grounding in information theory, such asJensen–Shannon, SED, and triangular divergence have been shown to have improved semantics in at least some contexts.^[14]

Cosine similarity is related toEuclidean distance as follows. Denote Euclidean distance by the usual${\displaystyle \Vert A-B\Vert }$, and observe that

${\displaystyle \Vert A-B{\Vert }^2=(A-B)\cdot (A-B)=\Vert A{\Vert }^2+\Vert B{\Vert }^2-2(A\cdot B)}$ (polarization identity)

byexpansion. WhenA andB are normalized to unit length,${\displaystyle \Vert A{\Vert }^2=\Vert B{\Vert }^2=1}$ so this expression is equal to

${\displaystyle 2(1-\cos (A,B)).}$

In short, the cosine distance can be expressed in terms of Euclidean distance as

${\displaystyle D_C(A,B)=\frac{\Vert A-B{\Vert }^2}{2}\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{n}\Vert A{\Vert }^2=\Vert B{\Vert }^2=1}$.

The Euclidean distance is called thechord distance (because it is the length of the chord on the unit circle) and it is the Euclidean distance between the vectors which were normalized to unit sum of squared values within them.

Null distribution: For data which can be negative as well as positive, thenull distribution for cosine similarity is the distribution of thedot product of two independent randomunit vectors. This distribution has amean of zero and avariance of${\displaystyle 1/n}$ (where${\displaystyle n}$ is the number of dimensions), and although the distribution is bounded between −1 and +1, as${\displaystyle n}$ grows large the distribution is increasingly well-approximated by thenormal distribution.^[15][16] Other types of data such asbitstreams, which only take the values 0 or 1, the null distribution takes a different form and may have a nonzero mean.^[17]

The ordinarytriangle inequality for angles (i.e., arc lengths on a unit hypersphere) gives us that

${\displaystyle |\mathrm{\angle }AC-\mathrm{\angle }CB|\le \mathrm{\angle }AB\le \mathrm{\angle }AC+\mathrm{\angle }CB.}$

Because the cosine function decreases as an angle in[0,π] radians increases, the sense of these inequalities is reversed when we take the cosine of each value:

${\displaystyle \cos (\mathrm{\angle }AC-\mathrm{\angle }CB)\ge \cos (\mathrm{\angle }AB)\ge \cos (\mathrm{\angle }AC+\mathrm{\angle }CB).}$

Using the cosine addition and subtraction formulas, these two inequalities can be written in terms of the original cosines,

${\displaystyle \cos (A,C)\cdot \cos (C,B)+\sqrt{\left(1-\cos (A,C{)}^2\right)\cdot \left(1-\cos (C,B{)}^2\right)}\ge \cos (A,B),}$

${\displaystyle \cos (A,B)\ge \cos (A,C)\cdot \cos (C,B)-\sqrt{\left(1-\cos (A,C{)}^2\right)\cdot \left(1-\cos (C,B{)}^2\right)}.}$

This form of the triangle inequality can be used to bound the minimum and maximum similarity of two objects A and B if the similarities to a reference object C is already known. This is used for example in metric data indexing, but has also been used to accelerate sphericalk-means clustering^[18] the same way the Euclidean triangle inequality has been used to accelerate regular k-means.

A soft cosine or ("soft" similarity) between two vectors considers similarities between pairs of features.^[19] The traditional cosine similarity considers thevector space model (VSM) features as independent or completely different, while the soft cosine measure proposes considering the similarity of features in VSM, which help generalize the concept of cosine (and soft cosine) as well as the idea of (soft) similarity.

For example, in the field ofnatural language processing (NLP) the similarity among features is quite intuitive. Features such as words,n-grams, or syntacticn-grams^[20] can be quite similar, though formally they are considered as different features in the VSM. For example, words "play" and "game" are different words and thus mapped to different points in VSM; yet they are semantically related. In case ofn-grams or syntacticn-grams,Levenshtein distance can be applied (in fact, Levenshtein distance can be applied to words as well).

For calculating soft cosine, the matrixs is used to indicate similarity between features. It can be calculated through Levenshtein distance,WordNet similarity, or othersimilarity measures. Then we just multiply by this matrix.

Given twoN-dimension vectors${\displaystyle a}$ and${\displaystyle b}$, the soft cosine similarity is calculated as follows:

${\displaystyle \begin{array}{r}{\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{\_}\mathrm{c}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{e}}_1(a,b)=\frac{{\sum }_{i,j}^N{s}_{ij}{a}_i{b}_j}{\sqrt{{\sum }_{i,j}^N{s}_{ij}{a}_i{a}_j}\sqrt{{\sum }_{i,j}^N{s}_{ij}{b}_i{b}_j}},\end{array}}$

wheres_ij = similarity(feature_i, feature_j).

If there is no similarity between features (s_ii = 1,s_ij = 0 fori ≠j), the given equation is equivalent to the conventional cosine similarity formula.

Thetime complexity of this measure is quadratic, which makes it applicable to real-world tasks. Note that the complexity can be reduced to subquadratic.^[21] An efficient implementation of such soft cosine similarity is included in theGensim open source library.

• Sørensen–Dice coefficient
• Hamming distance
• Correlation
• Jaccard index
• SimRank
• Information retrieval

• Weighted cosine measure
• A tutorial on cosine similarity using Python

• Information retrieval techniques
• Similarity measures
• Data analysis

• CS1 maint: numeric names: authors list
• Articles with short description
• Short description is different from Wikidata
• Articles containing Japanese-language text