Cosine Similarity; For checking similarity of documents, etc.
Cosine similarity is a measure of checking the similarity between two documents, texts, strings, etc.
It does so by representing the query as vectors in n-dimensional space. It then measures the angle between these vectors and gives the similarity based on the cosine of this angle.
If the queries are completely similar the angle will be zero; Thus the cosine similarity will be: > cos(angle_between_the _vectors)=cos(0)= 1
If the queries are completely dissimilar the vectors will be perpendicular; Thus the cosine similarity will be: > cos(angle_between_the _vectors)=cos(90)= 0
If the queries are completely opposite the vectors will be opposite to each other; Thus the cosine similarity will be: > cos(angle_between_the _vectors)=cos(180)= -1
The cosine similarity, mathematically, is given by:
Let's see an example:
Doc1 = "this is the first document" Doc2 = "this document is second in this order"
Vector representation of these documents: Doc1 = [1,1,1,0,1,1,0,0] Doc2 = [1,0,1,1,1,2,1,1]
ΣAiBi = (1*1)+(1*0)+(0*1)+(1*1)+(0*1)+(0*1)+(1*0)+(1*2) = 4 √(ΣAi)^2 = √(1+1+0+1+0+0+1+1) = √5 √(ΣBi)^2 = √(1+0+1+1+1+1+0+4) = √9
Cosine similarity = 4/(√5*√9) = 0.59
The Cosine Similarity is a better metric than Euclidean distance because if the two text document far apart by Euclidean distance, there are still chances that they are close to each other in terms of their context.