Repository files navigation

TextDistance -- python library for compare distance between two or more sequences by many algorithms.

Features:

• 30+ algorithms
• Pure python implementation
• Simple usage
• More than two sequences comparing
• Some algorithms have more than one implementation in one class.
• Optional numpy usage for maximum speed.

Work in progress. Now all algorithms compare two strings as array of bits, not by chars.

NCD - normalized compression distance.

Functions:

1. bz2_ncd
2. lzma_ncd
3. arith_ncd
4. rle_ncd
5. bwtrle_ncd
6. zlib_ncd

Stable:

pip install textdistance

Dev:

pip install -e git+https://github.com/orsinium/textdistance.git#egg=textdistance

All algorithms have 2 interfaces:

1. Class with algorithm-specific params for customizing.
2. Class instance with default params for quick and simple usage.

All algorithms have some common methods:

1. .distance(*sequences) -- calculate distance between sequences.
2. .similarity(*sequences) -- calculate similarity for sequences.
3. .maximum(*sequences) -- maximum possible value for distance and similarity. For any sequence:distance + similarity == maximum.
4. .normalized_distance(*sequences) -- normalized distance between sequences. The return value is a float between 0 and 1, where 0 means equal, and 1 totally different.
5. .normalized_similarity(*sequences) -- normalized similarity for sequences. The return value is a float between 0 and 1, where 0 means totally different, and 1 equal.

Most common init arguments:

1. qval -- q-value for split sequences into q-grams. Possible values:
   • 1 (default) -- compare sequences by chars.
   • 2 or more -- transform sequences to q-grams.
   • None -- split sequences by words.
2. as_set -- for token-based algorithms:
   • True --t andttt is equal.
   • False (default) --t andttt is different.

For example,Hamming distance:

importtextdistancetextdistance.hamming('test','text')# 1textdistance.hamming.distance('test','text')# 1textdistance.hamming.similarity('test','text')# 3textdistance.hamming.normalized_distance('test','text')# 0.25textdistance.hamming.normalized_similarity('test','text')# 0.75textdistance.Hamming(qval=2).distance('test','text')# 2

Any other algorithms have same interface.