Repository files navigation

A library for finding occurrences of many patterns at once with SIMDacceleration in some cases. This library provides multiple patternsearch principally through an implementation of theAho-Corasick algorithm,which builds a finite state machine for executing searches in linear time.Features include case insensitive matching, overlapping matches, fast searchingvia SIMD and optional full DFA construction and search & replace in streams.

Dual-licensed under MIT or theUNLICENSE.

https://docs.rs/aho-corasick

Runcargo add aho-corasick to automatically add this crate as a dependencyin yourCargo.toml file.

This example shows how to search for occurrences of multiple patternssimultaneously. Each match includes the pattern that matched along with thebyte offsets of the match.

use aho_corasick::{AhoCorasick,PatternID};let patterns =&["apple","maple","Snapple"];let haystack ="Nobody likes maple in their apple flavored Snapple.";let ac =AhoCorasick::new(patterns).unwrap();letmut matches =vec![];for matin ac.find_iter(haystack){    matches.push((mat.pattern(), mat.start(), mat.end()));}assert_eq!(matches, vec![(PatternID::must(1),13,18),(PatternID::must(0),28,33),(PatternID::must(2),43,50),]);

This is like the previous example, but matchesSnapple case insensitivelyusingAhoCorasickBuilder:

use aho_corasick::{AhoCorasick,PatternID};let patterns =&["apple","maple","snapple"];let haystack ="Nobody likes maple in their apple flavored Snapple.";let ac =AhoCorasick::builder().ascii_case_insensitive(true).build(patterns).unwrap();letmut matches =vec![];for matin ac.find_iter(haystack){    matches.push((mat.pattern(), mat.start(), mat.end()));}assert_eq!(matches, vec![(PatternID::must(1),13,18),(PatternID::must(0),28,33),(PatternID::must(2),43,50),]);

This example shows how to execute a search and replace on a stream withoutloading the entire stream into memory first.

use aho_corasick::AhoCorasick;let patterns =&["fox","brown","quick"];let replace_with =&["sloth","grey","slow"];// In a real example, these might be `std::fs::File`s instead. All you need to// do is supply a pair of `std::io::Read` and `std::io::Write` implementations.let rdr ="The quick brown fox.";letmut wtr =vec![];let ac =AhoCorasick::new(patterns).unwrap();ac.stream_replace_all(rdr.as_bytes(),&mut wtr, replace_with).expect("stream_replace_all failed");assert_eq!(b"The slow grey sloth.".to_vec(), wtr);

In the textbook description of Aho-Corasick, its formulation is typicallystructured such that it reports all possible matches, even when they overlapwith another. In many cases, overlapping matches may not be desired, such asthe case of finding all successive non-overlapping matches like you might witha standard regular expression.

Unfortunately the "obvious" way to modify the Aho-Corasick algorithm to dothis doesn't always work in the expected way, since it will report matches assoon as they are seen. For example, consider matching the regexSamwise|Samagainst the textSamwise. Most regex engines (that are Perl-like, ornon-POSIX) will reportSamwise as a match, but the standard Aho-Corasickalgorithm modified for reporting non-overlapping matches will reportSam.

A novel contribution of this library is the ability to change the matchsemantics of Aho-Corasick (without additional search time overhead) such thatSamwise is reported instead. For example, here's the standard approach:

use aho_corasick::AhoCorasick;let patterns =&["Samwise","Sam"];let haystack ="Samwise";let ac =AhoCorasick::new(patterns).unwrap();let mat = ac.find(haystack).expect("should have a match");assert_eq!("Sam",&haystack[mat.start()..mat.end()]);

And now here's the leftmost-first version, which matches how a Perl-likeregex will work:

use aho_corasick::{AhoCorasick,MatchKind};let patterns =&["Samwise","Sam"];let haystack ="Samwise";let ac =AhoCorasick::builder().match_kind(MatchKind::LeftmostFirst).build(patterns).unwrap();let mat = ac.find(haystack).expect("should have a match");assert_eq!("Samwise",&haystack[mat.start()..mat.end()]);

In addition to leftmost-first semantics, this library also supportsleftmost-longest semantics, which match the POSIX behavior of a regularexpression alternation. SeeMatchKind in the docs for more details.

This crate's minimum supportedrustc version is1.60.0.

The current policy is that the minimum Rust version required to use this cratecan be increased in minor version updates. For example, ifcrate 1.0 requiresRust 1.20.0, thencrate 1.0.z for all values ofz will also require Rust1.20.0 or newer. However,crate 1.y fory > 0 may require a newer minimumversion of Rust.

In general, this crate will be conservative with respect to the minimumsupported version of Rust.

• G-Research/ahocorasick_rsis a Python wrapper for this library.
• tmikus/ahocorasick_rs is a Gowrapper for this library.