Repository files navigation

This crate provides routines for searching strings for matches of aregularexpression (aka "regex"). The regex syntax supported by this crate is similarto other regex engines, but it lacks several features that are not known how toimplement efficiently. This includes, but is not limited to, look-around andbackreferences. In exchange, all regex searches in this crate have worst caseO(m * n) time complexity, wherem is proportional to the size of the regexandn is proportional to the size of the string being searched.

Module documentation with examples.The module documentation also includes a comprehensive description of thesyntax supported.

Documentation with examples for the various matching functions and iteratorscan be found on theRegex type.

To bring this crate into your repository, either addregex to yourCargo.toml, or runcargo add regex.

Here's a simple example that matches a date in YYYY-MM-DD format and prints theyear, month and day:

use regex::Regex;fnmain(){let re =Regex::new(r"(?x)(?P<year>\d{4})  # the year-(?P<month>\d{2}) # the month-(?P<day>\d{2})   # the day").unwrap();let caps = re.captures("2010-03-14").unwrap();assert_eq!("2010",&caps["year"]);assert_eq!("03",&caps["month"]);assert_eq!("14",&caps["day"]);}

If you have lots of dates in text that you'd like to iterate over, then it'seasy to adapt the above example with an iterator:

use regex::Regex;fnmain(){let re =Regex::new(r"(\d{4})-(\d{2})-(\d{2})").unwrap();let hay ="On 2010-03-14, foo happened. On 2014-10-14, bar happened.";letmut dates =vec![];for(_,[year, month, day])in re.captures_iter(hay).map(|c| c.extract()){        dates.push((year, month, day));}assert_eq!(dates, vec![("2010","03","14"),("2014","10","14"),]);}

It is an anti-pattern to compile the same regular expression in a loop sincecompilation is typically expensive. (It takes anywhere from a few microsecondsto a fewmilliseconds depending on the size of the regex.) Not only iscompilation itself expensive, but this also prevents optimizations that reuseallocations internally to the matching engines.

In Rust, it can sometimes be a pain to pass regular expressions around ifthey're used from inside a helper function. Instead, we recommend using theonce_cell crate to ensure thatregular expressions are compiled exactly once. For example:

use{    once_cell::sync::Lazy,    regex::Regex,};fnsome_helper_function(haystack:&str) ->bool{staticRE:Lazy<Regex> =Lazy::new(||Regex::new(r"...").unwrap());RE.is_match(haystack)}fnmain(){assert!(some_helper_function("abc"));assert!(!some_helper_function("ac"));}

Specifically, in this example, the regex will be compiled when it is used forthe first time. On subsequent uses, it will reuse the previous compilation.

The main API of this crate (regex::Regex) requires the caller to pass a&str for searching. In Rust, an&str is required to be valid UTF-8, whichmeans the main API can't be used for searching arbitrary bytes.

To match on arbitrary bytes, use theregex::bytes::Regex API. The API isidentical to the main API, except that it takes an&[u8] to search on insteadof an&str. The&[u8] APIs also permit disabling Unicode mode in the regexeven when the pattern would match invalid UTF-8. For example,(?-u:.) isnot allowed inregex::Regex but is allowed inregex::bytes::Regex since(?-u:.) matches any byte except for\n. Conversely,. will match theUTF-8 encoding of any Unicode scalar value except for\n.

This example shows how to find all null-terminated strings in a slice of bytes:

use regex::bytes::Regex;let re =Regex::new(r"(?-u)(?<cstr>[^\x00]+)\x00").unwrap();let text =b"foo\xFFbar\x00baz\x00";// Extract all of the strings without the null terminator from each match.// The unwrap is OK here since a match requires the `cstr` capture to match.let cstrs:Vec<&[u8]> =    re.captures_iter(text).map(|c| c.name("cstr").unwrap().as_bytes()).collect();assert_eq!(vec![&b"foo\xFFbar"[..],&b"baz"[..]], cstrs);

Notice here that the[^\x00]+ will match anybyte except forNUL,including bytes like\xFF which are not valid UTF-8. When using the main API,[^\x00]+ would instead match any valid UTF-8 sequence except forNUL.

This demonstrates how to use aRegexSet to match multiple (possiblyoverlapping) regular expressions in a single scan of the search text:

use regex::RegexSet;let set =RegexSet::new(&[r"\w+",r"\d+",r"\pL+",r"foo",r"bar",r"barfoo",r"foobar",]).unwrap();// Iterate over and collect all of the matches.let matches:Vec<_> = set.matches("foobar").into_iter().collect();assert_eq!(matches, vec![0,2,3,4,6]);// You can also test whether a particular regex matched:let matches = set.matches("foobar");assert!(!matches.matched(5));assert!(matches.matched(6));

Theregex-automata directory contains a crate thatexposes all of the internal matching engines used by theregex crate. Theidea is that theregex crate exposes a simple API for 99% of use cases, butregex-automata exposes oodles of customizable behaviors.

Documentation forregex-automata.

This repository contains a crate that provides a well tested regular expressionparser, abstract syntax and a high-level intermediate representation forconvenient analysis. It provides no facilities for compilation or execution.This may be useful if you're implementing your own regex engine or otherwiseneed to do analysis on the syntax of a regular expression. It is otherwise notrecommended for general use.

Documentation forregex-syntax.

This crate comes with several features that permit tweaking the trade offbetween binary size, compilation time and runtime performance. Users of thiscrate can selectively disable Unicode tables, or choose from a variety ofoptimizations performed by this crate to disable.

When all of these features are disabled, runtime match performance may be muchworse, but if you're matching on short strings, or if high performance isn'tnecessary, then such a configuration is perfectly serviceable. To disableall such features, use the followingCargo.toml dependency configuration:

[dependencies.regex]version ="1.3"default-features =false# Unless you have a specific reason not to, it's good sense to enable standard# library support. It enables several optimizations and avoids spin locks. It# also shouldn't meaningfully impact compile times or binary size.features = ["std"]

This will reduce the dependency tree ofregex down to two crates:regex-syntax andregex-automata.

The full set of features one can disable arein the "Crate features" section of the documentation.

One of the goals of this crate is for the regex engine to be "fast." What thatis a somewhat nebulous goal, it is usually interpreted in one of two ways.First, it means that all searches take worst caseO(m * n) time, wherem is proportional tolen(regex) andn is proportional tolen(haystack).Second, it means that even aside from the time complexity constraint, regexsearches are "fast" in practice.

While the first interpretation is pretty unambiguous, the second one remainsnebulous. While nebulous, it guides this crate's architecture and the sorts ofthe trade offs it makes. For example, here are some general architecturalstatements that follow as a result of the goal to be "fast":

• When given the choice between faster regex searches and fasterRust compiletimes, this crate will generally choose faster regex searches.
• When given the choice between faster regex searches and fasterregex compiletimes, this crate will generally choose faster regex searches. That is, it isgenerally acceptable forRegex::new to get a little slower if it means thatsearches get faster. (This is a somewhat delicate balance to strike, becausethe speed ofRegex::new needs to remain somewhat reasonable. But this is whyone should avoid re-compiling the same regex over and over again.)
• When given the choice between faster regex searches and simpler APIdesign, this crate will generally choose faster regex searches. For example,if one didn't care about performance, we could like get rid of both oftheRegex::is_match andRegex::find APIs and instead just rely onRegex::captures.

There are perhaps more ways that being "fast" influences things.

While this repository used to provide its own benchmark suite, it has sincebeen moved torebar. The benchmarks arequite extensive, and there are many more than what is shown in rebar's README(which is just limited to a "curated" set meant to compare performance betweenregex engines). To run all of this crate's benchmarks, first start by cloningand installingrebar:

$ git clone https://github.com/BurntSushi/rebar$ cd rebar$ cargo install --path ./

Then build the benchmark harness for just this crate:

$ rebar build -e '^rust/regex$'

Run all benchmarks for this crate as tests (each benchmark is executed once toensure it works):

$ rebar measure -e '^rust/regex$' -t

Record measurements for all benchmarks and save them to a CSV file:

$ rebar measure -e '^rust/regex$' | tee results.csv

Explore benchmark timings:

$ rebar cmp results.csv

See therebar documentation for more details on how it works and how tocompare results with other regex engines.

Theregex crate is, for the most part, a pretty thin wrapper around themeta::Regexfrom theregex-automata crate.Therefore, if you're looking to work on the internals of this crate, you'lllikely either want to look inregex-syntax (for parsing) orregex-automata(for construction of finite automata and the search routines).

Myblog on regex internalsgoes into more depth.

This crate's minimum supportedrustc version is1.65.0.

The policy is that the minimum Rust version required to use this crate can beincreased in minor version updates. For example, if regex 1.0 requires Rust1.20.0, then regex 1.0.z for all values ofz will also require Rust 1.20.0 ornewer. However, regex 1.y fory > 0 may require a newer minimum version ofRust.

This project is licensed under either of

• Apache License, Version 2.0, (LICENSE-APACHE orhttps://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT orhttps://opensource.org/licenses/MIT)

at your option.

The data inregex-syntax/src/unicode_tables/ is licensed under the UnicodeLicense Agreement(LICENSE-UNICODE).