rare¶

GitHub all releases GitHub

A fast text scanner/regex extractor and realtime summarizer. Quickly search, reformat and visualize text filessuch as logs, csv, json, etc.

Supports various CLI-based graphing and metric formats (filter (grep-like), histogram, table, bargraph, heatmap, reduce).

rare is a play on "more" and "less", but can also stand for "realtime aggregated regular expressions".

rare gif

Features¶

Multiplesummary formats including: filter (like grep), histogram, bar graphs, tables, heatmaps, sparklines, reduce, and numerical analysis
Parse usingregex (-m) ordissect tokenizer (-d)
Manipulate results with handlebars-likeexpressions (-e)
Fileglob expansions (eg/var/log/* or/var/log/*/*.log) and recursion-R withpath filters (include/exclude patterns)
Optionalgzip decompression (with-z)
Following-f or re-open following-F (use--poll to poll, and--tail to tail)
Ignoring lines that match an expression (with-i)
Aggregating and realtime summary (Don't have to wait for all data to be scanned)
Multi-threaded reading, parsing, and aggregation (It's fast)
Color-coded outputs (optionally)
Pipe support (stdin for reading, stdout will disable realtime, and--csv formatting) eg.tail -f | rare ... > out

Take a look atexamples to see more of whatrare does.

Installation¶

A Note on PCRE (Perl Compatible Regular Expressions)

Besides your standard OS versions, there is an additionalpcre build which is ~4x faster than go'sre2 implementation in moderately complex cases. In order to use this, you must make sure thatlibpcre2 is installed (egapt install libpcre2-8-0 ordnf install pcre2). Right now, it is only bundled with the linux distribution.

PCRE2 also comes with pitfalls, two of the most important are:1. Thatrare is now dynamically linked, meaning that you need to have libc and libpcre installed2. That pcre is an exponential-time algorithm (re2 is linear). While it can be significantly faster than go'sre2, it can also be catastrophically slower in some situations. There is a good posthere that talks about regexp timings.

I will leave it up to the user as to which they find suitable to use for their situation. Generally, if you know whatrare is getting as an input, the pcre version is perfectly safe and can be much faster.

Bash Script¶

Warning: Verify Script

As a best practice, you should always verify anyscript before piping to bash.

This script downloads the latest version from github, and installs it to~/.local/bin (/usr/bin ifroot).

curl-sfLhttps://rare.zdyn.net/install.sh|bash

Manual (Prebuilt Binary)¶

Download appropriate binary or package fromReleases

Homebrew¶

brewtapzix99/rarebrewinstallrare

Community Contributed¶

Note

The below install methods have been contributed by the community, and aren't maintained directly.

MacPorts¶

sudoportselfupdatesudoportinstallrare

From code¶

Clone the repo, and:

Requires GO 1.23 or higher

gomoddownload# Build binarygobuild.# OR, with experimental featuresgobuild-tagsexperimental.

Available tags:

experimental Enable experimental features (eg. fuzzy search)
pcre2 Enables PCRE 2 (v10) where able. Currently linux only
rare_no_pprof Disables profiling capabilities, which reduces binary size
urfave_cli_no_docs Disables man and markdown documentation generation, which reduces binary size

Quickstart¶

The easiest way to start usingrare is by creating ahistogram.

Each execution is usually composed of two parts, theregex extracted match (-m) and theexpression (-e).

$rarehisto\-m'"(\w{3,4}) ([A-Za-z0-9/.]+).*" (\d{3})'\# The regex that extracts match-groups-e'{3} {1}'\# The expression will be the key, referencing the match-groupsaccess.log# One or more files (or -R for recursion)200GET160663404GET857304GET53200HEAD18403GET14

Next¶

To learn more, check out theaggregators, read someexamples or dig into theoverview.

Movatterモバイル変換