Examples¶
Note
Please feel free to contribute your own examples on github
Simple Text¶
Histogram Values in Text File¶
$catinput.txt121310$rarehistoinput.txt13012131Matched:6/6(Groups:4)Extract Numbers from Text¶
$rarefilter--match"(\d+)"input.txtExtract matched value as JSON¶
$raref--match"(?P<val>\d+)"-e"{.}"simple.log{"val":1}{"val":2}{"val":1}Histogram of Numbers in Text¶
$rarehisto--match"(\d+)"-e"{1}"-xinput.txt13012131Matched:6/6(Groups:4)# Or with Bars/percentages./rarehisto--match"(\d+)"-e"{1}"-xsimple.log17[58.3%]||||||||||||||||||||||||||||||||||||||||||||||||||32[16.7%]||||||||||||||22[16.7%]||||||||||||||01[8.3%]|||||||Matched:12/12(Groups:4)# Logarithmic Scale./rarehisto--match"(\d+)"-e"{1}"--scalelog10-xsimple.log17[58.3%]||||||||||||||||||||||||||||||||||||||||||32[16.7%]|||||||||||||||22[16.7%]|||||||||||||||01[8.3%]Matched:12/12(Groups:4)Nginx¶
Example line:
123.123.123.123 - - [19/Aug/2019:02:26:25 +0000] "GET / HTTP/1.1" 200 546 "-" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/98 Safari/537.4 (StatusCake)"Highlight / Extract HTTP Code and Size¶
# Will colorize HTTP code and size in full log$rarefilter-m"(\d{3}) (\d+)"access.log# Will only display http code and size$rarefilter-m"(\d{3}) (\d+)"-e"{1} {2}"access.logHTTP Status Histogram¶
Parse error codes and graph in a histogram
$rareh-m"\" (\d+)"-e"{1}"-z-xtestdata/*4045,557,374[66.4%]||||||||||||||||||||||||||||||||||||||||||||||||||2002,564,984[30.6%]|||||||||||||||||||||||400243,282[2.9%]||4055,708[0.1%]4081,397[0.0%]Matched:8,373,328/8,383,717(Groups:8)Extracting Page Sizes¶
Page sizes, ignoring 0-sized pages
$rareh-m"\" (\d+) (\d+)"-e"{bytesize {bucket {2} 1024}}"-i"{lt {2} 1024}"-z-xtestdata/*234KB3,602[14.6%]||||||||||||||||||||||||||||||||||||||||||||||||||149KB2,107[8.5%]|||||||||||||||||||||||||||||193KB1,519[6.2%]|||||||||||||||||||||192KB1,470[6.0%]||||||||||||||||||||191KB1,421[5.8%]|||||||||||||||||||Matched:24,693/8,383,717(Groups:96)(Ignored:8,348,635)Extract number of bytes sent by bucket, and format¶
This shows an example of how to bucket the values into size of1000. In this case, it doesn't makesense to see the histogram by number of bytes, but we might want to know the ratio of various orders-of-magnitudes.
$rarehisto-m'"(\w{3,4}) ([A-Za-z0-9/.]+).*" (\d{3}) (\d+)'-e"{bucket {4} 10000}"-n10-baccess.log0144239||||||||||||||||||||||||||||||||||||||||||||||||||190000259910000129018000082120000496300004454000044020000042714000032370000222Matched:161622/161622Groups:1203Table of URLs to HTTP Status¶
Know how your URLs are responding by their http statuses
$raret-m"\"(\w+) (.+).*\" (\d+) (\d+)"-e"{$ {3} {substr {2} 0 20}}"-ztestdata/*404200400/HTTP/1.10127,6245,681/HTTP/1.005,2220/test.phpHTTP/1.13,24100/1.phpHTTP/1.12,50800/qq.phpHTTP/1.11,90800/index.phpHTTP/1.11,77600/shell.phpHTTP/1.11,75000/cmd.phpHTTP/1.11,58800/x.phpHTTP/1.11,57300/log.phpHTTP/1.11,26100/confg.phpHTTP/1.11,25300/api.phpHTTP/1.11,24100/ss.phpHTTP/1.11,23300/mirror/distros/vlit01,1220/robots.txtHTTP/1.11,05600/vendor/phpunit/phpu1,05500/aaa.phpHTTP/1.195400/hell.phpHTTP/1.194800/z.phpHTTP/1.194800Matched:465,348/470,163(R:2396;C:8)Heatmap of Requests per Week¶
$ rare heatmap -m '\[(.+?)\].*" (\d+)' -e "{timeattr {time {1}} yearweek}" -e "{2}" access.log - 0 4 22,602 9 45,204 2019-34..2019-43..2019-52..2020-17..2020-25..2020-33...2020-9200 11--111111-11111-111111111-1111111-111111-11111-1111-11--1-1-206 -------------------------------------------------------------301 -------------------------------------------------------------304 -------------------------------------------------------------400 -------------------------------------------------------------404 33415264914143253212111-1411-13-131-1312-122111--13-1-1-13211405 -------------------------------------------------------------408 -------------------------------------------------------------Matched: 1,035,666 / 1,035,666 (R: 8; C: 61)Bargraph status codes per year¶
NOTE: For stacking (-s), the results will be color-coded (not shown here)
$rarebars-z-m"\[(.+?)\].*\" (\d+)"-e"{buckettime {1} year}"-e"{2}"-stestdata/*0200120623013304440054046405740820190000000005555555555555555555555555555553,742,444202000000000000000000044555555555555555555555555555554,631,884Matched:8,374,328/8,384,811