Script to pull DMARC records, process and pass it to splunk.
imap-client.py - Pull attachments from mail imap server and storeit in the given directory. This is a generic program that can be used tofetch emails and/or attachments using IMAP protocol.
dmarc-parser.py - Convert the xml files to comma-seperated key=valuepair (line oriented output for splunk). This script can handle large xml files
dmarc-convertor.sh - An uber script to manage the workflow end-to-end:
- Download attachments from mail server
- Unzip the attachments
- Parse unzipped xml files and convert it line oriented format for splunk
imap-client.py [-h] [-v] [--attachmentsonly] [--disablereadonly] [--quiet] -s HOST [-p PORT] -c CACERTS -u USER -f FOLDER -o OUTDIR [-S SEARCH] [-P PWDFILE]optional arguments: -h, --help show this help message and exit -v, --verbose increase output verbosity --attachmentsonly download attachments only --disablereadonly enable state changes on server; Default readonly --quiet supress all comments (stdout) -s HOST, --host HOST imap server; eg. imap.mail.yahoo.com -p PORT, --port PORT imap server port; Default is 993 -c CACERTS, --cacerts CACERTS CA certificates, which are used to validate certificates passed from imap server -u USER, --user USER user's email id -f FOLDER, --folder FOLDER mail folder from which the mail to retrieve -o OUTDIR, --outdir OUTDIR directory to output -S SEARCH, --search SEARCH search criteria, defined in IMAP RFC 3501; eg. "SINCE \"8-Sep-2014\"" -P PWDFILE, --pwdfile PWDFILE A file that stores IMAP user password. If not set, the user is prompted to provide a passwdExample: % imap-client.py -s imap.example.com -c ./cacert.pem -u dmarc@example.com -f inbox -o ./mymail -S "SINCE \"8-Sep-2014\"" -P./paswdfile
dmarc-parser.py [-h] dmarcfilepositional arguments: dmarcfile dmarc file in XML formatoptional arguments: -h, --help show this help message and exitExample: % dmarc-parser.py dmarc-xml-file 1> outfile.csv
dmarc-convertor.sh -u user_emailid -s imapserver -c cacertfile [-p port] [-P pwdfile] [-h] Options: -u User email id -P File that contains user password. Default: The user will be prompted to provide password if you leave this option. WARNING: The file should be with permission 0400 or 0440 (ie should NOT be world readable) -s IMAP server name -p IMAP port number. Default: 993 -c CA certificate file (eg. cacert.pem), used to validate certificates passed from IMAP server -h HelpExample: % dmarc-convertor.sh -u dmarc@example.com -P ./pwd -s imap.example.com -p 993 -c ./cacert.pem
The
dmarcReportProcessor.servicedmarcReportProcessor.timer
shows a possible systemd call to execute the report collection. The env RUAFOLDER defines the IMAP folder where the reports are.
NOTE The above script expectsimap-client.py anddmarc-parser.py available in $ROOT/bin. You may change the path by modifiyingdmarc-convertor.sh.
Tested on python 2.7