- Notifications
You must be signed in to change notification settings - Fork6
chappjc/dcrspy
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
dcrspy is a program to continuously monitor and log changes in various data onthe Decred network. It works by connecting to both dcrd and dcrwallet andresponding to varioius event detected on the network vianotifiers registeredwith dcrd over a websocket. Communication with dcrd and dcrwallet uses theDecred JSON-RPC API.
- After Decred (i.e. dcrwallet and dcrd) v0.6.0, the notifications API waschanged, requiring dcrspy to update it's notification handlers. Practically,this means that for version 0.6.0 and earlier of Decred it is required to usethecompatibility release"Millbarge" or theold-ntfns branch, and forany Decred releaseafter 0.6.0 useat least dcrspy v0.7.0, preferablylatest, or master. The version ofdcrspy on master will use the new
versionRPC to check that the RPC server hasa compatible API version. - After Decred v0.7.0, the getbalance RPC response was changed. For Decredrelease 0.8.0 and builds of using dcrd commit
f5c0b7eand later, it is necessary to use at least v0.8.0 of dcrspy.
The types of information monitored are:
- Block chain (from dcrd)
- Stake and wallet (from your wallet, optional).
- Mempool, including ticket fees and transactions of interest (from dcrd)
A connection to dcrwallet is optional, but required for stake info and balances.SeeData Details below for more information.
Transactions sending towatched addresses may be reported (using thewatchaddress flag).
Multiple destinations for the data are planned:
- Plain text summary: balances, votes, current ticket price, mean fees,wallet status.DONE.
- JSON (stdout). JSON-formatted data is send to stdout.DONE.
- File system. JSON-formatted data is written to the file system.DONE.
- email: email notification upon receiving to a watched address.DONE.
- Database. Data is inserted into a MySQL or MongoDB database.
- RESTful API.
Details of the JSON output may be found inData Details. Theplain text summary looks something like the following (wallet data simulated):
Block 105001: Stake difficulty: 119.385 -> 119.385 (current -> next block) Estimated price in next window: 79.021 / [79.02, 315.71] ([min, max]) Window progress: 26 / 144 in price window number 729 Ticket fees: 0.0000, 0.0000, 0.0000 (mean, median, std), n=0 Node connections: 8Wallet and Stake Info at Height 105001:- Balances (by account) imported: 1234.5678 (any), 0.0000 (spendable), 1234.5678 (locked), 0.0000 (immat.) default: 4321.4321 (any), 3210.4000 (spendable), 0.0000 (locked), 1111.0321 (immat.)- Balances (by type) spendable: 3210.4000 (default), 3210.4000 (all) locked: 0.0000 (default), 1234.5678 (all), 1234.5678 (imported) immat. coinbase: 0.0000 (default), 0.0000 (all) immat. votes: 1111.0321 (default), 1111.0321 (all) any: 4321.4321 (default), 5555.9999 (all)- Stake Info: Mined tickets: 12 (immature), 57 (live) mempool tickets: 5 (own), 544 (everyone) ===> Mining enabled: false; Unlocked: false <=== Ticket price: 119.385 | Window progress: 26 / 144 Your limit: 1337.00000; Fee: 0.0470 DCR / KB (Approximately 26.7 tickets may be purchased with set fee.) Totals: 404 votes, 697.47 subsidy 3 missed, 3 revoked (2 expired)Note: Ticket pool value takes up to 10 seconds to compute, so by default it isnot requested from dcrd, and thus not shown in the summary. It is stillpresent in JSON, but the values are {0, -1, -1}. To get actualy ticket poolvalue, use-p, --poolvalue.
dcrspy may watch for transactions receiving into or sending from "watched"addresses. Watched addresses are specified with thewatchaddress flag, withmultiple addresses specified using repeatedwatchaddress flags (e.g. one perline in the config file). For example:
; Addresses to watch for incoming transactions; Decred developer (C0) address;watchaddress=Dcur2mcGjmENx4DhNqDctW5wJCVyT3Qeqkx; Some larger mining pool addresses:;watchaddress=DsYAN3vT15rjzgoGgEEscoUpPCRtwQKL7dQ;watchaddress=DshZYJySTD4epCyoKRjPMyVmSvBpFuNYuZ4To receive an email notification for each transaction receiving to a watchedaddress, concatenate at the end of the address: ",1" for mined transactions,",2" for transactions inserted into mempool, or ",3" for both. For example:
; Receive email notifications for this one;watchaddress=DsZWrNNyKDUFPNMcjNYD7A8k9a4HCM5xgsW,1; But not this one;watchaddress=Dsg2bQy2yt2onEcaQhT1X9UbTKNtqmHyMus,0; and not by default;watchaddress=DskFbReCFNUjVHDf2WQP7AUKdB27EfSPYYE; Mined and mempool for this:;watchaddress=DsWM9WbE3YyRKxGGNdBYXCwJoxiBtXFrZBL,3An SMTP server name, port, authentication information, and a recipient emailaddress must also be specified to use email notifications.
emailaddr=chappjc@receiving.comemailsubj="dcrspy tx notification"smtpuser=smtpuser@mailprovider.netsmtppass=suPErSCRTpasswurdsmtpserver=smtp.mailprovider.org:587If you have trouble getting it to work, try any alternate ports available onyour SMTP server (e.g. 587 instead of 465). You must specify the port.
When dcrspy receives a new block notification from dcrd, data collection andrecording is triggered. In addition, any system command may be executed inresponse to the new block. The flags to specify the command and the argumentsto be used are:
-c, --cmdname= Command name to run. Must be on %PATH%.-a, --cmdargs= Comma-separated list of arguments for command to run. The specifier %n is substituted for block number at execution, and %h is substituted for block hash.The command name must be an executable (binary or script) on your shell's PATH,which is$PATH in *NIX, and%PATH% in Windows.
TODO: Delay execution, or run after data saving is complete.
Any arguments must be specified withcmdargs as a comma-separated list of strings. For example:
-c ping -a "127.0.0.1,-n,8"will execute the following on Linux:
/usr/bin/ping 127.0.0.1 -n 8Specifying multiple arguments without commas, using spaces directly, isincorrect. For example,-a "127.0.0.1 -n 8" will not work.
Note that if your command line arguments need to start with a dash (-) it isnecessary to use the config file. For example,
cmdname=lscmdargs="-al"The new block hash and height at the time of command execution may be includedon the command line using %h and %n, which are substituted for block hash andnumber. For example,
cmdname=echocmdargs="New best block hash: %h; height: %n"results in the following log entries (date removed for brevity):
[INF] DCRD: Block height 36435 connected[INF] EXEC: New best block hash: 000000000000070f7a0593aee0728d6b3334c1e454da06efc0138008dc1b1cbd; height: 36435[INF] EXEC: Command execution complete (success).Note that the above command used a semicolon since a comma would have indicateda second argument and been replaced with a space by echo.
User-specified system command execution uses the logging subsystem tagged withEXEC in the logs. Both stdout and stderr for the executed command are sent tothe dcrspy log. The end of command execution is also logged, as shown in theexample above.
dcrspy is functional, but also awork-in-progress. However, I will try to keepmaster as stable as possible, and develop new features in separate branches.
- Go 1.7 or newer.
- Running
dcrdsynchronized to the current best block on the network. - (Optional, for stake info)
dcrwalletconnected todcrd.
Verify that the versions according to thecompatibility notices above.
The following instructions assume a Unix-like shell (e.g. bash).
Verify Go installation:
go env GOROOT GOPATHEnsure $GOPATH/bin is on your $PATH
Install glide
go get -u -v github.com/Masterminds/glideClone dcrspy repo
git clone https://github.com/chappjc/dcrspy.git $GOPATH/src/github.com/chappjc/dcrspyGlide install, and build executable
cd $GOPATH/src/github.com/chappjc/dcrspy glide install go install $(glide nv)Find dcrspy executable in
$GOPATH/bin, and copy elsewhere (recommended).
If you receive build errors, it may be due to "vendor" directories left byglide builds of dependencies such as dcrwallet. You may safely delete vendorfolders.
First, update the repository (assuming you havemaster checked out):
cd $GOPATH/src/github.com/chappjc/dcrspygit pullThen follow the install instructions starting at "Glide install...".
By default, dcrspy will monitor both block data and your wallet, and write aplain text summary of the data to stdout for each new block that is detected.
There are several program options, which may be utilized via:
- Command line arguments
- Config file (e.g. dcrspy.conf)
Quick tips:
- Get a quick summary and exit, with
-e, --nomonitor. - Enable mempool info with
-m, --mempool. - Dump all mempool ticket fees to file with
--dumpallmptix. - Stay connected and monitor for new blocks, writting:
- Plain text summary to stdout, with
-s, --summary(default.) - JSON to stdout, with
-o, --save-jsonstdout. - JSON to file system, with
-j, --save-jsonfile.
- Plain text summary to stdout, with
- To monitor only block data (no wallet connection), use
--nostakeinfo.
The full list of command line switches is below, with current directoryreplaced by...:
$ ./dcrspy -hUsage: dcrspy [OPTIONS]Application Options: -C, --configfile= Path to configuration file (./dcrspy.conf) -V, --version Display version information and exit --testnet Use the test network (default mainnet) --simnet Use the simulation test network (default mainnet) -d, --debuglevel= Logging level {trace, debug, info, warn, error, critical} (info) -q, --quiet Easy way to set debuglevel to error --logdir= Directory to log output (./logs) --cpuprofile= File for CPU profiling. -c, --cmdname= Command name to run. Must be on %PATH%. -a, --cmdargs= Comma-separated list of arguments for command to run. The specifier %n is substituted for block height at execution, and %h is substituted for block hash. -e, --nomonitor Do not launch monitors. Display current data and (e)xit. -m, --mempool Monitor mempool for new transactions, and report ticketfee info when new tickets are added. --mp-min-interval= The minimum time in seconds between mempool reports, regarless of number of new tickets seen. (4) --mp-max-interval= The maximum time in seconds between mempool reports (within a couple seconds), regarless of number of new tickets seen. (120) --mp-ticket-trigger= The number minimum number of new tickets that must be seen to trigger a new mempool report. (4) -r, --feewinradius= Half-width of a window around the ticket with the lowest mineable fee. --dumpallmptix Dump to file the fees of all the tickets in mempool. --noblockdata Do not collect block data (default false) --nostakeinfo Do not collect stake info data (default false) -p, --poolvalue Collect ticket pool value information (8-9 sec). -w, --watchaddress= Watched address (receiving). One per line. --smtpuser= SMTP user name --smtppass= SMTP password --smtpserver= SMTP host name --emailaddr= Destination email address for alerts --emailsubj= Email subject. (default "dcrspy transaction notification") (dcrspy transaction notification) -s, --summary Write plain text summary of key data to stdout -o, --save-jsonstdout Save JSON-formatted data to stdout -j, --save-jsonfile Save JSON-formatted data to file -f, --outfolder= Folder for file outputs (./spydata) --dcrduser= Daemon RPC user name --dcrdpass= Daemon RPC password --dcrdserv= Hostname/IP and port of dcrd RPC server to connect to (default localhost:9109, testnet: localhost:19109, simnet: localhost:19556) --dcrdcert= File containing the dcrd certificate file (~/.dcrd/rpc.cert) --nodaemontls Disable TLS for the daemon RPC client -- NOTE: This is only allowed if the RPC client is connecting to localhost --dcrwuser= Wallet RPC user name --dcrwpass= Wallet RPC password --dcrwserv= Hostname/IP and port of dcrwallet RPC server to connect to (default localhost:9110, testnet: localhost:19110, simnet: localhost:19557) --dcrwcert= File containing the dcrwallet certificate file (~/.dcrwallet/rpc.cert) --nowallettls Disable TLS for the wallet RPC client -- NOTE: This is only allowed if the RPC client is connecting to localhostHelp Options: -h, --help Show this help messageAll command line switches may be placed into the config file, which isdcrspy.conf by default.
[Application Options]debuglevel=debug;debuglevel=DSPY=info,EXEC=debug,MEMP=debug,DCRD=info,DCRW=info,RPCC=info;cmdname=echo;cmdargs="New best block hash: %h; height: %n";cmdname=ping;cmdargs="127.0.0.1,-n,8"; Monitor mempool for new tickets, displaying feesmempool=true;mp-min-interval=4;mp-max-interval=120;mp-ticket-trigger=4; Show the ticket fee at the limit of mineability (20th or highest), plus show; windows of fees above and below this limit within the specified radius.;feewinradius=20; Dump the prices of all tickets in mempool to disk each time it is reported; to stdout according to the settings above. JSON formatted output.;dumpallmptix=1; Addresses to watch for incoming transactions; Decred developer (C0) address;watchaddress=Dcur2mcGjmENx4DhNqDctW5wJCVyT3Qeqkx; Some larger mining pool addresses:;watchaddress=DsYAN3vT15rjzgoGgEEscoUpPCRtwQKL7dQ;watchaddress=DshZYJySTD4epCyoKRjPMyVmSvBpFuNYuZ4; receive email notifications for this one;watchaddress=DsZWrNNyKDUFPNMcjNYD7A8k9a4HCM5xgsW,1; but not this one;watchaddress=Dsg2bQy2yt2onEcaQhT1X9UbTKNtqmHyMus,0; and not by default;watchaddress=DskFbReCFNUjVHDf2WQP7AUKdB27EfSPYYE; SMTP server setup;emailaddr=you@receiving.com;emailsubj="dcrspy tx notification";smtpuser=smtpuser@mailprovider.net;smtppass=suPErSCRTpasswurd;smtpserver=smtp.mailprovider.org:587; Ticket pool value takes a long time, 8-9 sec, so the default is false.;poolvalue=false; Default outfolder is a folder called "dcrspy" in the working directory.; Change this with the outfolder option:; Windows; outfolder=%appdata%/dcrspy/spydata; Linux; outfolder=$HOME/dcrspy/spydatadcrduser=duserdcrdpass=asdfExampledcrdserv=localhost:9109dcrdcert=/home/me/.dcrd/rpc.cert;nodaemontls=1dcrwuser=wuserdcrwpass=qwertyExampledcrwserv=localhost:9110dcrwcert=/home/me/.dcrwallet/rpc.cert;nowallettls=1;noclienttls=false
Block chain data obtained from dcrd includes several types of data. Most of theinformation is written to stdout in a plaintext summary format. Optionally, thedata may be written to disk in JSON format. The JSON file written by dcrspy forblock data is namedblock-data-[BLOCKNUM].json. It contains a single JSONobject, with each data type as a tagged JSON child object.
- Block header (hash, voters, height, difficulty, nonce, time, etc.)
"block_header": {"hash":"00000000000014c19867d5cd0f60d9409cd9e4ea68f656dac50befa756866cf8","confirmations":1,"version":1,"previousblockhash":"00000000000010c295f2e808af78d8240c3365d9d52b28e2061f9a55ce9dcd29","merkleroot":"731c342c75237fe72e2d27a6820f6384498add97f96dcef9c6f2fd558a80f4c9","stakeroot":"c3d75f22a1e9bb4b50155f95d9b7a977f7ed1ecaf05024e157a77d7b5697fe04","votebits":1,"finalstate":"6acc9d2694f3","voters":5,"freshstake":3,"revocations":0,"poolsize":42084,"bits":"1a166536","sbits":22.66271576,"height":35552,"size":3764,"time":1465510687,"nonce":2821830658,"difficulty":749126.76453394}
- Ticket pool info. This is a custom data structure.
"ticket_pool_info": {"poolsize":42084,"poolvalue":745705.38747115,"poolvalavg":17.71945127}
- Ticket fee info (block). This is the usual output of
ticketfeeinfowithno extra arguments:
"ticketfeeinfo_block": {"height":35552,"number":3,"min":0.01010101,"max":0.01013513,"mean":0.01011238,"median":0.01010101,"stddev":1.97e-05}
- Current and estimated stake difficulty. These are the usual outputs of
estimatestakediffandgetstakedifficulty:
"estimatestakediff": {"min":23.80700879,"max":28.94194858,"expected":25.46730598},"currentstakediff": {"current":22.66271576,"next":22.66271576}
Wallet data is stored in a similar manner in filestake-info-[BLOCKNUM].json.There are three data types, tagged"getstakeinfo","walletinfo", and"balances". TODO: Update this README with a testnet example output.
Please report any issues using theGitHub issue tracker.
You may also visit thedcrspy thread at the official Decred forum.
dcrspy is licensed under thecopyfree ISC License.
dcrspy borrows its logging and config file facilities, plus some boilerplatecode in main.go, from the dcrticketbuyer project by the Decred developers.The rest is by chappjc.