Tool:Stashbot
Phabricator project:#Stashbot
 | |
|---|---|
| Website | https://stashbot.toolforge.org/ |
| Description | An IRC bot for storing data in Elasticsearch |
| Keywords | irc, python |
| Author(s) | BryanDavis |
| Maintainer(s) | BryanDavis,Yuvipanda,Greg Grossmeier (View all) |
| Source code | gerrit (diffusion) |
| License | GNU General Public License 3.0 or later |
| Issues | Open tasks ·Report a bug |
| Admin log | Tools.stashbot/SAL |
Stashbot is an IRC bot that stores data for use by other tools, responds to!log messages, and gives information about Phabricator objects referenced in messages.
Data collected by stashbot is stored in an Elasticsearch cluster in Toolforge and consumed by:
The bot archives all messages in IRC channels it joins to Elasticsearch indices that rotate monthly. It also examines the messages to see if they contain commands or other triggers.
!log processingIn the#wikimedia-operations connect,#wikimedia-sre connect,#wikimedia-fundraising connect,#wikimedia-traffic connect,#wikimedia-cloud connect,#wikimedia-releng connect,#wikimedia-analytics connect, and#countervandalism connect channels, messages beginning with!log are stored in a special index that is used by theSAL tool. The messages are also written to wiki pages on either wikitech or mediawiki.org. In the #wikimedia-operations channel, the messages are posted to the@sal@wikimedia.social fediverse account.
Phabricator tasks mentioned in!log messages will also be updated to include a reference to the SAL entry. This is commonly used inbackport windows and when performing other system maintenance related to an open issue.
!log messages in the#wikimedia-cloud connect channel are expected to begin with the name of a Cloud VPS project or Toolforge tool account (prefixed withtools.). For example:!log tools.stashbot Restarting to use updated configuration. The project/tool name is removed from the message and used to determine the storage location in both Elasticsearch and on-wiki. The bot will validate this project prefix using data from LDAP and tell you if the project is unknown.
The processing of!log messages in a channel can be limited to certain users by configuring an access control list of allowed or disallowed accounts. This feature can be used temporarily or permanently to block abuse by vandals.
!bash processingMessages in a channel that begin with!bash are added to a special Elasticsearch index that is used by theBash tool. Seethe tool's help page for more information.
Differential,Pholio, andManiphest objects mentioned by id in irc messages (e.g. T1234) will be looked up inPhabricator and then the title and URL of the object will be reported back to the channel. This feature can be disabled on a per-channel basis if the channel users decide that it is disruptive.
The bot runs fromToolforge under the "stashbot" account as akubernetes pod.
# Start the bot$./bin/stashbot.shstart# Stop the bot$./bin/stashbot.shstop# Restart the bot$./bin/stashbot.shrestart# Check job status$./bin/stashbot.shstatus# Tail log files$./bin/stashbot.shtail# Update bot to latest version$./bin/stashbot.shupdate$./bin/stashbot.shrestart
Maintainers/Members/Admins of the tool can do the below to enable the bot to join a new channel;
$sshlogin.toolforge.org$becomestashbot$vimetc/config-libera.yaml:# Add the channel in the irc.channels list.:# The list is in alphabetical order in 2 sections (priority channels, other channels)$bin/stashbot.shrestart
Stashbot started as a replacement for a set ofLogstash filters deployed in the thebeta cluster Logstash instance. The initial Logstash based system was functional, but had many limitations on the amount of processing that could be done. There was also no way for the irc connection made by Logstash to be used to send information back to the channels.
The code for the bot has borrowed heavily fromMorebots andJouncebot. Some innovations from stashbot have also been "upstreamed" back to jouncebot as part of theib3 python library.