 | This page is currently inactive and is retained forhistorical reference. Either the page is no longer relevant or consensus on its purpose has become unclear. To revive discussion, seek broader input via a forum such as thevillage pump. |
 | STiki has not been functional since March 2020. The creator has been unable to access the servers that STiki depends on in order to run. Users are advised to trythese alternatives. |
| STiki | |
|---|---|
 | |
 STiki showing vandalism | |
| Developers | Andrew G. West (west.andrew.g); Insup Lee[1] (advisor) |
| Initial release | June 2010; 15 years ago (2010-06) |
| Stable release | 2.1 / December 8, 2018; 6 years ago (2018-12-08) |
| Written in | Java |
| Platform | Java SE |
| Available in | English |
| Type | Vandalism detection on Wikipedia |
| License | GNU General Public License |
| Website | andrew-g-west |
STiki is a tool available totrusted users that is used to detect andrevertvandalism,spam, and other types ofunconstructive edits made at Wikipedia. STiki chooses edits to show to end users; if a displayed edit is judged to be vandalism, spam, etc., STiki streamlines the reversion and warning process. STiki facilitates collaboration in reverting vandalism; a centrally stored lists of edits to be inspected are served to STiki users to reduce redundant effort. STiki isnot aWikipedia bot: it is anintelligent routing tool that directs human users to potential vandalism for definitive classification.
To date, STiki has been used to revert1,265,447 edits that its users have identified as vandalism, spam, or otherwise unconstructive (see theleaderboard andeditor milestones).
STiki may only be used by editors with aWikipedia account. Additionally, the account must meet some qualifications to reduce the probability of users misidentifyingvandalism. The account must have any one of: (1) therollback permission/right, (2) at least 1000 article edits (in thearticle namespace, not to talk/user pages), or (3) special permission via thetalk page. We emphasize that users musttake responsibility for their actions with STiki.
After login, users primarily interact with theGUI tool by classifying edits into one of four categories:
| 1. | Vandalism | If an edit is blatantly unconstructive and intentional in its malice, then it constitutesvandalism. Pressing theVandalism button will revert the edit, and the "warn offending editor" box should be checked so the guilty party is notified of their transgression. Multiple warnings will result in reporting atAIV and subsequent blocking. However, youmay wish to avoidtemplating the regulars, as some construe this as poor wiki-etiquette. In exceptional cases, the small4im button can be used to give a more severe warning. |
| 2. | Good-faith revert | Sometimes edits are clearly unconstructive, but lack the intent and malice that characterizes vandalism. In these cases, one shouldassume good faith by undoing the changes using aGood-faith revert. In this case, the change is undone, but the offending editor is not issued a warning and the revert is not marked as minor. A dialogue allows the STiki user to notify the reverted user with a polite and informative (non-"warning")AGF message template. |
| 3. | Pass | If a STiki user is uncomfortable with labeling an edit as clear vandalism, they can skip orPass the edit. The revision will then remain live on Wikipedia and shown to another STiki user. Use pass only when you believe there is some chance the edit is vandalism, but you lack the subject expertise to firmly make that decision. |
| 4. | Innocent | If an edit is primarily constructive and not vandalism, it should be labeled asInnocent. This does not mean the edit must be perfect in nature. Indeed, STiki is anti-vandal focused and cannot remedy many issues, which should be handled outside the tool (using the provided hyperlinks). |
Uncertainty over constructiveness: If a user is uncertain about whether an edit is constructive, the quickest solution is often to perform a web search (e.g.,, with Google); this may reveal whether some "fact" is true. Of course, STiki users should consider the reliability of the source found. If no reliable source can be found, the correct response may be to add a{{Citation needed}} or{{Verify credibility}} tag, using the normal wiki interface. Where content has been removed, common sense is usually the best guide. Does the removed text have citations? (Note that checking the citations themselves may be necessary in content regardingliving people.) What is the edit summary? Does that explanation make sense? Is it discussed on the talk page? Regardless of the issue, anything that requires domain-specific expertise to resolve is probably best classified as "innocent" or "pass".
Uncertainty over malice: It can be tricky to differentiate between vandalism and good-faith edits that are nonetheless unconstructive. Test edits should be classified as "vandalism", as initial warnings and edit comments accommodate this case. If the unconstructive edit or the edit summary indicate Wikipedia inexperience, it may be best to label the edit "good faith" and leave a message on the new user's talk page, offering to help. Beyond that, common sense is usually the best guide. Consider the article in question. Is it something that young editors might be interested in? Is there any truth in what is being said (absent formatting, language, and organizational issues)?
Deeper investigation: Sometimes a revert ("vandalism" or "good faith") will not repair all the issues presented in a diffor the diff doesn't contain enough evidence to make a definitive classification. In these cases, use the hyperlinks (blue underlined text) to open relevant pages in the default web browser. This is helpful, for example, to: (1) view the article talk page to see if some issue was discussed, (2) make changes using the normal interface, and (3) use other tools likePopups,Twinkle, andwikEdDiff.
When you return to the STiki tool, you will still need to classify the edit. If you used the browser interface to edit the article, pressing "vandalism" or "good-faith revert" willnot revert your changes or have any direct effect on Wikipedia. Classify the displayed edit as best you can. Making such classifications will help STiki to identify similar edits in the future.
Interface tips: STiki hashotkeys to ease user interaction with the tool. After a single edit has been classified with the mouse (giving the button panel "focus"), the keysV,G,P andI will mark edits as "vandalism", "good faith", "pass", and "innocent" respectively. While in the same mode, thePage Up, Page Down,Up Arrow (↑), andDown Arrow (↓) keys will also scroll the diff browser. Also note thathyperlinks which appear in diffs can be opened in your web-browser, assuming that the "Activate Ext-Links" option (under the "Options" tab) is turned on. STiki stores your settings in a file named.STiki.props.xml, so it is possible to quickly edit your settings there.
The followingfeatures make STiki distinctive:
| 1. | Sophisticated algorithms | STiki uses multiple algorithms to identify potential vandalism; all are rooted inmachine learning. Currently active queues have demonstrated a 25%–30% hit rate (percentage of reverts for all edits displayed in the GUI). Random search will result in hit-rates < 3%. |
| 2. | The server coordinates tasks | STiki users are shown edits from a centrally maintained queue. When a user is shown an edit, they have a "reservation" so that no other STiki users are viewing the edit simultaneously. Moreover, if a user marks an edit as "innocent", no one will be forced to review this edit in the future. In both cases, redundant work (edit conflicts, multiple reviews of good edits) is avoided. |
| 3. | The server remembers | The STiki server is always watching changes and computing vandalism probabilities, even if no one is currently using the GUI tool. When edits are popped out to end-users, this is done purely based on vandalism probabilities, not how recent the changes are. STiki has identified instances of vandalism that were several months old. |
| 4. | Simple interface | STiki's interface is a minimal one. This is due to a belief STiki should focus exclusively on vandalism/spam removal, rather than becoming a general-purpose framework for a diversity of unconstructive edits. When more information is needed the interface provides links to relevant pages of the wiki, which open in a normal web browser. |
| 5. | Cross-platform | Developed in Java, STiki iscross-platform. |
STiki orders the edits to be displayed to end-users into priority queues. The priority an edit takes is based upon its evaluation by an anti-damage scoring system. Different systems produce different scores/queues, and users can explicitly select a queue to access using the "Rev. Queue" menu. All approaches are rooted inmachine learning, of which there are two active, and two inactive approaches:
| 1. | Cluebot-NG | Using anartificial neural network (ANN) to score edits is theClueBot NG approach. The worst-scoring edits are undone automatically. However, there are many edits that CBNG is quite confident are vandalism, but cannot revert due to a low false-positive tolerance, theone-revert-rule, or other constraints. These scores are consumed from anIRC feed. | |
| 2. | STiki (metadata) | The "original" queue used by STiki, using metadatafeatures and analternating decision tree to arrive at vandalism predictions. More detail about this technique is available in the "metadata scoring and origins" section below. | |
| 3. | Link spam | Parses new external links from revisions and measures theirexternal link spam potential. Probabilities are generated over 50 features capturing Wikipedia metadata, landing site analysis, and 3rd party data. See ourWikiSym'11 paper.(Inactive: Rarely used and incurs monetary cost. May return w/o paid 3rd party features.) | |
| 4. | Wikitrust | Built upon editor reputations calculated from content-persistence is theWikiTrust system of Adleret al. More details are available at theirwebsite. WikiTrust scores are consumed via theirAPI. (Inactive: Rarely used, poor accuracy, and 3rd party server was unreliable in computing real-time probabilities) |
When STiki is experiencing considerable use, the frequency of vandalism found in one queue may reduce significantly, a phenomenon called "queue exhaustion". In such cases, it may be wise to try an alternative queue. Users should also recognize there is a finite amount of vandalism on Wikipedia. The more people who use STiki, the less percentage any one user will see. This does not mean STiki is doing "bad"; it means the encyclopedia is doing "good".
Here we highlight a particular scoring system, based on machine-learning over metadata properties. This system was developed by the same authors as the STiki frontend GUI, was theonly system shipped with the first versions, and shares a code-base/distribution with the STiki GUI. This system also gave the entire software package its name (derived fromSpatioTemporal processing on Wikipedia), though this acronymic meaning is now downplayed.
The "metadata system" examines only four fields of an edit when scoring: (1) timestamp, (2) editor, (3) article, and (4) revision comment. These fields are used to calculate features pertaining to the editor's registration status, edit time-of-day, edit day-of-week,geographical origin, page history, category memberships, revision comment length,etc. These signals are given to anADTree classifier to arrive at vandalism probabilities. The ML models are trained over classifications provided on the STiki frontend. A more rigorous discussion of the technique can be found in aEUROSEC 2010 publication.
An API has been developed to give other researchers/developers access to the raw metadata features and the resulting vandalism probabilities. AREADME describes API details.
The paper was an academic attempt to show thatlanguage properties were not necessary to detect Wikipedia vandalism. It succeeded in this regard, but since then the system has been relaxed for general-purpose use. For example, the engine now includes some simple language features. Moreover, there was the decision to integrate other scoring systems in the GUI frontend.
STiki uses aserver/client architecture:
1.Back-end processing that watches all recent changes to Wikipedia and calculates/fetches the probability that each is vandalism. This enginecalculates scores for theMetadata Scoring System, and uses APIs/feeds toretrieve the scores calculated by third-party systems. Edits populate a series of inter-linked priority queues, where the vandalism scores are the priority. Queue maintenance ensures that only the most-recent edit to an article is eligible to be viewed. Back-end work is done on STiki's servers (hosted at theUniversity of Pennsylvania), relying heavily on aMySQL database.
2.Front-end GUI that displaysdiffs, that the back end has determined likely contain vandalism, to human users and asks for definitive classification. STiki's user interface is aJava desktop application. It streamlines the process ofreverting poor edits and issuing warnings/AIV-notices to guilty editors. The interface is designed to enable quick review. Moreover, the classification process establishes a feedback loop to improve detection algorithms.
STiki's authors are committed to working towards collaborative solutions to vandalism. To this end, anAPI is available to STiki's internally calculated scores. A live feed of scores is also published to channel "#arm-stiki-scores" on IRC server "armstrong.cis.upenn.edu". Moreover, all STiki code is open-sourced.
In the course of our research, we have collected large amounts of data, both passively regarding Wikipedia, and through users' active use of the STiki tool. We are interested in sharing this data with other researchers. Finally, STiki distributions contain a program called the Offline Review Tool (ORT), which allows a user-provided set of edits to be quickly reviewed and annotated. We believe this tool will prove helpful to corpus-building researchers.
STiki was written byAndrew G. West (west.andrew.g) while a doctoral student in computer science at theUniversity of Pennsylvania, under the guidance ofInsup Lee. Theacademic paper which shaped the STiki methodology was co-authored bySampath Kannan andInsup Lee. The work was supported in part byONR-MURI-N00014-07-1-0907.
In addition to the already discussed academic paper, there have been several STiki-specific write-ups/publications that may prove useful to anti-vandalism developers. The STiki software was presented in aWikiSym 2010 demonstration, and aWikiSym 2010 poster visualizes this content and provides some STiki-revert statistics. STiki was also presented at Wikimania 2010, with the followingpresentation slides. Anadditional writing (not peer reviewed) examines STiki and anti-vandalism techniques as they relate to the larger issue oftrust in collaborative applications. Finally, the anti-damage ecosystem and STiki's technical contribution were summarized in the developer'sPhD dissertation. That work is novel in analyzing ~1 million STiki classification actions to learn about human/social aspects of the patrolling process.
Beyond STiki in isolation, aCICLing 2011 paper examined STiki's metadata scoring technique relative (and in combination with)NLP and content-persistence features (the top 2 finishers from the2010 PAN Competition) – and set new performance baselines in the process. A 2011 edition of the PAN-CLEF competition was also held and required multiple natural-languages to be processed; theSTiki entry won at all tasks. AWikimania 2011 Presentation surveyed the rapid anti-vandalism progress (both academic and on-wiki) of the 2010–2011 time period. Finally, aresearch bulletin published byEDUCAUSE looks at the issue of Wikipedia/wiki damage from an organizational and higher-education perspective with particular emphasis on the protection of institutional welfare.
Queries not addressed by these writings should be addressed to STiki's authors.
For those who would like to show their support for STiki via auserbox, the following have been created/made-available:
|
|
| ||||||
|
|
| ||||||
|
|
| ||||||
 {{STiki topicon}} |
Other STiki images, adverts, promotional material, and statistics:
| ||||
|
 | The da Vinci Barnstar | |
| I used to be a staunch Huggle user for about a year. Then when I stumbled across STiki, I found it to be faster and much more enjoyable to use. Consider me converted. :)15:21, 30 January 2011 (UTC)[reply] |
| The da Vinci Barnstar | |
| In recognition of an outstanding technical achievement. :)™23:00, 22 February 2011 (UTC)[reply] |
 | The Anti-Vandalism Barnstar |
| A cool tool!Lotje (talk)06:31, 14 March 2012 (UTC)[reply] |
 | The Technical Barnstar |
| For your work on STiki – thanks!Allens (talk |contribs)10:49, 12 April 2012 (UTC)[reply] |
| The Anti-Vandalism Barnstar |
| This is in honor of your fight against vandalism on Wikipedia!Allen (Morriswa) (talk)00:18, 13 April 2012 (UTC)[reply] |
 | The Defender of the Wiki Barnstar |
| This is for your work on STiki.Allen (Morriswa) (talk)00:19, 13 April 2012 (UTC)[reply] |
 | The Graphic Designer's Barnstar | |
| For this amazing tool-Amazing graphics as well.User:Deathlaser |
 | Here's atoast to thehost |
| On Completion of 100,000 Reverts by STiki. Cheers !!!--20:31, 28 April 2012 (UTC)[reply] | |
| The Anti-Vandalism Barnstar |
| For your contributions to STiki and the fight against vandalism in Wikipedia. Let's make STiki and Wikipedia even better! ★ Oliverlyc ★ ✈✈✈Pop me a message!10:46, 13 May 2012 (UTC)[reply] |
 | The Tireless Contributor Barnstar | |
| For all your marvelous contributions and creating such a marvelous tool for us.Faizan Munawar Varyatalk10:01, 21 June 2012 (UTC)[reply] |
| The Technical Barnstar |
| Most deserving!!!!!Morning277 (talk)15:10, 28 June 2012 (UTC)[reply] |
 | The Platinum STiki Barnstar of Merit | |
| Congratulations, STiki! You're receiving this barnstar of merit because you recently crossed the50,000 classification threshold usingSTiki. We thank you both foryour contributions to Wikipedia at-large and your use of the tool. We hope you continue your ascent up theleaderboard and stay in touch at thetalk page. Thank you and keep up the good work!West.andrew.g (talk)03:07, 17 September 2012 (UTC)[reply] |
 | The Random Acts of Kindness Barnstar |
| Thank you for the Bronze STiki Barnstar of Merit! Your tool is really such a big help in this encyclopedia. Have a nice day! :) 00:21, 25 November 2012 (UTC)[reply] |
 | The Technical Barnstar | |
| Developing STiki, one of the most awesome anti-vandalism tools on Wikipedia._ |
 | The Original Barnstar |
| Thanks for all the work you made! I appreciate it.Eyesnore19:02, 2 February 2013 (UTC)[reply] |
 | The Pakistan Barnstar of National Merit | |
| For your untiring and determined efforts for the Encyclopedia. Thank you forSTiki!Faizan13:36, 21 May 2013 (UTC)[reply] |
 | CVU Anti-Vandalism Award | |
| For creating a tool that not only helps you to revert vandalism but enables others to do the same efficiently and effectively!Jackson Peebles (talk)06:30, 25 May 2013 (UTC)[reply] |
 | Anti-Vandalism Hall Of Fame |
| Congrats! For creating STiki, I award you with a spot in the AV HOF.User:Buffbills7701/Anti-Vandalism Hall Of Famebuffbills770115:50, 30 July 2013 (UTC)[reply] |
 | The Barnstar of Integrity | |
| STiki roots out COI edits as well as vandalism, and is an outstanding weapon against selfish interests exploiting Wikipedia.Coretheapple (talk)18:06, 16 November 2013 (UTC)[reply] |
 | The Brilliant Idea Barnstar |
| For all your bright ideas at STIKI ! —Keithbob •Talk •21:17, 9 December 2013 (UTC)[reply] |
| The Anti-Vandalism Barnstar |
| Very good program used to fight vandalism!TheEpTic (talk)15:51, 16 December 2013 (UTC)[reply] |
| The Anti-Vandalism Barnstar | |
| Clever use of ClueBot NG data to filter out vandalism; much more efficient thanHuggle's filtered edits feature.Esquivaliencet02:59, 1 May 2015 (UTC)[reply] |
| The Original Barnstar |
| STiki is the best anti-vandalism tool on the Wiki, and I appreciate all you do to fight vandalism. Thank you for making Wikipedia a better place! -Liancetalk/contribs04:02, 4 December 2015 (UTC)[reply] |
 | The STiki Barnstar | |
| You’re the first person to receive this barnstar; you indisputably deserve this honor because you created STiki in the first place.Centibyte(talk)14:18, 28 March 2018 (UTC)[reply] |
| The Anti-Vandalism Barnstar |
| STiki has enabled me to find months-old vandalism that probably would have stayed online for much longer had it not been queued. Thank you for creating this amazing tool!EclipseDude (talk)08:34, 2 September 2018 (UTC)[reply] |
| The Technical Barnstar |
| This is for your hardworks on enhancing Wikipedia through technical works especially developing the powerful anti-vandalism tool STiki. Thank you.PATHSLOPU09:30, 3 June 2019 (UTC)[reply] |
