macbre/nodemwPublic

NotificationsYou must be signed in to change notification settings
Fork56
Star244

MediaWiki API and WikiData client written in Node.js

License

BSD-2-Clause license

244 stars 56 forks Branches Tags Activity

Star

Notifications

You must be signed in to change notification settings

Branches Tags

Folders and files

Name		Name	Last commit message	Last commit date
Latest commit History 1,170 Commits
.github		.github
examples		examples
lib		lib
test		test
.gitignore		.gitignore
.npmignore		.npmignore
.prettierignore		.prettierignore
.snyk		.snyk
AUTHORS		AUTHORS
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
LICENSE		LICENSE
README.md		README.md
eslint.config.js		eslint.config.js
jest.config.js		jest.config.js
package-lock.json		package-lock.json
package.json		package.json
tsconfig.json		tsconfig.json

Repository files navigation

nodemw

MediaWiki API client written in node.js

Requirements

At minimumNode.js at the maintenance version

Installation

Using npm

npm install nodemw

OrDownload the latest stable version via GitHub.

Development version

git clone https://github.com/macbre/nodemw.git

Features

HTTP requests are stored in the queue and performed in parallel with limited number of "threads" (i.e. there's no risk of flooding the server)
articles creation / edit / move / delete
file uploads (using given content or via provided URL)
Special:Log processing
listing articles in categories
and much more
getting claims fromWikiData

Where it's used

Over 25k edits onPoznań Wiki asPyrabot -scripts repository

First script

An example script can be found in/examples directory.

cd examplesnode pagesInCategory.js

You can enterdebug mode by settingDEBUG enviromental variable:

DEBUG=1 node examples/pagesInCategory.js

You can enterdry-run mode (all "write" operations like edits and uploads will be disabled) by settingDRY_RUN environmental variable (ordryRun entry in the config):

DRY_RUN=1 node examples/pagesInCategory.js

Running unit tests

npmtest

How to use it?

Creating a bot instance

varbot=require("nodemw");// pass configuration objectvarclient=newbot({protocol:"https",// Wikipedia now enforces HTTPSserver:"en.wikipedia.org",// host name of MediaWiki-powered sitepath:"/w",// path to api.php scriptdebug:false,// is more verbose when set to true});client.getArticle("foo",function(err,data){// error handlingif(err){console.error(err);return;}// ...});

Config file

nodemw can use config files as well as objects directly provided tobot object constructor.

// read config from external filevarclient=newbot("config.js");

Config file is a JSON-encoded object with the following fields (see/examples/config-DIST.js file):

{"protocol":"https",// default to 'http'"server":"en.wikipedia.org",// host name of MediaWiki-powered site"path":"/w",// path to api.php script"debug":false,// is more verbose when set to true"username":"foo",// account to be used when logIn is called (optional)"password":"bar",// password to be used when logIn is called (optional)"domain" :"auth.bar.net",// domain to be used when logIn is called (optional)"userAgent":"Custom UA",// define custom bot's user agent"concurrency":5// how many API requests can be run in parallel (defaults to 3)}

Making direct API calls

nodemw allows you make direct calls to MediaWiki API (example querying Semantic MediaWiki API):

varbot=require("nodemw"),client=newbot({server:"semantic-mediawiki.org",path:"/w",}),params={action:"ask",query:"[[Modification date::+]]|?Modification date|sort=Modification date|order=desc",};client.api.call(params/* api.php parameters */,function(err/* Error instance or null */,info/* processed query result */,next/* more results? */,data/* raw data */,){console.log(data&&data.query&&data.query.results);},);

Bot methods

The last parameter of each function in nodemw API is a callback which will be fired when the requested action is done.

Callbacks use node.js style -err is always passed as the first argument.

bot.logIn(username, password, callback)

Log-in using given credentials -read more

bot.getCategories(prefix, callback)

Gets the list of all categories on a wiki

bot.getAllPages(callback)

Gets the list of all pages from the main namespace (excludes redirects) -read more

bot.getPagesInCategory(category, callback)

Gets the list of pages in a given category -read more

bot.getPagesInNamespace(namespace, callback)

Gets the list of pages in a given namespace -read more

bot.getPagesByPrefix(prefix, callback)

Gets the list of pages by a given prefix -read more

bot.getPagesTranscluding(page, callback)

Gets the list of pages that transclude the given pages -read more

bot.getArticle(title, [redirect,] callback)

Gets article content and redirect info -read more

bot.getArticleRevisions(title, callback)

Gets all revisions of a given article -read more

bot.getArticleCategories(title, callback)

Gets all categories a given article is in -read more

bot.getArticleInfo(title, callback)

Gets all info of a given article -read more

bot.edit(title, content, summary, minor, callback)

Creates / edits an article (and mark the edit as minor ifminor is set to true) -read more

bot.append(title, content, summary, callback)

Adds given content to the end of the page -read more

bot.prepend(title, content, summary, callback)

Adds given content to the beginning of the page -read more

bot.addFlowTopic(title, topic, content, callback)

Add a Flow topic -read more

bot.delete(title, reason, callback)

Deletes an article -read more

bot.purge(titles, callback)

Purge a given list of articles (titles or page IDs can be provided) -read more

By providingCategory:Foo astitles argument you can purge all pages in a given category (available sinceMW 1.21)

bot.protect(title, protections, options, callback)

Protect a page (A title or page ID can be provided) -read more

Theprotections value is an Array of protection information in the format:

{    action: string,    level?: string = 'all',    expiry?: string | number = 'never'}

Calls to the Protect endpoint are not additive. Each call must include a list ofall intended protections, including any already in place. Each call willreplace all existing protections.

bot.sendEmail(username, subject, text, callback)

Send an email to an user -read more

bot.getToken(title, action, callback)

Returns token required for a number of MediaWiki API operations -read more /for MW 1.24+

bot.whoami(callback)

Gets information about current bot's user (including rights and rate limits) -read more

bot.whois(username, callback)

Gets information about a specific user (including rights, current block, groups) -read more

bot.whoare(usernames, callback)

Gets information about specific users (including rights, current block, groups) -read more

bot.createAccount(username, password, callback)

Create account using given credentials -read more

bot.move(from, to, summary, callback)

Moves (aka renames) given article -read more

bot.getImages(callback)

Gets list of all images on a wiki

bot.getImageUsage(filename, callback)

Gets list of all articles using given image

bot.getImagesFromArticle(title, callback)

Get list of all images that are used on a given page -read more

bot.getImageInfo(filename, callback)

Gets metadata (including uploader, size, dimensions and EXIF data) of given image

bot.getLog(type, start, callback)

Get entries form Special:Log -read more

bot.expandTemplates(content, title, callback)

Returns XML with preprocessed wikitext -read more

bot.parse(content, title, callback)

Returns parsed wikitext -read more

bot.fetchUrl(url, callback)

Makes a GET request to provided resource and returns its content.

bot.getRecentChanges(start, callback)

Returns entries from recent changes (starting from a given point)

bot.getSiteInfo(props, callback)

Returns site information entries -read more

bot.getSiteStats(props, callback)

Returns site statistics (number of articles, edits etc) -read more

bot.getMediaWikiVersion(callback)

Returns the version of MediaWiki given site uses -read more

client.getQueryPage(queryPage, callback)

Returns entries fromQueryPage-based special pages

bot.upload(filename, content, summary/* or extraParams */, callback)

Uploads a given raw content as a File:[filename] -read more

bot.uploadByUrl(filename, url, summary/* or extraParams */, callback)

Uploads a given external resource as a File:[filename]

bot.uploadVideo(fileName, url, callback)

Uploads a given video as a File:[filename] (Wikia-specific API)

bot.getTemplateParamFromXml(tmplXml, paramName)

Gets a value of a given template parameter from article's preparsed content (see expandTemplates)

bot.getExternalLinks(title, callback)

Gets all external links used in article

bot.getBacklinks(title, callback)

Gets all articles that links to given article

bot.search(query, callback)

Performs a search

Helpers

bot.getConfig(key, def)

Gets config entry value (returnsdef value if not found)

bot.setConfig(key, val)

Sets config entry value

bot.diff(old, current)

Returns a diff colored using ANSI colors (powered bydiff)

Wikia-specific bot methods

They're grouped inbot.wikia "namespace".

bot.wikia.getWikiVariables(callback)

Get wiki-specific settings (like ThemeDesigner colors and hubs).

bot.wikia.getUser(userId, callback)

Get information (avatar, number of edits) about a given user

bot.wikia.getUsers(userIds, callback)

Get information (avatar, number of edits) about a given set of users (by their IDs)

WikiData

This API is Promise-based, useawait keyword.

Examples:

constwikidata=require("nodemw/lib/wikidata");constclient=newwikidata();// Where is Saksun, Faroe Islands located?constgeo=awaitclient.getEntityClaim("Q928875"/* Saksun */,"P625"/* place location */,);// will give you the geolocation of the placeexpect(geo[0].mainsnak.datavalue.value).toMatchObject({latitude:62.248888888889,longitude:-7.1758333333333,});// When was Albert Einstein born?constres=awaitclient.getArticleClaims("Albert Einstein");constdateOfBirth=res.P569[0].mainsnak.datavalue.value;expect(dateOfBirth.time).toMatch(/1879-03-14/);constdateOfDeath=res.P570[0].mainsnak.datavalue.value;expect(dateOfDeath.time).toMatch(/1955-04-18/);// interwiki links for a given artliceconstlinks=awaitclient.getArticleSitelinks("Albert Einstein");console.log(links.enwiki);// {site: "enwiki", title: "Albert Einstein", badges: ["Q17437798"]}