octokit/plugin-paginate-rest.jsPublic

NotificationsYou must be signed in to change notification settings
Fork33
Star56

Octokit plugin to paginate REST API endpoint responses

License

MIT license

56 stars 33 forks Branches Tags Activity

Star

Notifications

You must be signed in to change notification settings

Branches Tags

Folders and files

Name		Name	Last commit message	Last commit date
Latest commit History 757 Commits
.github		.github
scripts		scripts
src		src
test		test
.gitignore		.gitignore
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
package-lock.json		package-lock.json
package.json		package.json
tsconfig.json		tsconfig.json
vite.config.js		vite.config.js

Repository files navigation

plugin-paginate-rest.js

Octokit plugin to paginate REST API endpoint responses

Usage

Browsers

Browsers	Load`@octokit/plugin-paginate-rest` and`@octokit/core` (or core-compatible module) directly fromesm.sh <scripttype="module">import{Octokit}from"https://esm.sh/@octokit/core";import{paginateRest,composePaginateRest,}from"https://esm.sh/@octokit/plugin-paginate-rest";</script>
Node	Install with`npm install @octokit/core @octokit/plugin-paginate-rest`. Optionally replace`@octokit/core` with a core-compatible module import{Octokit}from"@octokit/core";import{paginateRest,composePaginateRest,}from"@octokit/plugin-paginate-rest";

Load@octokit/plugin-paginate-rest and@octokit/core (or core-compatible module) directly fromesm.sh

<scripttype="module">import{Octokit}from"https://esm.sh/@octokit/core";import{paginateRest,composePaginateRest,}from"https://esm.sh/@octokit/plugin-paginate-rest";</script>

Node

Install withnpm install @octokit/core @octokit/plugin-paginate-rest. Optionally replace@octokit/core with a core-compatible module

import{Octokit}from"@octokit/core";import{paginateRest,composePaginateRest,}from"@octokit/plugin-paginate-rest";

constMyOctokit=Octokit.plugin(paginateRest);constoctokit=newMyOctokit({auth:"secret123"});// See https://developer.github.com/v3/issues/#list-issues-for-a-repositoryconstissues=awaitoctokit.paginate("GET /repos/{owner}/{repo}/issues",{owner:"octocat",repo:"hello-world",since:"2010-10-01",per_page:100,});

If you want to utilize the pagination methods in another plugin, usecomposePaginateRest.

functionmyPlugin(octokit,options){return{allStars({owner, repo})=>{returncomposePaginateRest(octokit,"GET /repos/{owner}/{repo}/stargazers",{owner, repo})}}}

Important

As we useconditional exports, you will need to adapt yourtsconfig.json. See the TypeScript docs onpackage.json "exports".

`octokit.paginate()`

ThepaginateRest plugin adds a newoctokit.paginate() method which accepts the same parameters asoctokit.request. Only "List ..." endpoints such asList issues for a repository are supporting pagination. Theirresponse includes a Link header. For other endpoints,octokit.paginate() behaves the same asoctokit.request().

Theper_page parameter is usually defaulting to30, and can be set to up to100, which helps retrieving a big amount of data without hitting the rate limits too soon.

An optionalmapFunction can be passed to map each page response to a new value, usually an array with only the data you need. This can help to reduce memory usage, as only the relevant data has to be kept in memory until the pagination is complete.

constissueTitles=awaitoctokit.paginate("GET /repos/{owner}/{repo}/issues",{owner:"octocat",repo:"hello-world",since:"2010-10-01",per_page:100,},(response)=>response.data.map((issue)=>issue.title),);

ThemapFunction gets a 2nd argumentdone which can be called to end the pagination early.

constissues=awaitoctokit.paginate("GET /repos/{owner}/{repo}/issues",{owner:"octocat",repo:"hello-world",since:"2010-10-01",per_page:100,},(response,done)=>{if(response.data.find((issue)=>issue.title.includes("something"))){done();}returnresponse.data;},);

Alternatively you can pass arequest method as first argument. This is great when using in combination with@octokit/plugin-rest-endpoint-methods:

constissues=awaitoctokit.paginate(octokit.rest.issues.listForRepo,{owner:"octocat",repo:"hello-world",since:"2010-10-01",per_page:100,});

`octokit.paginate.iterator()`

If your target runtime environments supports async iterators (such as most modern browsers and Node 10+), you can iterate through each response

constparameters={owner:"octocat",repo:"hello-world",since:"2010-10-01",per_page:100,};forawait(constresponseofoctokit.paginate.iterator("GET /repos/{owner}/{repo}/issues",parameters,)){// do whatever you want with each response, break out of the loop, etc.constissues=response.data;console.log("%d issues found",issues.length);}

Alternatively you can pass arequest method as first argument. This is great when using in combination with@octokit/plugin-rest-endpoint-methods:

constparameters={owner:"octocat",repo:"hello-world",since:"2010-10-01",per_page:100,};forawait(constresponseofoctokit.paginate.iterator(octokit.rest.issues.listForRepo,parameters,)){// do whatever you want with each response, break out of the loop, etc.constissues=response.data;console.log("%d issues found",issues.length);}

`composePaginateRest` and`composePaginateRest.iterator`

Thecompose* methods work just like theiroctokit.* counterparts described above, with the differenct that both methods require anoctokit instance to be passed as first argument

How it works

octokit.paginate() wrapsoctokit.request(). As long as arel="next" link value is present in the response'sLink header, it sends another request for that URL, and so on.

Most of GitHub's paginating REST API endpoints return an array, but there are a few exceptions which return an object with a key that includes the items array. For example:

Search repositories (keyitems)
List check runs for a specific ref (key:check_runs)
List check suites for a specific ref (key:check_suites)
List repositories for an installation (key:repositories)
List installations for a user (keyinstallations)
Compare commits (keycommits)

octokit.paginate() is working around these inconsistencies so you don't have to worry about it.

If a response is lacking theLink header,octokit.paginate() still resolves with an array, even if the response returns a single object.

Types

The plugin also exposes some types and runtime type guards for TypeScript projects.

Types	import{PaginateInterface,PaginatingEndpoints,}from"@octokit/plugin-paginate-rest";
Guards	import{isPaginatingEndpoint}from"@octokit/plugin-paginate-rest";

PaginateInterface

Aninterface that declares all the overloads of the.paginate method.

PaginatingEndpoints

Aninterface which describes all API endpoints supported by the plugin. Some overloads of.paginate() method andcomposePaginateRest() function depend onPaginatingEndpoints, using thekeyof PaginatingEndpoints as a type for one of its arguments.

import{Octokit}from"@octokit/core";import{PaginatingEndpoints,composePaginateRest,}from"@octokit/plugin-paginate-rest";typeDataType<T>="data"extendskeyofT ?T["data"] :unknown;asyncfunctionmyPaginatePlugin<EextendskeyofPaginatingEndpoints>(octokit:Octokit,endpoint:E,parameters?:PaginatingEndpoints[E]["parameters"],):Promise<DataType<PaginatingEndpoints[E]["response"]>>{returnawaitcomposePaginateRest(octokit,endpoint,parameters);}

isPaginatingEndpoint

A type guard,isPaginatingEndpoint(arg) returnstrue ifarg is one of the keys inPaginatingEndpoints (iskeyof PaginatingEndpoints).

import{Octokit}from"@octokit/core";import{isPaginatingEndpoint,composePaginateRest,}from"@octokit/plugin-paginate-rest";asyncfunctionmyPlugin(octokit:Octokit,arg:unknown){if(isPaginatingEndpoint(arg)){returnawaitcomposePaginateRest(octokit,arg);}// ...}