Continuous Integration

Extension integration tests can be run on CI services. The@vscode/test-electron library helps you set up extension tests on CI providers and contains asample extension setup on Azure Pipelines. You can check out thebuild pipeline or jump directly to theazure-pipelines.yml file.

Automated publishing

You can also configure the CI to publish a new version of the extension automatically.

The publish command is similar to publishing from a local environment usingvsce, but you must somehow provide the Personal Access Token (PAT) in a secure way. By storing the PAT as aVSCE_PATsecret variable,vsce will be able to use it. Secret variables are never exposed, so they are safe to use in a CI pipeline.

Azure Pipelines

Azure Pipelines is great for running VS Code extension tests as it supports running the tests on Windows, macOS, and Linux. For Open Source projects, you get unlimited minutes and 10 free parallel jobs. This section explains how to set up an Azure Pipelines for running your extension tests.

First, create a free account onAzure DevOps and create anAzure DevOps project for your extension.

Then, add the followingazure-pipelines.yml file to the root of your extension's repository. Other than thexvfb setup script for Linux that is necessary to run VS Code in headless Linux CI machines, the definition is straight-forward:

trigger:  branches:    include:    -main  tags:    include:    -v*strategy:  matrix:    linux:      imageName:'ubuntu-latest'    mac:      imageName:'macos-latest'    windows:      imageName:'windows-latest'pool:  vmImage:$(imageName)steps:-task:NodeTool@0  inputs:    versionSpec:'10.x'  displayName:'Install Node.js'-bash:|    /usr/bin/Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 &    echo ">>> Started xvfb"  displayName:Start xvfb  condition:and(succeeded(), eq(variables['Agent.OS'], 'Linux'))-bash:|    echo ">>> Compile vscode-test"    yarn && yarn compile    echo ">>> Compiled vscode-test"    cd sample    echo ">>> Run sample integration test"    yarn && yarn compile && yarn test  displayName:Run Tests  env:    DISPLAY:':99.0'

Finally,create a new pipeline in your DevOps project and point it to theazure-pipelines.yml file. Trigger a build and voilà:

You can enable the build to run continuously when pushing to a branch and even on pull requests. SeeBuild pipeline triggers to learn more.

Azure Pipelines automated publishing

1. Set upVSCE_PAT as a secret variable using theAzure DevOps secrets instructions.
2. Installvsce as adevDependencies (npm install @vscode/vsce --save-dev oryarn add @vscode/vsce --dev).
3. Declare adeploy script inpackage.json without the PAT (by default,vsce will use theVSCE_PAT environment variable as the Personal Access Token).

"scripts": {  "deploy":"vsce publish --yarn"}

4. Configure the CI so the build will also run when tags are created:

trigger:  branches:    include:    -main  tags:    include:    -refs/tags/v*

5. Add apublish step inazure-pipelines.yml that callsyarn deploy with the secret variable.

-bash:|    echo ">>> Publish"    yarn deploy  displayName:Publish  condition:and(succeeded(), startsWith(variables['Build.SourceBranch'], 'refs/tags/'), eq(variables['Agent.OS'], 'Linux'))  env:    VSCE_PAT:$(VSCE_PAT)

Thecondition property tells the CI to run the publish step only in certain cases.

In our example, the condition has three checks:

• succeeded() - Publish only if the tests pass.
• startsWith(variables['Build.SourceBranch'], 'refs/tags/') - Publish only if a tagged (release) build.
• eq(variables['Agent.OS'], 'Linux') - Include if your build runs on multiple agents (Windows, Linux, etc.). If not, remove that part of the condition.

SinceVSCE_PAT is a secret variable, it is not immediately usable as an environment variable. Thus, we need to explicitly map the environment variableVSCE_PAT to the secret variable.

GitHub Actions

You can also configure GitHub Actions to run your extension CI. In headless Linux CI machinesxvfb is required to run VS Code, so if Linux is the current OS run the tests in an Xvfb enabled environment:

on:  push:    branches:      -mainjobs:  build:    strategy:      matrix:        os: [macos-latest,ubuntu-latest,windows-latest]    runs-on:${{ matrix.os }}    steps:    -name:Checkout      uses:actions/checkout@v4    -name:Install Node.js      uses:actions/setup-node@v4      with:        node-version:18.x    -run:npm install    -run:xvfb-run -a npm test      if:runner.os == 'Linux'    -run:npm test      if:runner.os != 'Linux'

GitHub Actions automated publishing

1. Set upVSCE_PAT as an encrypted secret using theGitHub Actions secrets instructions.
2. Installvsce as adevDependencies (npm install @vscode/vsce --save-dev oryarn add @vscode/vsce --dev).
3. Declare adeploy script inpackage.json without the PAT.

"scripts": {  "deploy":"vsce publish --yarn"}

4. Configure the CI so the build will also run when tags are created:

on:  push:    branches:    -main  release:    types:    -created

5. Add apublish job to the pipeline that callsnpm run deploy with the secret variable.

-name:Publish  if:success() && startsWith(github.ref, 'refs/tags/') && matrix.os == 'ubuntu-latest'  run:npm run deploy  env:    VSCE_PAT:${{ secrets.VSCE_PAT }}

Theif property tells the CI to run the publish step only in certain cases.

In our example, the condition has three checks:

• success() - Publish only if the tests pass.
• startsWith(github.ref, 'refs/tags/') - Publish only if a tagged (release) build.
• matrix.os == 'ubuntu-latest' - Include if your build runs on multiple agents (Windows, Linux, etc.). If not, remove that part of the condition.

GitLab CI

GitLab CI can be used to test and publish the extension in headless Docker containers. This can be done by pulling a preconfigured Docker image, or installingxvfb and the libraries required to run Visual Studio Code during the pipeline.

image:node:12-busterbefore_script:  -npm installtest:  script:    -|      apt update      apt install -y libasound2 libgbm1 libgtk-3-0 libnss3 xvfb      xvfb-run -a npm run test

GitLab CI automated publishing

1. Set upVSCE_PAT as a masked variable using theGitLab CI documentation.
2. Installvsce as adevDependencies (npm install @vscode/vsce --save-dev oryarn add @vscode/vsce --dev).
3. Declare adeploy script inpackage.json without the PAT.

"scripts": {  "deploy":"vsce publish --yarn"}

4. Add adeploy job that callsnpm run deploy with the masked variable which will only trigger on tags.

deploy:  only:    -tags  script:    -npm run deploy

Common questions

Do I need to use Yarn for continuous integration?

All of the above examples refer to a hypothetical project built withYarn, but can be adapted to usenpm,Grunt,Gulp, or any other JavaScript build tool.