Repository files navigation

A repository for Ceph (and Ceph-related projects) so that they can beautomatically configured in Jenkins.

The current state of the repo is oftransition from single scripts toa properly structured one with directories that represent each project.

The structure is strict and provides a convention to set the order of executionof build scripts.

Job configuration is done via the CLI appJenkins Job Builderon the actual directory forits own job (the job has itsdefinition and its build process automated).

The JJB configuration defines the rules needed to generate and update/createall other Jenkins Jobs in this repo as long as they define theconfig/definitions along with a valid YAML file.

This script should have all the rules and requirements for generating theJenkins configurations needed from the YAML files to create/update the jenkinsjob.

Any script in the top level of this repo has been removed, and temporarilyplaced in thedeprecated branch. If a job requires any of those it shouldbe ported to follow the structure of the Jenkins Job Builder project, like allthe current jobs in this repository.

Thedeprecated branch will be removed by the end of 2018.

Any jobs removed from this repo will be automatically deleted by JJB.

The rules and structure for the builds arestrictly enforced. If theconvention is not followed, the builds will not work.

Changing Jenkins jobs in Jenkins isstrongly discouraged. Changingsomething in the Jenkins UI does not guarantee it will persist and willprobably be overwritten.

By default, this is how a directory tree would look like for a build fora project calledfoo that uses every choice available:

foo├── config|   ├── config|   └── definitions|       └── foo.yml├── setup|   ├── setup|   ├── post|   └── pre└── build    ├── build    ├── post    └── pre

This structure consists of two directories with scripts and one forconfiguration. The scripts should be included in thefoo.yml file inwhatever order the job requires.

For example, this is how it could look in thebuilders section for itsconfiguration:

builders:  # Setup scripts  - shell: !include-raw ../../setup/pre  - shell: !include-raw ../../setup/setup  - shell: !include-raw ../../setup/post  # Build scripts  - shell: !include-raw ../../build/pre  - shell: !include-raw ../../build/build  - shell: !include-raw ../../build/post

These scripts will be added to the Jenkins server so that they can be executedas part of a job.

Each Jenkins job has two names:

1. The main name for a job. This is thename: parameter in YAML.
2. The human-friendly "display name" for a job. This is thedisplay-name:parameter in YAML.

For regular jobs, we name the Jenkins job after the git repository name. Forexample, the "ceph-deploy" package is athttps://github.com/ceph/ceph-deploy,so the job name is "ceph-deploy".

For Pull Request jobs, we use a similar convention for both the internal jobname and the human readable "display name". For example, if the git repositoryis "ceph-deploy", then we name the Jenkins jobceph-deploy-pull-requests.Thedisplay-name is set toceph-deploy: Pull Requests. In other words,to determine adisplay-name for a job that handles pull requests, simplyappend: Pull Requests to thename value.

In other words, for building pull requests to ceph-deploy, the Jenkins job YAMLwill have the following settings:

• Git repo:https://github.com/ceph/ceph-deploy
• Jenkins jobname:ceph-deploy-pull-requests
• Jenkins jobdisplay-name:ceph-deploy: Pull Requests

Scripts that may hang should be using thetimeout command:

timeout 600 ./bad-script.sh

The above command will make the job expire after ten minutes (the argument isin seconds).

When configuring a new job that will build pull requests, you must alsoconfigure GitHub's repository to notify Jenkins of new pull requests.

1. In GitHub's web interface, click the "Settings" button for your repository.
2. Click the "Webhooks & Services" link in the "Options" menu on the left.
3. Under the "Webhooks" section, set the "Payload URL" tohttp://jenkins.ceph.com/ghprbhook/.
4. Click the "Content type" dropdown and selectapplication/x-www-form-urlencoded.
5. For the question "Which events would you like to trigger this webhook?",select theLet me select individual events. radio, and check thePullRequest andIssue comment boxes.
6. Click the green "Update Webhook" button to save your changes.

On the Jenkins side, you should set up the job's GitHub project URL like so:

- job:    name: jenkins-slave-chef-pull-requests    ...    properties:      - github:          url: https://github.com/ceph/jenkins-slave-chef

This will tell the Jenkins GitHub Pull Requests plugin that it shouldassociate the incoming webhooks with this particular job.

You should also use thetriggers setting for the job, like so:

- job:    name: jenkins-slave-chef-pull-requests    ...    triggers:      - github-pull-request:          cron: '* * * * *'          admin-list:            - alfredodeza            - ktdreyer          org-list:            - ceph          trigger-phrase: 'retest this please'          only-trigger-phrase: false          github-hooks: true          permit-all: false          auto-close-on-fail: false

Some jobs don't actually run code; they simply build a project's documentationand upload the docs to ceph.com. One example is the "teuthology-docs-build"job.

For these jobs, note that the destination directory must be created on theceph.com web server before thersync command will succeed.

Jenkins can periodically poll Git repos on github.com for changes, but this isslow and inefficient. Instead of polling GitHub, it's best to use GitHub's web hooks instead.

See the "jenkins-job-builder" job as an example.

1. Set up thetriggers section:

   triggers:  - github

2. Visit the GitHub repository's "settings" page, eghttps://github.com/ceph/ceph-build/settings/hooks, and add a new web hook.

   • The Payload URL should behttps://jenkins.ceph.com/github-webhook/(note the trailing slash)
   • TheContent type should beapplication/x-www-form-urlencoded
   • Secret should be blank
   • SelectJust send the push event.

Sometimes it's useful to test a JJB change by hand prior to merging a pullrequest.

1. Installjenkins-job-builder on your local computer.

2. Create$HOME/.jenkins_jobs.ini on your local computer:

   [jenkins]user=ktdreyerpassword=a8b767bb9cf0938dc7f40603f33987e5url=https://jenkins.ceph.com/

Whereuser is your Jenkins (ie GitHub) account username, andpasswordis your Jenkins API token. (Note, your Jenkins API token can be found @https://jenkins.ceph.com/ , for examplehttps://jenkins.ceph.com/user/ktdreyer/configure)

3. Switch to the Git branch with the JJB changes that you wish to test:

   git checkout <branch with your changes>

Let's say this git branch makes a change in themy-cool-job job.

4. Run JJB to test the syntax of your changes:

   jenkins-jobs --conf ~/.jenkins_jobs.ini test my-cool-job/config/definitions/my-cool-job.yml

   If everything goes ok, this will cause JJB to output the XML of your job(s).If there is a problem, JJB will print an error/backtrace instead.

5. Run JJB to push your changes live to job on the main:

   jenkins-jobs --conf ~/.jenkins_jobs.ini update my-cool-job/config/definitions/my-cool-job.yml

6. Run a throwaway build with your change, and verify that your change didn'tbreak anything and does what you want it to do.

(Note: if anyone merges anything to main during this time, Jenkins will resetall jobs to the state of what is in main, and your customizations will bewiped out. This "by-hand" testing procedure is only intended for short-livedtests.)

We found one Jenkins controller wasn't enough to handle all the jobs we weredemanding of it. The CI now supports multiple Jenkins controllers. If you wish torun your job on a different Jenkins controller:

1. Create aconfig/JENKINS_URL file in your job directory containing onlythe FQDN of the target Jenkins controller:

   # Example$ cat my-cool-job/config/JENKINS_URL2.jenkins.ceph.com

Like many software projects, the Ceph project has undertaken the task ofmigrating to more inclusive language. In the Ceph CI,

master branches are nowmain

slave is nowbuilder

When referring to the main Jenkins server,master is nowcontroller

Remaining references (like the Jenkinsssh-slaves plugin) are hardcodedand could not be changed.