Repository files navigation

This repo enables matplotlib.org hosting usingCaddy.

At the top level, Caddy exposes thempl-brochure-site repository withfallback to files in thematplotlib.github.com repository. The followingrelated projects are also exposed as toplevel directories:

• basemap
• cheatsheets
• cycler
• devdocs
• governance
• matplotblog
• mpl-altair
• mpl-bench
• mpl-gui
• mpl-sphinx-theme
• mpl-third-party
• data-prototype

When running on DigitalOcean hosting, an Ansible playbook is used to configurethe server with consistent settings.

Before you can run our Ansible playbooks, you need to meet the followingprerequisites:

• Create a DigitalOcean API token, and pass it to the inventory generator bysetting theDO_API_TOKEN environment variable. The API token must haveaccess to the following scopes:
  • Read: droplet, firewall, monitoring, project, ssh_key
  • Create: droplet
  • Update: droplet, monitoring, project
• If you are creating a new droplet, and want to configure DNS as well, thencreate a CloudFlare API token with DNS edit permissions, and pass it to theAnsible playbook by setting theCLOUDFLARE_TOKEN environment variable.
• Set the vault decryption password of the Ansible vaulted file with oursecrets. This may be done by setting theANSIBLE_VAULT_PASSWORD_FILEenvironment variable to point to a file containing the password.
• Download all the collections the playbooks depend on with the followingcommand:

  ansible-galaxy collection install \  --requirements-file collections/requirements.yml

You may wish to usedirenv to set environment variables.

There is currently only one playbook:

• matplotlib.org.yml, for the main matplotlib.org hosting. This playbookoperates on droplets with thewebsite tag in DigitalOcean.

When a new repository is added to the Matplotlib organization withdocumentation (or an existing repository adds documentation), it will benecessary to re-configure the server to serve those files. Note, it iscurrently assumed that the documentation is on thegh-pages branch of therepository, and it will be served from the top-level subdirectory with the samename as the repository (similar to GitHub Pages.) There are 4 steps to achievethis:

1. Generate a secret to secure the webhook. You can followGitHub'sinstructions for creatingone.

2. Add repository to Ansible:

   1. Add an entry to therepos variable at the top ofmatplotlib.org.yml.
   2. Add the webhook secret tofiles/webhook_vars.yml.

3. Re-run Ansible on the playbook likebelow. This shouldclone the new repository and update the webhook handler.

4. Configure a webhook on the new repository with the following settings:

   • Payload URL ofhttps://do.matplotlib.org/gh/<repository>
   • Content type of application/json
   • Use the secret generated in step 1
   • Trigger only on "push" events

If everything is done correctly, the GitHub webhook should have posted aninitial "ping" event successfully, and documentation should be available athttps://matplotlib.org/<repository>.

We follow a simplified version of the naming scheme onthis blogpost:

• Servers are named<prefix>.matplotlib.org in A records, pointing to theIPv4 address of the droplet.
• Servers get a functional CNAME alias (e.g.,web01.matplotlib.org) pointingto the hostname<prefix>.matplotlib.org.
• matplotlib.org is a CNAME alias of the functional CNAME of a server.

We useplanets in our Solar Systemfor the name prefix. When creating a new server, pick the next one in the list.

The summary of the initial setup is:

1. Create the droplet with monitoring and relevant SSH keys.
2. Assign new droplet to the matplotlib.org project.
3. Add DNS entries pointing to the server on CloudFlare.
4. Grab the SSH host fingerprints.
5. Reboot.

We currently use a simple $12 droplet from DigitalOcean. You can create onefrom the control panel, or using thecreate.yml Ansible playbook:

ansible-playbook create.yml

This playbook will prompt you for 3 settings:

1. The host name of the droplet, which should follow the naming conventionabove.
2. The functional CNAME alias of the droplet.
3. The names of SSH keys to add to the droplet.

You may also pass these directly to Ansible as:

ansible-playbook create.yml --extra-vars "host=pluto functional=web99 ssh_keys='a b c'"

The playbook will create the server, as well as add DNS records on CloudFlare.Note, you must setDO_API_TOKEN andCLOUDFLARE_TOKEN in the environment toaccess these services. The droplet ID and IP address will be printed at theend of the playbook.

Then, to ensure you are connecting to the expected server, you should grab theSSH host keys via the DigitalOcean Droplet Console:

for f in /etc/ssh/ssh_host_*_key; do  ssh-keygen -l -f $f;done

Note down the outputs to verify later, e.g.,

# Use these for comparison when connecting yourself.256 SHA256:p6MiA8+IO1WcpXHDOQ4rhiVCo+MDxWB7ehfNfxvbDkU root@venus.matplotlib.org (ECDSA)256 SHA256:RfDahJqnQFLeFN+zl9f+hmB+W05OoZK26NfNQkj6KtY root@venus.matplotlib.org (ED25519)3072 SHA256:tYwdULlz5/XP5Ze7PCj9XpO3VIMEZkiOiFuhr9nke34 root@venus.matplotlib.org (RSA)

Finally, you should reboot the droplet. This is due to a bug in cloud-init onDigitalOcean, which generates a new machine ID after startup, causing systemlogs to be seem invisible.

This can be done from the Console, or via the CLI:

doctl compute droplet-action reboot <droplet-id>

You must setup Ansible as described above. Verify that the new droplet isvisible to Ansible by running:

ansible-inventory --graph

which should list the new droplet in thewebsite tag:

@all:  |--@website:  |  |--venus.matplotlib.org

Then execute the Ansible playbook on the servers by running:

ansible-playbook --user root matplotlib.org.yml

During the initial "Gathering Facts" task, you will be prompted to accept theserver's SSH fingerprint, which you should verify against the values foundearlier. If there are existing servers that you don't want to touch, then youcan use the--limit option. If you are using a non-default SSH key, you maywish to use the--private-key option.

You can verify that the server is running correctly by connecting tohttps://<prefix>.matplotlib.org in your browser.

Once everything is running, you should flip the DNS for the main site, changingthematplotlib.org CNAME to point to the new server'swebNN.matplotlib.orgfunctional name.