Repository files navigation

This repo enables matplotlib.org hosting usingCaddy.

At the top level, Caddy exposes thempl-brochure-site repository withfallback to files in thematplotlib.github.com repository. The followingrelated projects are also exposed as toplevel directories:

• basemap
• cheatsheets
• cycler
• devdocs
• governance
• matplotblog
• mpl-altair
• mpl-bench
• mpl-gui
• mpl-sphinx-theme
• mpl-third-party
• data-prototype

When running on DigitalOcean hosting, an Ansible playbook is used to configurethe server with consistent settings.

Before you can run our Ansible playbooks, you need to meet the followingprerequisites:

• Create a DigitalOcean API token, and pass it to the inventory generator bysetting theDO_API_TOKEN environment variable.
• Set the vault decryption password of the Ansible vaulted file with oursecrets. This may be done by setting theANSIBLE_VAULT_PASSWORD_FILEenvironment variable to point to a file containing the password.
• Download all the collections the playbooks depend on with the followingcommand:

  ansible-galaxy collection install \  --requirements-file collections/requirements.yml

You may wish to usedirenv to set environment variables.

There is currently only one playbook:

• matplotlib.org.yml, for the main matplotlib.org hosting. This playbookoperates on droplets with thewebsite tag in DigitalOcean.

When a new repository is added to the Matplotlib organization withdocumentation (or an existing repository adds documentation), it will benecessary to re-configure the server to serve those files. Note, it iscurrently assumed that the documentation is on thegh-pages branch of therepository, and it will be served from the top-level subdirectory with the samename as the repository (similar to GitHub Pages.) There are 4 steps to achievethis:

1. Generate a secret to secure the webhook. You can followGitHub'sinstructions for creatingone.

2. Add repository to Ansible:

   1. Add an entry to therepos variable at the top ofmatplotlib.org.yml.
   2. Add the webhook secret tofiles/webhook_vars.yml.

3. Re-run Ansible on the playbook likebelow. This shouldclone the new repository and update the webhook handler.

4. Configure a webhook on the new repository with the following settings:

   • Payload URL ofhttps://do.matplotlib.org/gh/<repository>
   • Content type of application/json
   • Use the secret generated in step 1
   • Trigger only on "push" events

If everything is done correctly, the GitHub webhook should have posted aninitial "ping" event successfully, and documentation should be available athttps://matplotlib.org/<repository>.

We follow a simplified version of the naming scheme onthis blogpost:

• Servers are named<prefix>.matplotlib.org in A records.
• Servers get a functional CNAME alias (e.g.,web01.matplotlib.org).
• matplotlib.org is a CNAME to the functional CNAME of a server.

We useplanets in our Solar Systemfor the name prefix. When creating a new server, pick the next one in the list.

The summary of the initial setup is:

1. Create the droplet with monitoring and relevant SSH keys.
2. Assign new droplet to the matplotlib.org project and the Web firewall.
3. Grab the SSH host fingerprints.
4. Reboot.

We currently use a simple $10 droplet from DigitalOcean. You can create onefrom the control panel, or using thedoctl utility. Be sure to enablemonitoring, and add thewebsite tag and relevant SSH keys to the droplet. Anexample of usingdoctl is the following:

doctl compute droplet create \    --image fedora-35-x64 \    --region tor1 \    --size s-1vcpu-2gb \    --ssh-keys <key-id>,<key-id> \    --tag-name website \    --enable-monitoring \    venus.matplotlib.org

Note, you will have to usedoctl compute ssh-key list to get the IDs of therelevant SSH keys saved on DigitalOcean, and substitute them above. Save the IDof the new droplet from the output, e.g., in:

ID           Name       Public IPv4    Private IPv4    Public IPv6    Memory    VCPUs    Disk    Region    Image            VPC UUID    Status    Tags       Features                    Volumes294098687    mpl.org                                                  2048      1        50      tor1      Fedora 35 x64                new       website    monitoring,droplet_agent

the droplet ID is 294098687.

You should also assign the new droplet to thematplotlib.org project and theWeb firewall:

doctl projects list# Get ID of the matplotlib.org project from the output.doctl projects resources assign <project-id> --resource=do:droplet:<droplet-id>doctl compute firewall list# Get ID of the Web firewall from the output.doctl compute firewall add-droplets <firewall-id> --droplet-ids <droplet-id>

Then, to ensure you are connecting to the expected server, you should grab theSSH host keys via the DigitalOcean Droplet Console:

for f in /etc/ssh/ssh_host_*_key; do  ssh-keygen -l -f $f;done

Note down the outputs to verify later, e.g.,

# Use these for comparison when connecting yourself.1024 SHA256:ExviVyBRoNKsZpgmIfBaejh1ElOpJ/9fC+ki2Fn5Xj4 root@venus.matplotlib.org (DSA)256 SHA256:hLA7ePr0D4AgiC21IXowtbpcUNnTGgpPB7NOYepQtxg root@venus.matplotlib.org (ECDSA)256 SHA256:MggFZQbZ7wID1Se2EmOwAm8AaJeA97L8sD8DhSrKy1g root@venus.matplotlib.org (ED25519)3072 SHA256:MCkDgfbn0sMTCtvAtfD0HmGJV3LVTjpUj6IcfWRHRQo root@venus.matplotlib.org (RSA)

Finally, you should reboot the droplet. This is due to a bug in cloud-init onDigitalOcean, which generates a new machine ID after startup, causing systemlogs to be seem invisible.

1. Add an A record for<prefix>.matplotlib.org to the IPv4 address of the newdroplet.
2. Add a CNAME record forwebNN.matplotlib.org pointing to the given<prefix.matplotlib.org>.

You must setup Ansible as described above. Verify that the new droplet isvisible to Ansible by running:

ansible-inventory --graph

which should list the new droplet in thewebsite tag:

@all:  |--@website:  |  |--venus.matplotlib.org

Then execute the Ansible playbook on the servers by running:

ansible-playbook --user root matplotlib.org.yml

During the initial "Gathering Facts" task, you will be prompted to accept theserver's SSH fingerprint, which you should verify against the values foundearlier. If there are existing servers that you don't want to touch, then youcan use the--limit option. If you are using a non-default SSH key, you maywish to use the--private-key option.

You can verify that the server is running correctly by connecting tohttps://<prefix>.matplotlib.org in your browser.

Once everything is running, you should flip the DNS for the main site, changingthematplotlib.org CNAME to point to the new server'swebNN.matplotlib.orgfunctional name.