Admin tasks and tools
Toolforge components
Packaging
Runbooks and emergency guides
Other
[edit]
Toolforge has an internalapt repository managed withaptly. The repository exists on thetools-services nodes.
Repositories are declared in puppet, but packages should be added to the aptly repository by hand. We usually have one repository per operating system and project, i.e:
Quick example of packages being stored here are:
(among others)
The repository data, located at/srv/packages is stored in a mounted cinder volume.
Information on how the setup is deployed, and the different components.
Usually a VM with a cinder volume to store repository data.
There is an horizon web proxy calleddeb-tools.wmcloud.org that should point to TCP/80 on the server. This allows to build docker images using toolforge internal packages.
Other than that, servers do not have any special DNS or addressing. They do not have floating IPs.
Worth noting that these servers in thetools cloudvps project may offer services for thetoolsbeta project as well.
The main role in use isrole::wmcs::toolforge::services.
Information on maintenance and administration of this setup.
Is managed as a standardaptly repo.
Some interesting bits to check if you want to know the status/health of the server.
sudo aptly repo list andsudo aptly repo show --with-packages=true stretch-toolsdf -h /We do not have a specific failover mechanism rather than building a new VM and re-attach the cinder volume.
Care should be taken to do not loss aptly repo data, since generating it from scratch can take some time.
This was heavily remodeled when migrating the grid to SGE and to Stretch. Previous to the migration, the services nodes used to storeBigbrother (deprecated), andwebservicemonitor (moved to cron servers).
Again, when migrating from Stretch to Buster, the 2 VM approach was dropped in favor of storing the data in a cinder volume, seehttps://phabricator.wikimedia.org/T278354.