Repository files navigation

A driver forQueues. UsesFluent to store job metadata in an SQL database.

This package makes use of theSKIP LOCKED feature supported by some of the major database engines (most notablyPostgresSQL andMySQL) when available to make a best-effort guarantee that a job won't be picked up by multiple workers.

This package should be compatible with any SQL database supported by the various Fluent drivers. It is specifically known to work with:

• PostgreSQL 11.0+
• MySQL 5.7+
• MariaDB 10.5+
• SQLite

AddQueuesFluentDriver to yourPackage.swift as a dependency:

dependencies:[.package(url:"https://github.com/vapor-community/vapor-queues-fluent-driver.git", from:"3.0.0"),...]

Then addQueuesFluentDriver to the target you want to use it in:

targets:[.target(name:"MyFancyTarget", dependencies:[.product(name:"QueuesFluentDriver",package:"vapor-queues-fluent-driver"),])]

Or use SwiftPM's dependency management commands:

swift package add-dependency 'https://github.com/vapor-community/vapor-queues-fluent-driver.git' --from '3.0.0'swift package add-target-dependency --package vapor-queues-fluent-driver QueuesFluentDriver MyFancyTarget

This package includes a migration to create the database table which holds job metadata. Add it to your Fluent configuration as you would any other migration:

app.migrations.add(JobModelMigration())

If you were previously a user of the 1.x or 2.x releases of this driver and have an existing job metadata table in the old data format, you can useJobModelOldFormatMigration instead to transparently upgrade the old table to the new format:

app.migrations.add(JobModelOldFormatMigration())

Finally, load theQueuesFluentDriver driver:

app.queues.use(.fluent())

The.fluent() driver method accepts several configuration options.

The driver may be configured with aDatabaseID other than the default to use for queue operations. The default ofnil corresponds to Fluent's default database. The database ID must be registered withapp.databases.use(...)before configuring the Queues driver.

Example:

extensionDatabaseID{staticvarqueues:Self{.init(string:"my_queues_db")}}func configure(_ app:Application)asyncthrows{    app.databases.use(.postgres(configuration:...), as:.queues)    app.queues.use(.fluent(.queues))}

By default, once a job has finished, it is removed entirely from the jobs table in the database. Setting thepreserveCompletedJobs parameter totrue configures the driver to leave finished jobs in the jobs table, with a state ofcompleted.

Example:

func configure(_ app:Application)asyncthrows{    app.queues.use(.fluent(preserveCompletedJobs:true))}

By default, the jobs table is created in the default space (e.g. the current schema - usuallypublic - in PostgreSQL, or the current database in MySQL and SQLite) and has the name_jobs_meta. The table name and space may be configured, using thejobsTableName andjobsTableSpace parameters respectively. IfJobModelMigration orJobModelOldFormatMigration are in use (as is recommended), the same name and space must be passed to both its initializer and the driver for the migration to work correctly.

Example:

func configure(_ app:Application)asyncthrows{    app.migrations.add(JobModelMigration(jobsTableName:"_my_jobs", jobsTableSpace:"not_public"))    // OR    app.migrations.add(JobModelOldFormatMigration(jobsTableName:"_my_jobs", jobsTableSpace:"not_public"))        app.queues.use(.fluent(jobsTableName:"_my_jobs", jobsTableSpace:"not_public"))}

By default, the Vapor Queues system starts 2 workers per available CPU core, with each worker would polling the database once per second. On a 4-core system, this would results in 8 workers querying the database every second. Most configurations do not need this many workers. Additionally, when using SQLite as the underlying database it is generally inadvisable to run more than one worker at a time, as SQLite does not have the necessary support for locking to make this safe.

The polling interval can be changed using therefreshInterval configuration setting:

app.queues.configuration.refreshInterval=.seconds(5)

Likewise, the number of workers to start can be changed via theworkerCount setting:

app.queues.configuration.workerCount=1