Already have an account? Sign in

Advanced Replication Scheduling Using Cron Expressions

The Advanced Scheduler feature allows you to specify granular start times for data extraction. Using cron expressions, you can specify the exact times, days of the week, or even days of the month data extraction should begin.

Note: All replication scheduling methods (Replication Frequency, Anchor Scheduling, and Advanced Scheduling) define when data extractions begin. They do not control how long a replication job runs or when data is loaded into a destination.

In this guide, we’ll cover:

• The uses for Advanced Scheduling

• How to access Advanced Scheduling

• The basics of cron scheduling

• Examples of replication schedules using cron expressions

• Limitations of cron scheduling in Stitch

• How to create an Advanced Schedule for an integration

• Additional cron resources

Uses for Advanced Scheduling

Using Advanced Scheduling, you can:

1. Run reports on specified days. Using Advanced Scheduling, you can create a replication schedule that ensures reports for a weekly 9:00AM meeting are up-to-date.

2. Whitelist hours for starting data extractions. For example: Scheduling replication during off-peak hours will reduce load on your production database.

   Note: An extraction may run over into “blackout” hours as the Advanced Scheduler only controls the times jobsstart.See the Limitations section for more info.

3. Reduce your row usage. Only scheduling data extractions when you need them can not only reduce load on data sources and your destination, it can reduce your overall row usage in Stitch.

4. Reduce the re-replication of data. Because replication scheduling applies to all selected tables in an integration, tables usingFull Table Replication will replicate in full each time a replication job runs. Reducing the number of replication jobs overall will decrease the number of times the same record is replicated.

Access to Advanced Scheduling

Advanced Scheduling is available during the Free Trial or on an Advanced or Premium plan. ContactStitch Sales for more info about these plans.

Plan downgrades

When the Free Trial ends or if you downgrade from an Advanced or Premium plan, you’ll lose access to the Advanced Scheduling feature.

In the event that you downgrade, Stitch will automatically pause any integrations using Advanced Scheduling and reset theirReplication Frequencies to their defaults. You will need to manually un-pause the integrations to continue replication.

Advanced Scheduling basics

Stitch’s Advanced Scheduler feature uses cron scheduling to create replication schedules. In this section, we’ll:

• Introduce you to cron and cron expressions

• Describe cron expression syntax

• Explain how to use special characters

Introduction to cron

So, what’s cron?Cron is a time-based scheduler used in Unix-like operating systems such as Mac OS, Linux, etc. Tasks, or jobs, created through cron are calledcron jobs. Stitch uses theQuartz standard for cron scheduling.

To create a cron job - in this case, an integration’s replication schedule - acron expression is used. A cron expression describes the details of the schedule, and when combined, translates to the schedule Stitch will use to extract data from the integration.

Using a cron expression, you can create replication schedules such as“At 12:00AM every day” or“At 7:00PM every Monday, Wednesday, and Friday”. Stitch will then start replication jobs at these times.

Cron expression syntax

A cron expression in Stitch is made up of six fields that describe the elements of the schedule, separated by spaces.

Fields in the expression must be in the following order, and an expression must have all six fields to be considered valid:

[seconds] [minute] [hour] [dayofmonth] [month] [dayofweek]

Note: The expression uses UTC time. The time zone cannot be customized, but the corresonding local time is displayed in theSample schedule (local time) when creating the expression.

Fields can contain any of the allowed values, along with various combinations of the allowed special characters for that field.Note: Entering anything other than a field’s allowed values or special characters will result in an error when you try to save the integration’s settings. Refer to theTroubleshooting section for help resolving these errors.

Allowed special characters

Each field in a cron expression has its own list of allowed special characters. These characters allow you to select all values, a list or range of values, specify increments, and more. Fields may also contain both a range and a list. Refer to theExample replication schedules section for examples.

Example replication schedules

Based on scheduling feedback we’ve collected, we’ve put together a list of some of the most commonly requested replication schedules. You can use these examples to define an integration’s replication schedule in Stitch, or create your own. Keep in mind that there are somelimitations.

• Run at midnight UTC every day

• Run at midnight UTC every day from Monday through Friday

• Run once a week at 11:00PM UTC on Sunday

• Run at noon and midnight UTC every day

• Run Tuesday through Thursday every week at 3:00AM UTC

• Run at midnight UTC every Monday, Wednesday, and Friday

• Run at minute 30 and every minute between :05 and :10, every hour, every day in January

Run at midnight UTC every day

Run at midnight UTC every day from Monday through Friday

Run once a week at 11:00PM UTC on Sunday

Run at noon and midnight UTC every day

Run Tuesday through Thursday every week at 3:00AM UTC

Run at midnight UTC every Monday, Wednesday, and Friday

Run at minute 30 and every minute between :05 and :10, every hour, every day in January

Limitations of cron scheduling

While using cron expressions will give you the most control over your integrations’ replication schedules, there are some limitations to keep in mind:

1. You can specify a Day of week OR Day of month value, but not both. Quartz, the cron implementation used by Stitch,doesn’t currently support specifying values for both fields in an expression. One of these fields must contain the? character for the expression to be considered valid.

2. Advanced Scheduling can only be used to whitelist extraction start times. This means that a job could start during a whitelisted time period but continue running beyond that window, depending on the duration of the extraction.

   For example: An integration has a schedule that tells it to run every 20 minutes between the hours of noon and 2:00PM, starting at noon.

   On average, extractions for this integration take between 2-5 minutes. However, the extraction that starts at 1:40PM takes longer than average, causing the job to continue running even after the 2:00PM mark. As a result, the job scheduled for 2:00PM will be skipped and resume at the next scheduled interval.

Create an Advanced Schedule for an integration

You can create an Advanced Schedule in an integration’sSettings page.

1. To access this page, click the integration from the Stitch Dashboard and then click theSettings tab.

2. In theReplication Frequency section, check theAdvanced box located under theAnchor time menu. This will open theAdvanced Scheduler.

   Note: This feature is only availableduring the Free Trial or on an Advanced or Premium plan.

3. Enter the values you want into each of the fields. Stitch will validate the schedule after each change. If the schedule is valid, a sample schedule will display under the fields.

4. When finished, click theSave Integration button.

Additional cron resources

• Quartz Scheduler - The official documentation for the Quartz Scheduler.

• Quartz Cron Trigger Tutorial - A cron tutorial from the makers of Quartz, the cron implementation used in Stitch.

• Quartz Cron Expression Generator - A simple, web-based editor for Quartz cron expressions.

Did this article help? If you have questions or feedback, feel free tosubmit a pull request with your suggestions,open an issue on GitHub, orreach out to us.