- Notifications
You must be signed in to change notification settings - Fork7
Go implementation of different backoff strategies useful for retrying operations and heartbeating.
License
ssgreg/repeat
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Go implementation of different backoff strategies useful for retrying operations and heartbeating.
Let's imagine that we need to do a REST call on remote server but it could fail with a bunch of different issues. We can repeat failed operation using exponential backoff policies.
Exponential backoff is an algorithm that uses feedback to multiplicatively decrease the rate of some process, in order to gradually find an acceptable rate.
The example below tries to repeat operation 10 times using a full jitter backoff.See algorithm details here.
// An example operation that do some useful stuff.// It fails five first times.varlast time.Timeop:=func(cint)error {printInfo(c,&last)ifc<5 {returnrepeat.HintTemporary(errors.New("can't connect to a server")) }returnnil }// Repeat op on any error, with 10 retries, with a backoff.err:=repeat.Repeat(// Our op with additional call counter.repeat.FnWithCounter(op),// Force the repetition to stop in case the previous operation// returns nil.repeat.StopOnSuccess(),// 10 retries max.repeat.LimitMaxTries(10),// Specify a delay that uses a backoff.repeat.WithDelay(repeat.FullJitterBackoff(500*time.Millisecond).Set(), ), )
The example of output:
Attempt #0, Delay 0sAttempt #1, Delay 373.617912msAttempt #2, Delay 668.004225msAttempt #3, Delay 1.220076558sAttempt #4, Delay 2.716156336sAttempt #5, Delay 6.458431017sRepetition process is finished with: <nil>The example below is almost the same as the previous one. It adds one important feature - possibility to cancel operation repetition using context's timeout.
// A context with cancel.// Repetition will be cancelled in 3 seconds.ctx,cancelFunc:=context.WithCancel(context.Background())gofunc() {time.Sleep(3*time.Second)cancelFunc() }()// Repeat op on any error, with 10 retries, with a backoff.err:= repeat.Repeat(...// Specify a delay that uses a backoff.repeat.WithDelay(repeat.FullJitterBackoff(500*time.Millisecond).Set(),repeat.SetContext(ctx), ),... )
The example of output:
Attempt #0, Delay 0sAttempt #1, Delay 358.728046msAttempt #2, Delay 845.361787msAttempt #3, Delay 61.527485msRepetition process is finished with: context canceledLet's imagine we need to periodically report execution progress to remote server. The example below repeats the operation each second until it will be cancelled using passed context.
// An example operation that do heartbeat.varlast time.Timeop:=func(cint)error {printInfo(c,&last)returnnil }// A context with cancel.// Repetition will be cancelled in 7 seconds.ctx,cancelFunc:=context.WithCancel(context.Background())gofunc() {time.Sleep(7*time.Second)cancelFunc() }()err:=repeat.Repeat(// Heartbeating op.repeat.FnWithCounter(op),// Delay with fixed backoff and context.repeat.WithDelay(repeat.FixedBackoff(time.Second).Set(),repeat.SetContext(ctx), ), )
The example of output:
Attempt #0, Delay 0sAttempt #1, Delay 1.001129426sAttempt #2, Delay 1.000155727sAttempt #3, Delay 1.001131014sAttempt #4, Delay 1.000500428sAttempt #5, Delay 1.0008985sAttempt #6, Delay 1.000417057sRepetition process is finished with: context canceledThe example below is almost the same as the previous one but it will be cancelled using special error timeout. This timeout resets each time the operations return nil.
// An example operation that do heartbeat.// It fails 5 times after 3 successfull tries.varlast time.Timeop:=func(cint)error {printInfo(c,&last)ifc>3&&c<8 {returnrepeat.HintTemporary(errors.New("can't connect to a server")) }returnnil }err:=repeat.Repeat(// Heartbeating op.repeat.FnWithCounter(op),// Delay with fixed backoff and error timeout.repeat.WithDelay(repeat.FixedBackoff(time.Second).Set(),repeat.SetErrorsTimeout(3*time.Second), ), )
The example of output:
Attempt #0, Delay 0sAttempt #1, Delay 1.001634616sAttempt #2, Delay 1.004912408sAttempt #3, Delay 1.001021358sAttempt #4, Delay 1.001249459sAttempt #5, Delay 1.004320833sRepetition process is finished with: can't connect to a serverAbout
Go implementation of different backoff strategies useful for retrying operations and heartbeating.