Overview

The sequential probability ratio test (SPRT) was proposed byWald (1947),Wald andWolfowitz (1948). as a way to do continuous sampling to establishor raise concerns about product quality. There is a wide literature onthis topic which we do not attempt to summarize here. In clinicaltrials, the SPRT for a single arm binary endpoint can be useful to raiseor alleviate concerns about a short-term endpoint such as occurrence ofan important safety endpoint or, for efficacy, a response rate. ThefunctionbinomialSPRT() implements a single arm version ofthe SPRT for a binary outcome. While comparative SPRT tests are alsoavailable for comparing multiple arms, we do not cover those here.

You may think that having an sequential design for a trial obligatesyou to do an evaluation after every observation. An alternative view isthat you can analyze whenever you want and not worry about whether TypeI error is controlled.

Response rate example

Consider a single arm where there is historical data suggesting the apositive response to treatment occurs in no more than 10% of patientswith currently available treatments. Assume that there is interest inhaving the trial be well-powered to detect a response rate of 35% in anew treatment. The SPRT is defined as a continuous testing procedurewithout a maximum sample size. Practically speaking, this is implementedwith a minimum and maximum sample size. For our example we assume aminimum sample size of 10 and a maximum sample size of 25. We willinitially set a one-sided Type I error of\(\alpha=0.08\) and power of 80% (\(1-\beta = 0.2\)):

library(gsDesign)

b<-binomialSPRT(p0 = .1,p1 = .35,alpha = .08,beta = .2,minn =10,maxn =25)plot(b)

The above plot tests first after 10 patients. If 4/10 have responded,you can reject the null hypothesis of a 10% response rate. If 0 or 1 of10 have responded, you can conclude that the targeted 35% response rateis not realistic. Note that the number of responses required to cross abound is a step function due to the discrete nature of the problem. Wesee at the maximum sample size of 25:

• If 4 or fewer patients have responded we will accept the nullhypothesis of a 10% event rate.
• If 7 or more patients have responded we will reject the nullhypothesis in favor of a larger response rate.
• If 5 or 6 patients respond we have an indeterminate result where wecan reject neither the null hypothesis of a 10% response rate nor thealternate hypothesis of a 35% response rate.

Summarizing design properties

Functions are available to summarize design properties. For example,we can make a power plot:

library(ggplot2)

p<-plot(b,plottype =2)p+scale_y_continuous(breaks =seq(0,90,10))

Probability of three possible outcomes are summarized by theunderlying response rate:

• Solid line: indeterminate outcome (no bound crossed from 10 to 25patients). Note that probabilities are small, even for response rateshalf-way between the null and alternative hypotheses.
• Short-dashed line: reject H0. While we targeted a 10% Type I errorrate, we can see here that since we have truncated the design to analyzeonly after 10 to 25 patient that the Type I error is less 5%; this isbelow the targeted 1-sided\(\alpha=0.08.\)
• Long-dashed line: Reject H1. We can see that we have > 90% chanceof rejecting a 35% response rate if the true response rate is 10%.

We now provide a summary table for operating characteristics. Theuser can ignore reviewing the code, but may copy if wishing to produce asimilar table.

library(dplyr)library(tidyr)

# Compute boundary crossing probabilities for selected response ratesb_power<-gsBinomialExact(k =length(b$n.I),theta =seq(.1, .45, .05),n.I = b$n.I,a = b$lower$bound,b = b$upper$bound)

b_power%>%as_table()%>%as_gt()

Safety monitoring example

Next we consider a safety monitoring example. Suppose a new treatmenthas a mechanism of action that has potential for an elevated rate of aspecific adverse experience (AE); e.g., serious rash. Suppose that thisalready occurs with some low frequency in the population proposed for astudy at a rate of about 4% and that a 10% rate would be consideredunacceptable. While a comparison of the two arms could be consideredwith an SPRT, we demonstrate here a monitoring bound for theexperimental arm only. We assume the proposed sample size for the studyis 75 per arm and that we will not stop the trial for serious rashbefore 4 patients have been studied in the experimental group.

safety_design<-binomialSPRT(p0 = .04,p1 = .1,alpha = .04,beta = .2,minn =4,maxn =75)plot(safety_design)

We see above that if we have no serious rashes in the first 25experimental group patients or 1 in the first 40 that we reject the 10%rate of concern. On the other hand, if the first 4 of the first 4 to 14patients have serious rashes or 5 of the first 15 to 29 patients haveserious rashes we can reject the hypothesis that there is no elevationover the presumed 4% population rate.

The design operating characteristics are now summarized both in aplot and a table.

plot(safety_design,plottype =2)

We see that there can be a fairly high possibility of anindeterminate outcome at the end of the trial. If we include moreextreme values in the summary table, we can see that the indeterminateprobability lowers outside the range of 0.04 to 0.10. The average samplesize also goes down; however, this does not account for enrollment thatmay occur but not be included in the analysis due to inadequatefollow-up to assess the endpoint. Note that sequential patients shouldbe analyzed, but if you cross a high rate bound with some incompletedata included assafe that you can still declare that youcrossed the high rate bound as the number of adverse experience casescannot go down when the missing cases filled in.

safety_power<-gsBinomialExact(k =length(safety_design$n.I),theta =seq(.02, .16, .02),n.I = safety_design$n.I,a = safety_design$lower$bound,b = safety_design$upper$bound)safety_power%>%as_table()%>%as_gt(theta_label = gt::html("Underlying<br>AE rate"),prob_decimals =3,bound_label =c("low rate","high rate")  )

Summary

We have shown how to:

• derive bounds for a truncated sequential probability ratio test(SPRT),
• plot the bounds for the design,
• plot the operating characteristics for the design, and
• print a table summarizing the design.

We noted that the observed power and Type I error is lower than thespecified Type I error and power can be higher. This means the userneeds to consider inputs on an iterative basis to control operatingcharacteristics of the truncated SPRT.

This design can be use for both responses and failures. For instance,it can provide a method of early monitoring for excessive risk of keyadverse events in a study.

References