sgd is an R package for large scale estimation. It features manystochastic gradient methods, built-in models, visualization tools,automated hyperparameter tuning, model checking, interval estimation,and convergence diagnostics.
At the core of the package is the function
sgd(formula, data, model, model.control, sgd.control)It estimates parameters for a given data set and model usingstochastic gradient descent. The optional argumentsmodel.control andsgd.control specifyattributes about the model and stochastic gradient method. Takingadvantage of the bigmemory package, sgd also operates on data sets whichare too large to fit in RAM as well as streaming data.
Example of large-scale linear regression:
library(sgd)# DimensionsN <- 1e5 # number of data pointsd <- 1e2 # number of features# Generate data.X <- matrix(rnorm(N*d), ncol=d)theta <- rep(5, d+1)eps <- rnorm(N)y <- cbind(1, X) %*% theta + epsdat <- data.frame(y=y, x=X)sgd.theta <- sgd(y ~ ., data=dat, model="lm")Any loss function may be specified. For convenience the following arebuilt-in: * Linear models * Generalized linear models * Method ofmoments * Generalized method of moments * Cox proportional hazards model* M-estimation
The following stochastic gradient methods exist: * (Standard)stochastic gradient descent * Implicit stochastic gradient descent *Averaged stochastic gradient descent * Averaged implicit stochasticgradient descent * Classical momentum * Nesterov’s acceleratedgradient
Check out the vignette invignettes/ or examples indemo/. In R, the equivalent commands arevignette(package="sgd") anddemo(package="sgd").
To install the latest version from CRAN:
install.packages("sgd")To install the latest development version from Github:
# install.packages("devtools")devtools::install_github("airoldilab/sgd")sgd is written byDustin Tran,Junhyung Lyle Kim andPanos Toulis. Please feel free tocontribute by submitting any issues or requests—or by solving anycurrent issues!
We thank all other members of the Airoldi Lab (led by Prof. EdoAiroldi) for their feedback and contributions.
@article{tran2015stochastic, author = {Tran, Dustin and Toulis, Panos and Airoldi, Edoardo M}, title = {Stochastic gradient descent methods for estimation with large data sets}, journal = {arXiv preprint arXiv:1509.06459}, year = {2015}}