Movatterモバイル変換

Pipe operator

Description

Seemagrittr::%>% for details.

Usage

lhs %>% rhs

Create an accelerator

Description

Create an accelerator

Usage

accelerator(  device_placement = TRUE,  cpu = FALSE,  cuda_index = torch::cuda_current_device())

Arguments

Creates a dataloader from its input

Description

as_dataloader is used internally by luz to convert inputdata andvalid_data as passed tofit.luz_module_generator() to atorch::dataloader

Usage

as_dataloader(x, ...)## S3 method for class 'dataset'as_dataloader(x, ..., batch_size = 32)## S3 method for class 'iterable_dataset'as_dataloader(x, ..., batch_size = 32)## S3 method for class 'list'as_dataloader(x, ...)## S3 method for class 'dataloader'as_dataloader(x, ...)## S3 method for class 'matrix'as_dataloader(x, ...)## S3 method for class 'numeric'as_dataloader(x, ...)## S3 method for class 'array'as_dataloader(x, ...)## S3 method for class 'torch_tensor'as_dataloader(x, ...)

Arguments

Details

as_dataloader methods should have sensible defaults for batch_size,parallel workers, etc.

It allows users to quickly experiment withfit.luz_module_generator() by not requiringto create atorch::dataset and atorch::dataloader in simpleexperiments.

Methods (by class)

• as_dataloader(dataset): Converts atorch::dataset() to atorch::dataloader().

• as_dataloader(iterable_dataset): Converts atorch::iterable_dataset() into atorch::dataloader()

• as_dataloader(list): Converts a list of tensors or arrays with the samesize in the first dimension to atorch::dataloader()

• as_dataloader(dataloader): Returns the same dataloader

• as_dataloader(matrix): Converts the matrix to a dataloader

• as_dataloader(numeric): Converts the numeric vector to a dataloader

• as_dataloader(array): Converts the array to a dataloader

• as_dataloader(torch_tensor): Converts the tensor to a dataloader

Overriding

You can implement your ownas_dataloader S3 method if you want your datastructure to be automatically supported by luz'sfit.luz_module_generator().The method must satisfy the following conditions:

• The method should return atorch::dataloader().

• The only required argument isx. You have good default for all otherarguments.

It's better to avoid implementingas_dataloader methods for common S3 classeslikedata.frames. In this case, its better to assign a different class tothe inputs and implementas_dataloader for it.

Context object

Description

Context object storing information about the model training context.See alsoctx.

Public fields

Active bindings

Methods

Public methods

• context$new()

• context$log()

• context$log_metric()

• context$get_log()

• context$get_metrics()

• context$get_metric()

• context$get_formatted_metrics()

• context$get_metrics_df()

• context$set_verbose()

• context$clean()

• context$call_callbacks()

• context$state_dict()

• context$unsafe_set_records()

• context$clone()

Methodnew()

Initializes the context object with minimal necessary information.

Usage

context$new(verbose, accelerator, callbacks, training)

Arguments

Methodlog()

Allows logging arbitrary information in thectx.

Usage

context$log(what, set, value, index = NULL, append = TRUE)

Arguments

Methodlog_metric()

Log a metric by its name and value.Metric values are indexed by epoch.

Usage

context$log_metric(name, value)

Arguments

Methodget_log()

Get a specific value from the log.

Usage

context$get_log(what, set, index = NULL)

Arguments

Methodget_metrics()

Get all metric given an epoch and set.

Usage

context$get_metrics(set, epoch = NULL)

Arguments

Methodget_metric()

Get the value of a metric given its name, epoch and set.

Usage

context$get_metric(name, set, epoch = NULL)

Arguments

Methodget_formatted_metrics()

Get formatted metrics values

Usage

context$get_formatted_metrics(set, epoch = NULL)

Arguments

Methodget_metrics_df()

Get a data.frame containing all metrics.

Usage

context$get_metrics_df()

Methodset_verbose()

Allows setting theverbose attribute.

Usage

context$set_verbose(verbose = NULL)

Arguments

Methodclean()

Removes unnecessary information from the context object.

Usage

context$clean()

Methodcall_callbacks()

Call the selected callbacks. Wherename is the callback types to call, eg'on_epoch_begin'.

Usage

context$call_callbacks(name)

Arguments

Methodstate_dict()

Returns a list containing minimal information from the context. Used tocreate the returned values.

Usage

context$state_dict()

Methodunsafe_set_records()

Are you sure you know what you are doing?

Usage

context$unsafe_set_records(records)

Arguments

Methodclone()

The objects of this class are cloneable with this method.

Usage

context$clone(deep = FALSE)

Arguments

Context object

Description

Context objects used in luz to share information between model methods,metrics and callbacks.

Details

Thectx object is used in luz to share information between thetraining loop and callbacks, model methods, and metrics. The table belowdescribes information available in thectx by default. Other callbackscould potentially modify these attributes or add new ones.

Context attributes

See Also

Context object:context

Evaluates a fitted model on a dataset

Description

Evaluates a fitted model on a dataset

Usage

evaluate(  object,  data,  ...,  metrics = NULL,  callbacks = list(),  accelerator = NULL,  verbose = NULL,  dataloader_options = NULL)

Arguments

Details

Once a model has been trained you might want to evaluate its performanceon a different dataset. For that reason, luz provides the?evaluatefunction that takes a fitted model and a dataset and computes themetrics attached to the model.

Evaluate returns aluz_module_evaluation object that you can query formetrics using theget_metrics function or simplyprint to see theresults.

For example:

evaluation <- fitted %>% evaluate(data = valid_dl)metrics <- get_metrics(evaluation)print(evaluation)

See Also

Other training:fit.luz_module_generator(),predict.luz_module_fitted(),setup()

Fit ann_module

Description

Fit ann_module

Usage

## S3 method for class 'luz_module_generator'fit(  object,  data,  epochs = 10,  callbacks = NULL,  valid_data = NULL,  accelerator = NULL,  verbose = NULL,  ...,  dataloader_options = NULL)

Arguments

Value

A fitted object that can be saved withluz_save() and can beprinted withprint() and plotted withplot().

See Also

predict.luz_module_fitted() for how to create predictions.setup() to find out how to create modules that can be trained withfit.

Other training:evaluate(),predict.luz_module_fitted(),setup()

Get metrics from the object

Description

Get metrics from the object

Usage

get_metrics(object, ...)## S3 method for class 'luz_module_fitted'get_metrics(object, ...)

Arguments

Value

A data.frame containing the metric values.

Methods (by class)

• get_metrics(luz_module_fitted): Extract metrics from a luz fitted model.

Learning Rate Finder

Description

Learning Rate Finder

Usage

lr_finder(  object,  data,  steps = 100,  start_lr = 1e-07,  end_lr = 0.1,  log_spaced_intervals = TRUE,  ...,  verbose = NULL)

Arguments

Value

A dataframe with two columns: learning rate and loss

Examples

if (torch::torch_is_installed()) {library(torch)ds <- torch::tensor_dataset(x = torch_randn(100, 10), y = torch_randn(100, 1))dl <- torch::dataloader(ds, batch_size = 32)model <- torch::nn_linearmodel <- model %>% setup(  loss = torch::nn_mse_loss(),  optimizer = torch::optim_adam) %>%  set_hparams(in_features = 10, out_features = 1)records <- lr_finder(model, dl, verbose = FALSE)plot(records)}

Create a new callback

Description

Create a new callback

Usage

luz_callback(  name = NULL,  ...,  private = NULL,  active = NULL,  parent_env = parent.frame(),  inherit = NULL)

Arguments

Details

Let’s implement a callback that prints ‘Iterationn’ (wheren is theiteration number) for every batch in the training set and ‘Done’ when anepoch is finished. For that task we use theluz_callback function:

print_callback <- luz_callback(  name = "print_callback",  initialize = function(message) {    self$message <- message  },  on_train_batch_end = function() {    cat("Iteration ", ctx$iter, "\n")  },  on_epoch_end = function() {    cat(self$message, "\n")  })

luz_callback() takes named functions as... arguments, where thename indicates the moment at which the callback should be called. Forinstanceon_train_batch_end() is called for every batch at the end ofthe training procedure, andon_epoch_end() is called at the end ofevery epoch.

The returned value ofluz_callback() is a function that initializes aninstance of the callback. Callbacks can have initialization parameters,like the name of a file where you want to log the results. In that case,you can pass aninitialize method when creating the callbackdefinition, and save these parameters to theself object. In the aboveexample, the callback has amessage parameter that is printed at theend of each epoch.

Once a callback is defined it can be passed to thefit function viathecallbacks parameter:

fitted <- net %>%  setup(...) %>%  fit(..., callbacks = list(    print_callback(message = "Done!")  ))

Callbacks can be called in many different positions of the trainingloop, including combinations of them. Here’s an overview of possiblecallbackbreakpoints:

Start Fit   - on_fit_begin  Start Epoch Loop     - on_epoch_begin    Start Train       - on_train_begin      Start Batch Loop         - on_train_batch_begin          Start Default Training Step            - on_train_batch_after_pred            - on_train_batch_after_loss            - on_train_batch_before_backward            - on_train_batch_before_step            - on_train_batch_after_step          End Default Training Step:         - on_train_batch_end      End Batch Loop       - on_train_end    End Train    Start Valid       - on_valid_begin      Start Batch Loop         - on_valid_batch_begin          Start Default Validation Step            - on_valid_batch_after_pred            - on_valid_batch_after_loss          End Default Validation Step         - on_valid_batch_end      End Batch Loop       - on_valid_end    End Valid      - on_epoch_end  End Epoch Loop   - on_fit_endEnd Fit

Every step marked with⁠on_*⁠ is a point in the training procedure thatis available for callbacks to be called.

The other important part of callbacks is thectx (context) object. Seehelp("ctx") for details.

By default, callbacks are called in the same order as they were passedtofit (orpredict orevaluate), but you can provide aweightattribute that will control the order in which it will be called. Forexample, if one callback hasweight = 10 and another hasweight = 1,then the first one is called after the second one. Callbacks that don’tspecify aweight attribute are consideredweight = 0. A few built-incallbacks in luz already provide a weight value. For example, the?luz_callback_early_stopping has a weight ofInf, since in generalwe want to run it as the last thing in the loop.

Value

Aluz_callback that can be passed tofit.luz_module_generator().

Prediction callbacks

You can also use callbacks when usingpredict(). In this case the supportedcallback methods are detailed below:

Start predict - on_predict_begin Start prediction loop  - on_predict_batch_begin  - on_predict_batch_end End prediction loop - on_predict_endEnd predict

Evaluate callbacks

Callbacks can also be used withevaluate(), in this case, the callbacks thatare used are equivalent to those of the validation loop when usingfit():

Start Valid - on_valid_begin Start Batch Loop  - on_valid_batch_begin  Start Default Validation Step   - on_valid_batch_after_pred   - on_valid_batch_after_loss  End Default Validation Step  - on_valid_batch_end End Batch Loop - on_valid_endEnd Valid

See Also

Other luz_callbacks:luz_callback_auto_resume(),luz_callback_csv_logger(),luz_callback_early_stopping(),luz_callback_interrupt(),luz_callback_keep_best_model(),luz_callback_lr_scheduler(),luz_callback_metrics(),luz_callback_mixed_precision(),luz_callback_mixup(),luz_callback_model_checkpoint(),luz_callback_profile(),luz_callback_progress(),luz_callback_resume_from_checkpoint(),luz_callback_train_valid()

Examples

print_callback <- luz_callback( name = "print_callback", on_train_batch_end = function() {   cat("Iteration ", ctx$iter, "\n") }, on_epoch_end = function() {   cat("Done!\n") })

Resume training callback

Description

This callback allows you to resume training a model.

Usage

luz_callback_auto_resume(path = "./state.pt")

Arguments

Details

When using it, model weights, optimizer state are serialized at the end ofeach epoch. If something fails during training simply re-running the samescript will restart the model training from the epoch right after the lastepoch that was serialized.

Customizing serialization

By default model, optimizer state and records are serialized. Callbacks canbe used to customize serialization by implementing thestate_dict() andload_state_dict() methods.If those methods are implemented, thenstate_dict() is called at the end ofeach epoch andload_state_dict() is called when the model is resumed.

Note

In general you will want to add this callback as the last in the callbackslist, this way, the serialized state is likely to contain all possible changesthat other callbacks could have made at'on_epoch_end'. The defaultweightattribute of this callback isInf.

Read the checkpointing article in the pkgdown website for moreinformation.

See Also

Other luz_callbacks:luz_callback(),luz_callback_csv_logger(),luz_callback_early_stopping(),luz_callback_interrupt(),luz_callback_keep_best_model(),luz_callback_lr_scheduler(),luz_callback_metrics(),luz_callback_mixed_precision(),luz_callback_mixup(),luz_callback_model_checkpoint(),luz_callback_profile(),luz_callback_progress(),luz_callback_resume_from_checkpoint(),luz_callback_train_valid()

Examples

if (torch::torch_is_installed()) {library(torch)library(luz)x <- torch_randn(1000, 10)y <- torch_randn(1000, 1)model <- nn_linear %>%  setup(optimizer = optim_sgd, loss = nnf_mse_loss) %>%  set_hparams(in_features = 10, out_features = 1) %>%  set_opt_hparams(lr = 0.01)# simulate a failure in the middle of epoch 5 happening only once.callback_stop <- luz_callback(  "interrupt",  failed = FALSE,  on_epoch_end = function() {    if (ctx$epoch == 5 && !self$failed) {      self$failed <- TRUE      stop("Error on epoch 5")    }  })path <- tempfile()autoresume <- luz_callback_auto_resume(path = path)interrupt <- callback_stop()# try once and the model failstry({  results <- model %>% fit(    list(x, y),    callbacks = list(autoresume, interrupt),    verbose = FALSE  )})# model resumes and completesresults <- model %>% fit(  list(x, y),  callbacks = list(autoresume, interrupt),  verbose = FALSE)get_metrics(results)}

CSV logger callback

Description

Logs metrics obtained during training a file on disk.The file will have 1 line for each epoch/validation.

Usage

luz_callback_csv_logger(path)

Arguments

See Also

Other luz_callbacks:luz_callback(),luz_callback_auto_resume(),luz_callback_early_stopping(),luz_callback_interrupt(),luz_callback_keep_best_model(),luz_callback_lr_scheduler(),luz_callback_metrics(),luz_callback_mixed_precision(),luz_callback_mixup(),luz_callback_model_checkpoint(),luz_callback_profile(),luz_callback_progress(),luz_callback_resume_from_checkpoint(),luz_callback_train_valid()

Early stopping callback

Description

Stops training when a monitored metric stops improving

Usage

luz_callback_early_stopping(  monitor = "valid_loss",  min_delta = 0,  patience = 0,  mode = "min",  baseline = NULL)

Arguments

Value

Aluz_callback that does early stopping.

Note

This callback adds aon_early_stopping callback that can be used tocall callbacks as soon as the model stops training.

Ifverbose=TRUE infit.luz_module_generator() a message is printed whenearly stopping.

See Also

Other luz_callbacks:luz_callback(),luz_callback_auto_resume(),luz_callback_csv_logger(),luz_callback_interrupt(),luz_callback_keep_best_model(),luz_callback_lr_scheduler(),luz_callback_metrics(),luz_callback_mixed_precision(),luz_callback_mixup(),luz_callback_model_checkpoint(),luz_callback_profile(),luz_callback_progress(),luz_callback_resume_from_checkpoint(),luz_callback_train_valid()

Examples

cb <- luz_callback_early_stopping()

Gradient clipping callback

Description

By adding the GradientClip callback, the gradientnorm_type (default:2) normis clipped to at mostmax_norm (default:1) usingtorch::nn_utils_clip_grad_norm_(),which can avoid loss divergence.

Usage

luz_callback_gradient_clip(max_norm = 1, norm_type = 2)

Arguments

References

See FastAIdocumentationfor the GradientClip callback.

Interrupt callback

Description

Adds a handler that allows interrupting the training loop usingctrl + C.Also registers aon_interrupt breakpoint so users can register callbacks tobe run on training loop interruption.

Usage

luz_callback_interrupt()

Value

Aluz_callback

Note

In general you don't need to use these callback by yourself because it's alwaysincluded by default infit.luz_module_generator().

See Also

Other luz_callbacks:luz_callback(),luz_callback_auto_resume(),luz_callback_csv_logger(),luz_callback_early_stopping(),luz_callback_keep_best_model(),luz_callback_lr_scheduler(),luz_callback_metrics(),luz_callback_mixed_precision(),luz_callback_mixup(),luz_callback_model_checkpoint(),luz_callback_profile(),luz_callback_progress(),luz_callback_resume_from_checkpoint(),luz_callback_train_valid()

Examples

interrupt_callback <- luz_callback_interrupt()

Keep the best model

Description

Each epoch, if there's improvement in the monitored metric we serialize themodel weights to a temp file. When training is done, we reload weights fromthe best model.

Usage

luz_callback_keep_best_model(  monitor = "valid_loss",  mode = "min",  min_delta = 0)

Arguments

See Also

Other luz_callbacks:luz_callback(),luz_callback_auto_resume(),luz_callback_csv_logger(),luz_callback_early_stopping(),luz_callback_interrupt(),luz_callback_lr_scheduler(),luz_callback_metrics(),luz_callback_mixed_precision(),luz_callback_mixup(),luz_callback_model_checkpoint(),luz_callback_profile(),luz_callback_progress(),luz_callback_resume_from_checkpoint(),luz_callback_train_valid()

Examples

cb <- luz_callback_keep_best_model()

Learning rate scheduler callback

Description

Initializes and runstorch::lr_scheduler()s.

Usage

luz_callback_lr_scheduler(  lr_scheduler,  ...,  call_on = "on_epoch_end",  opt_name = NULL)

Arguments

Value

Aluz_callback() generator.

See Also

Other luz_callbacks:luz_callback(),luz_callback_auto_resume(),luz_callback_csv_logger(),luz_callback_early_stopping(),luz_callback_interrupt(),luz_callback_keep_best_model(),luz_callback_metrics(),luz_callback_mixed_precision(),luz_callback_mixup(),luz_callback_model_checkpoint(),luz_callback_profile(),luz_callback_progress(),luz_callback_resume_from_checkpoint(),luz_callback_train_valid()

Examples

if (torch::torch_is_installed()) {cb <- luz_callback_lr_scheduler(torch::lr_step, step_size = 30)}

Metrics callback

Description

Tracks metrics passed tosetup() during training and validation.

Usage

luz_callback_metrics()

Details

This callback takes care of 2ctx attributes:

• ctx$metrics: stores the current metrics objects that are initialized once for epoch,and are furtherupdate()d andcompute()d every batch. You will rarely needto work with these metrics.

• ctx$records$metrics: Stores metrics per training/validation and epoch. Thestructure is very similar toctx$losses.

Value

Aluz_callback

Note

In general you won't need to explicitly use the metrics callback as it'sused by default infit.luz_module_generator().

See Also

Other luz_callbacks:luz_callback(),luz_callback_auto_resume(),luz_callback_csv_logger(),luz_callback_early_stopping(),luz_callback_interrupt(),luz_callback_keep_best_model(),luz_callback_lr_scheduler(),luz_callback_mixed_precision(),luz_callback_mixup(),luz_callback_model_checkpoint(),luz_callback_profile(),luz_callback_progress(),luz_callback_resume_from_checkpoint(),luz_callback_train_valid()

Automatic Mixed Precision callback

Description

This callback will enabletorch::local_autocast() training model forwardand during loss computation. It will then disable autocast and scale the lossbeforebackward() andopt$step(). Seeherefor more information.

Usage

luz_callback_mixed_precision(...)

Arguments

Value

Aluz_callback

See Also

Other luz_callbacks:luz_callback(),luz_callback_auto_resume(),luz_callback_csv_logger(),luz_callback_early_stopping(),luz_callback_interrupt(),luz_callback_keep_best_model(),luz_callback_lr_scheduler(),luz_callback_metrics(),luz_callback_mixup(),luz_callback_model_checkpoint(),luz_callback_profile(),luz_callback_progress(),luz_callback_resume_from_checkpoint(),luz_callback_train_valid()

Mixup callback

Description

Implementation of'mixup: Beyond Empirical Risk Minimization'.As of today, tested only for categorical data,where targets are expected to be integers, not one-hot encoded vectors.This callback is supposed to be used together withnn_mixup_loss().

Usage

luz_callback_mixup(alpha = 0.4, ..., run_valid = FALSE, auto_loss = FALSE)

Arguments

Details

Overall, we follow thefastai implementationdescribedhere.Namely,

• We work with a single dataloader only, randomly mixing two observations from the same batch.

• We linearly combine losses computed for both targets:loss(output, new_target) = weight * loss(output, target1) + (1-weight) * loss(output, target2)

• We draw different mixing coefficients for every pair.

• We replaceweight withweight = max(weight, 1-weight) to avoid duplicates.

Value

Aluz_callback

See Also

nn_mixup_loss(),nnf_mixup()

Other luz_callbacks:luz_callback(),luz_callback_auto_resume(),luz_callback_csv_logger(),luz_callback_early_stopping(),luz_callback_interrupt(),luz_callback_keep_best_model(),luz_callback_lr_scheduler(),luz_callback_metrics(),luz_callback_mixed_precision(),luz_callback_model_checkpoint(),luz_callback_profile(),luz_callback_progress(),luz_callback_resume_from_checkpoint(),luz_callback_train_valid()

Examples

if (torch::torch_is_installed()) {mixup_callback <- luz_callback_mixup()}

Checkpoints model weights

Description

This saves checkpoints of the model according to the specified metric andbehavior.

Usage

luz_callback_model_checkpoint(  path,  monitor = "valid_loss",  save_best_only = FALSE,  mode = "min",  min_delta = 0)

Arguments

Note

mode andmin_delta are only used whensave_best_only=TRUE.save_best_only will overwrite the saved models if thepath parameterdon't differentiate by epochs.

Read the checkpointing article in the pkgdown website for moreinformation.

See Also

Other luz_callbacks:luz_callback(),luz_callback_auto_resume(),luz_callback_csv_logger(),luz_callback_early_stopping(),luz_callback_interrupt(),luz_callback_keep_best_model(),luz_callback_lr_scheduler(),luz_callback_metrics(),luz_callback_mixed_precision(),luz_callback_mixup(),luz_callback_profile(),luz_callback_progress(),luz_callback_resume_from_checkpoint(),luz_callback_train_valid()

Examples

luz_callback_model_checkpoint(path= "path/to/dir")luz_callback_model_checkpoint(path= "path/to/dir/epoch-{epoch:02d}/model.pt")luz_callback_model_checkpoint(path= "path/to/dir/epoch-{epoch:02d}/model-{monitor:.2f}.pt")

Profile callback

Description

Computes the times for high-level operations in the training loops.

Usage

luz_callback_profile()

Details

Records are saved inctx$records$profile. Times are stored as seconds.Data is stored in the following structure:

• fit time for the entire fit procedure.

• epoch times per epoch

Value

Aluz_callback

Note

In general you don't need to use these callback by yourself because it's alwaysincluded by default infit.luz_module_generator().

See Also

Other luz_callbacks:luz_callback(),luz_callback_auto_resume(),luz_callback_csv_logger(),luz_callback_early_stopping(),luz_callback_interrupt(),luz_callback_keep_best_model(),luz_callback_lr_scheduler(),luz_callback_metrics(),luz_callback_mixed_precision(),luz_callback_mixup(),luz_callback_model_checkpoint(),luz_callback_progress(),luz_callback_resume_from_checkpoint(),luz_callback_train_valid()

Examples

profile_callback <- luz_callback_profile()

Progress callback

Description

Responsible for printing progress during training.

Usage

luz_callback_progress()

Value

Aluz_callback

Note

In general you don't need to use these callback by yourself because it's alwaysincluded by default infit.luz_module_generator().

Printing can be disabled by passingverbose=FALSE tofit.luz_module_generator().

See Also

Other luz_callbacks:luz_callback(),luz_callback_auto_resume(),luz_callback_csv_logger(),luz_callback_early_stopping(),luz_callback_interrupt(),luz_callback_keep_best_model(),luz_callback_lr_scheduler(),luz_callback_metrics(),luz_callback_mixed_precision(),luz_callback_mixup(),luz_callback_model_checkpoint(),luz_callback_profile(),luz_callback_resume_from_checkpoint(),luz_callback_train_valid()

Allow resume model training from a specific checkpoint

Description

Allow resume model training from a specific checkpoint

Usage

luz_callback_resume_from_checkpoint(  path,  ...,  restore_model_state = TRUE,  restore_records = FALSE,  restore_optimizer_state = FALSE,  restore_callbacks_state = FALSE)

Arguments

Note

Read the checkpointing article in the pkgdown website for moreinformation.

See Also

luz_callback_model_checkpoint()

Other luz_callbacks:luz_callback(),luz_callback_auto_resume(),luz_callback_csv_logger(),luz_callback_early_stopping(),luz_callback_interrupt(),luz_callback_keep_best_model(),luz_callback_lr_scheduler(),luz_callback_metrics(),luz_callback_mixed_precision(),luz_callback_mixup(),luz_callback_model_checkpoint(),luz_callback_profile(),luz_callback_progress(),luz_callback_train_valid()

tfevents callback

Description

Logs metrics and other model information in the tfevents file format.Assuming tensorboard is installed, result can be visualized with

Usage

luz_callback_tfevents(logdir = "logs", histograms = FALSE, ...)

Arguments

Details

tensorboard --logdir=logs

Examples

if (torch::torch_is_installed()) {library(torch)x <- torch_randn(1000, 10)y <- torch_randn(1000, 1)model <- nn_linear %>%  setup(loss = nnf_mse_loss, optimizer = optim_adam) %>%  set_hparams(in_features = 10, out_features = 1) %>%  set_opt_hparams(lr = 1e-4)tmp <- tempfile()model %>% fit(list(x, y), valid_data = 0.2, callbacks = list(  luz_callback_tfevents(tmp, histograms = TRUE)))}

Train-eval callback

Description

Switches important flags for training and evaluation modes.

Usage

luz_callback_train_valid()

Details

It takes care of the threectx attributes:

• ctx$model: Responsible for callingctx$model$train() andctx$model$eval(),when appropriate.

• ctx$training: Sets this flag toTRUE when training andFALSE when invalidation mode.

• ctx$loss: Resets theloss attribute tolist() when finished training/ orvalidating.

Value

Aluz_callback

Note

In general you won't need to explicitly use the train_valid callback as it'sused by default infit.luz_module_generator().

See Also

Other luz_callbacks:luz_callback(),luz_callback_auto_resume(),luz_callback_csv_logger(),luz_callback_early_stopping(),luz_callback_interrupt(),luz_callback_keep_best_model(),luz_callback_lr_scheduler(),luz_callback_metrics(),luz_callback_mixed_precision(),luz_callback_mixup(),luz_callback_model_checkpoint(),luz_callback_profile(),luz_callback_progress(),luz_callback_resume_from_checkpoint()

Load trained model

Description

Loads a fitted model. See documentation inluz_save().

Usage

luz_load(path)

Arguments

See Also

Other luz_save:luz_save()

Loads a checkpoint

Description

Works with checkpoints created typically withluz_callback_model_checkpoint().

Usage

luz_load_checkpoint(obj, path, ...)

Arguments

Loads model weights into a fitted object.

Description

This can be useful when you have saved model checkpoints during training andwant to reload the best checkpoint in the end.

Usage

luz_load_model_weights(obj, path, ...)luz_save_model_weights(obj, path)

Arguments

Value

ReturnsNULL invisibly.

Warning

luz_save_model_weights operates inplace, ie modifies the model object to contain thenew weights.

Creates a new luz metric

Description

Creates a new luz metric

Usage

luz_metric(  name = NULL,  ...,  private = NULL,  active = NULL,  parent_env = parent.frame(),  inherit = NULL)

Arguments

Details

In order to implement a newluz_metric we need to implement 3 methods:

• initialize: defines the metric initial state. This function iscalled for each epoch for both training and validation loops.

• update: updates the metric internal state. This function is calledat every training and validation step with the predictions obtained bythe model and the target values obtained from the dataloader.

• compute: uses the internal state to compute metric values. Thisfunction is called whenever we need to obtain the current metricvalue. Eg, it’s called every training step for metrics displayed inthe progress bar, but only called once per epoch to record it’s valuewhen the progress bar is not displayed.

Optionally, you can implement anabbrev field that gives the metric anabbreviation that will be used when displaying metric information in theconsole or tracking record. If noabbrev is passed, the class namewill be used.

Let’s take a look at the implementation ofluz_metric_accuracy so youcan see how to implement a new one:

luz_metric_accuracy <- luz_metric(  # An abbreviation to be shown in progress bars, or   # when printing progress  abbrev = "Acc",   # Initial setup for the metric. Metrics are initialized  # every epoch, for both training and validation  initialize = function() {    self$correct <- 0    self$total <- 0  },  # Run at every training or validation step and updates  # the internal state. The update function takes `preds`  # and `target` as parameters.  update = function(preds, target) {    pred <- torch::torch_argmax(preds, dim = 2)    self$correct <- self$correct + (pred == target)$      to(dtype = torch::torch_float())$      sum()$      item()    self$total <- self$total + pred$numel()  },  # Use the internal state to query the metric value  compute = function() {    self$correct/self$total  })

Note: It’s good practice that thecompute metric returns regular Rvalues instead of torch tensors and other parts of luz will expect that.

Value

Returns new luz metric.

See Also

Other luz_metrics:luz_metric_accuracy(),luz_metric_binary_accuracy(),luz_metric_binary_accuracy_with_logits(),luz_metric_binary_auroc(),luz_metric_mae(),luz_metric_mse(),luz_metric_multiclass_auroc(),luz_metric_rmse()

Examples

luz_metric_accuracy <- luz_metric(  # An abbreviation to be shown in progress bars, or  # when printing progress  abbrev = "Acc",  # Initial setup for the metric. Metrics are initialized  # every epoch, for both training and validation  initialize = function() {    self$correct <- 0    self$total <- 0  },  # Run at every training or validation step and updates  # the internal state. The update function takes `preds`  # and `target` as parameters.  update = function(preds, target) {    pred <- torch::torch_argmax(preds, dim = 2)    self$correct <- self$correct + (pred == target)$      to(dtype = torch::torch_float())$      sum()$      item()    self$total <- self$total + pred$numel()  },  # Use the internal state to query the metric value  compute = function() {    self$correct/self$total  })

Accuracy

Description

Computes accuracy for multi-class classification problems.

Usage

luz_metric_accuracy()

Details

This metric expects to take logits or probabilities at everyupdate. It will then take the columnwise argmax and compareto the target.

Value

Returns new luz metric.

See Also

Other luz_metrics:luz_metric(),luz_metric_binary_accuracy(),luz_metric_binary_accuracy_with_logits(),luz_metric_binary_auroc(),luz_metric_mae(),luz_metric_mse(),luz_metric_multiclass_auroc(),luz_metric_rmse()

Examples

if (torch::torch_is_installed()) {library(torch)metric <- luz_metric_accuracy()metric <- metric$new()metric$update(torch_randn(100, 10), torch::torch_randint(1, 10, size = 100))metric$compute()}

Binary accuracy

Description

Computes the accuracy for binary classification problems where themodel returns probabilities. Commonly used when the loss istorch::nn_bce_loss().

Usage

luz_metric_binary_accuracy(threshold = 0.5)

Arguments

Value

Returns new luz metric.

See Also

Other luz_metrics:luz_metric(),luz_metric_accuracy(),luz_metric_binary_accuracy_with_logits(),luz_metric_binary_auroc(),luz_metric_mae(),luz_metric_mse(),luz_metric_multiclass_auroc(),luz_metric_rmse()

Examples

if (torch::torch_is_installed()) {library(torch)metric <- luz_metric_binary_accuracy(threshold = 0.5)metric <- metric$new()metric$update(torch_rand(100), torch::torch_randint(0, 1, size = 100))metric$compute()}

Binary accuracy with logits

Description

Computes accuracy for binary classification problems where the modelreturn logits. Commonly used together withtorch::nn_bce_with_logits_loss().

Usage

luz_metric_binary_accuracy_with_logits(threshold = 0.5)

Arguments

Details

Probabilities are generated usingtorch::nnf_sigmoid() andthreshold is used toclassify between 0 or 1.

Value

Returns new luz metric.

See Also

Other luz_metrics:luz_metric(),luz_metric_accuracy(),luz_metric_binary_accuracy(),luz_metric_binary_auroc(),luz_metric_mae(),luz_metric_mse(),luz_metric_multiclass_auroc(),luz_metric_rmse()

Examples

if (torch::torch_is_installed()) {library(torch)metric <- luz_metric_binary_accuracy_with_logits(threshold = 0.5)metric <- metric$new()metric$update(torch_randn(100), torch::torch_randint(0, 1, size = 100))metric$compute()}

Computes the area under the ROC

Description

To avoid storing all predictions and targets for an epoch we compute confusionmatrices across a range of pre-established thresholds.

Usage

luz_metric_binary_auroc(  num_thresholds = 200,  thresholds = NULL,  from_logits = FALSE)

Arguments

See Also

Other luz_metrics:luz_metric(),luz_metric_accuracy(),luz_metric_binary_accuracy(),luz_metric_binary_accuracy_with_logits(),luz_metric_mae(),luz_metric_mse(),luz_metric_multiclass_auroc(),luz_metric_rmse()

Examples

if (torch::torch_is_installed()){library(torch)actual <- c(1, 1, 1, 0, 0, 0)predicted <- c(0.9, 0.8, 0.4, 0.5, 0.3, 0.2)y_true <- torch_tensor(actual)y_pred <- torch_tensor(predicted)m <- luz_metric_binary_auroc(thresholds = predicted)m <- m$new()m$update(y_pred[1:2], y_true[1:2])m$update(y_pred[3:4], y_true[3:4])m$update(y_pred[5:6], y_true[5:6])m$compute()}

Mean absolute error

Description

Computes the mean absolute error.

Usage

luz_metric_mae()

Value

Returns new luz metric.

See Also

Other luz_metrics:luz_metric(),luz_metric_accuracy(),luz_metric_binary_accuracy(),luz_metric_binary_accuracy_with_logits(),luz_metric_binary_auroc(),luz_metric_mse(),luz_metric_multiclass_auroc(),luz_metric_rmse()

Examples

if (torch::torch_is_installed()) {library(torch)metric <- luz_metric_mae()metric <- metric$new()metric$update(torch_randn(100), torch_randn(100))metric$compute()}

Mean squared error

Description

Computes the mean squared error

Usage

luz_metric_mse()

Value

A luz_metric object.

See Also

Other luz_metrics:luz_metric(),luz_metric_accuracy(),luz_metric_binary_accuracy(),luz_metric_binary_accuracy_with_logits(),luz_metric_binary_auroc(),luz_metric_mae(),luz_metric_multiclass_auroc(),luz_metric_rmse()

Computes the multi-class AUROC

Description

The same definition asKerasis used by default. This is equivalent to the'micro' method in SciKit Learntoo. Seedocs.

Usage

luz_metric_multiclass_auroc(  num_thresholds = 200,  thresholds = NULL,  from_logits = FALSE,  average = c("micro", "macro", "weighted", "none"))

Arguments

Details

Note that class imbalance can affect this metric unlikethe AUC for binary classification.

Currently the AUC is approximated using the 'interpolation' method described inKeras.

See Also

Other luz_metrics:luz_metric(),luz_metric_accuracy(),luz_metric_binary_accuracy(),luz_metric_binary_accuracy_with_logits(),luz_metric_binary_auroc(),luz_metric_mae(),luz_metric_mse(),luz_metric_rmse()

Examples

if (torch::torch_is_installed()) {library(torch)actual <- c(1, 1, 1, 0, 0, 0) + 1Lpredicted <- c(0.9, 0.8, 0.4, 0.5, 0.3, 0.2)predicted <- cbind(1-predicted, predicted)y_true <- torch_tensor(as.integer(actual))y_pred <- torch_tensor(predicted)m <- luz_metric_multiclass_auroc(thresholds = as.numeric(predicted),                                 average = "micro")m <- m$new()m$update(y_pred[1:2,], y_true[1:2])m$update(y_pred[3:4,], y_true[3:4])m$update(y_pred[5:6,], y_true[5:6])m$compute()}

Root mean squared error

Description

Computes the root mean squared error.

Usage

luz_metric_rmse()

Value

Returns new luz metric.

See Also

Other luz_metrics:luz_metric(),luz_metric_accuracy(),luz_metric_binary_accuracy(),luz_metric_binary_accuracy_with_logits(),luz_metric_binary_auroc(),luz_metric_mae(),luz_metric_mse(),luz_metric_multiclass_auroc()

Creates a metric set

Description

A metric set can be used to specify metrics that are only evaluated duringtraining, validation or both.

Usage

luz_metric_set(metrics = NULL, train_metrics = NULL, valid_metrics = NULL)

Arguments

Saves luz objects to disk

Description

Allows saving luz fitted models to the disk. Objects can be loaded back withluz_load().

Usage

luz_save(obj, path, ...)

Arguments

Warning

Thectx is naively serialized. Ie, we only usesaveRDS() to serialize it.Don't expectluz_save to work correctly if you have unserializable objectsin thectx liketorch_tensors and external pointers in general.

Note

Objects are saved as plain.rds files butobj$model is serializedwithtorch_save before saving it.

See Also

Other luz_save:luz_load()

Loss to be used withcallbacks_mixup().

Description

In the training phase, computes individual losses with regard to two targets, weights them item-wise,and averages the linear combinations to yield the mean batch loss.For validation and testing, defers to the passed-in loss.

Usage

nn_mixup_loss(loss)

Arguments

Details

It should be used together withluz_callback_mixup().

See Also

luz_callback_mixup()

Mixup logic

Description

Logic underlyingluz_callback_mixup().

Usage

nnf_mixup(x, y, weight)

Arguments

Details

Based on the passed-in input and target batches, as well as applicable mixing weights,we return new tensors intended to replace the current batch.The new input batch is a weighted linear combination of input batch items, whilethe new target batch bundles the original targets, as well as the mixing weights, ina nested list.

Value

Alist of:

• x, the new, mixed-up input batch

• y, alist of:

  • ys, alist of:

    • y1, the original targety1

    • y2, the mixed-in targety2

  • weight, the mixing weights

See Also

luz_callback_mixup()

Examples

if (torch::torch_is_installed()) {batch_x <- torch::torch_randn(c(10, 768))batch_y <- torch::torch_randn(10)weight <- torch::torch_tensor(rep(0.9, 10))$view(c(10, 1))nnf_mixup(batch_x, batch_y, weight)}

Create predictions for a fitted model

Description

Create predictions for a fitted model

Usage

## S3 method for class 'luz_module_fitted'predict(  object,  newdata,  ...,  callbacks = list(),  accelerator = NULL,  verbose = NULL,  dataloader_options = NULL)

Arguments

See Also

Other training:evaluate(),fit.luz_module_generator(),setup()

Objects exported from other packages

Description

These objects are imported from other packages. Follow the linksbelow to see their documentation.

generics

fit

Set hyper-parameter of a module

Description

This function is used to define hyper-parameters before callingfit forluz_modules.

Usage

set_hparams(module, ...)

Arguments

Value

The same luz module

See Also

Other set_hparam:set_opt_hparams()

Set optimizer hyper-parameters

Description

This function is used to define hyper-parameters for the optimizer initializationmethod.

Usage

set_opt_hparams(module, ...)

Arguments

Value

The same luz module

See Also

Other set_hparam:set_hparams()

Set's up ann_module to use with luz

Description

The setup function is used to set important attributes and method fornn_modulesto be used with luz.

Usage

setup(module, loss = NULL, optimizer = NULL, metrics = NULL, backward = NULL)

Arguments

Details

It makes sure the module have all the necessary ingredients in order to be fitted.

Value

A luz module that can be trained withfit().

Note

It also adds adevice active field that can be used to query the currentmoduledevice within methods, with egself$device. This is useful whenctx() is not available, eg, when calling methods from outside theluzwrappers. Users can override the default by implementing adevice activemethod in the inputmodule.

See Also

Other training:evaluate(),fit.luz_module_generator(),predict.luz_module_fitted()