Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork3
mlr-org/mlr3batchmark
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
A connector betweenmlr3 andbatchtools. This allows to runlarge-scale benchmark experiments on scheduled high-performancecomputing clusters.
The package comes with two core functions for switching betweenmlr3
andbatchtools
to perform a benchmark:
- After creating a
design
object (as required formlr3
’sbenchmark()
function), instead ofbenchmark()
callbatchmark()
which populates anExperimentRegistry
for the computational jobs ofthe benchmark. You are now in the world ofbatchtools
where you canselectively submit jobs with different resources, monitor the progressor resubmit as needed. - After the computations are finished, collect the results with
reduceResultsBatchmark()
to return tomlr3
. The resulting objectis a regularBenchmarkResult
.
library("mlr3")library("batchtools")library("mlr3batchmark")tasks= tsks(c("iris","sonar"))learners= lrns(c("classif.featureless","classif.rpart"))resamplings= rsmp("cv",folds=3)design= benchmark_grid(tasks=tasks,learners=learners,resamplings=resamplings)reg= makeExperimentRegistry(NA)
## No readable configuration file found## Created registry in '/tmp/RtmpbcuMc4/registry27b8961304f5da' using cluster functions 'Interactive'
ids= batchmark(design,reg=reg)
## Adding algorithm 'run_learner'## Adding problem 'abc694dd29a7a8ce'## Exporting new objects: '2da7eeb80b94fc3b' ...## Exporting new objects: 'c905990877a775af' ...## Exporting new objects: '3acc41a799a260d8' ...## Exporting new objects: 'ecf8ee265ec56766' ...## Overwriting previously exported object: 'ecf8ee265ec56766'## Adding 6 experiments ('abc694dd29a7a8ce'[1] x 'run_learner'[2] x repls[3]) ...## Adding problem 'f9791e97f9813150'## Exporting new objects: '62ac3bb85aabfbaf' ...## Adding 6 experiments ('f9791e97f9813150'[1] x 'run_learner'[2] x repls[3]) ...
submitJobs()
## Submitting 12 jobs in 12 chunks using cluster functions 'Interactive' ...## Error in workhorse(iteration = job$repl, task = data, learner = learner, : ## unused argument (lgr_threshold = lgr::get_logger("mlr3")$threshold)## Error in workhorse(iteration = job$repl, task = data, learner = learner, : ## unused argument (lgr_threshold = lgr::get_logger("mlr3")$threshold)## Error in workhorse(iteration = job$repl, task = data, learner = learner, : ## unused argument (lgr_threshold = lgr::get_logger("mlr3")$threshold)## Error in workhorse(iteration = job$repl, task = data, learner = learner, : ## unused argument (lgr_threshold = lgr::get_logger("mlr3")$threshold)## Error in workhorse(iteration = job$repl, task = data, learner = learner, : ## unused argument (lgr_threshold = lgr::get_logger("mlr3")$threshold)## Error in workhorse(iteration = job$repl, task = data, learner = learner, : ## unused argument (lgr_threshold = lgr::get_logger("mlr3")$threshold)## Error in workhorse(iteration = job$repl, task = data, learner = learner, : ## unused argument (lgr_threshold = lgr::get_logger("mlr3")$threshold)## Error in workhorse(iteration = job$repl, task = data, learner = learner, : ## unused argument (lgr_threshold = lgr::get_logger("mlr3")$threshold)## Error in workhorse(iteration = job$repl, task = data, learner = learner, : ## unused argument (lgr_threshold = lgr::get_logger("mlr3")$threshold)## Error in workhorse(iteration = job$repl, task = data, learner = learner, : ## unused argument (lgr_threshold = lgr::get_logger("mlr3")$threshold)## Error in workhorse(iteration = job$repl, task = data, learner = learner, : ## unused argument (lgr_threshold = lgr::get_logger("mlr3")$threshold)## Error in workhorse(iteration = job$repl, task = data, learner = learner, : ## unused argument (lgr_threshold = lgr::get_logger("mlr3")$threshold)
getStatus()
## Status for 12 jobs at 2025-05-26 09:23:22:## Submitted : 12 (100.0%)## -- Queued : 0 ( 0.0%)## -- Started : 12 (100.0%)## ---- Running : 0 ( 0.0%)## ---- Done : 0 ( 0.0%)## ---- Error : 12 (100.0%)## ---- Expired : 0 ( 0.0%)
reduceResultsBatchmark()
## ## ── <BenchmarkResult> of 0 rows with 0 resampling run ───────────────────────────
- TheLarge-Scale Benchmarking chapter of themlr3book
About
Connector between mlr3 and batchtools
Topics
Resources
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Sponsor this project
Uh oh!
There was an error while loading.Please reload this page.
Contributors7
Uh oh!
There was an error while loading.Please reload this page.