This is the fifth rollout out of several towards a near-future majorrelease. This has been made possible due to a multi-year effort ofinternal re-designs, work with package maintainers, release, and repeat.This release fixes a few more regressions introduced infuture 1.40.0 (2025-04-10) and 1.67.0 (2025-07-29).
clusterEvalQ() on a FutureCluster, introducedinfuture 1.58.0 (2025-06-05), is now an error bydefault - it used to be just a warning.result() on a canceled and interrupted clusterfuture returned the future instead of producing aFutureInterruptError.
Thecluster backend failed when used with anMPIcluster as created byparallel::makeCluster(..., type = "MPI"). This bug wasintroduced infuture (>= 1.40.0)[2025-04-10].
SettingR_FUTURE_PLAN=multisession in an Renvironfile, or a shell startup script, would result in a “fork bomb” whenloading thefuture package. This happened because thesetup of the future backend happened eagerly when thefuture package was loaded instead of being deferred towhen the first future launched. This resulted in new, nested R workersbeing created recursively, until the machine ran out of resources. Thisbug was introduced infuture (>= 1.67.0)[2025-07-29].
value(..., reduce = structure(`+`, init = 42)) isnot supported, because`+` is a primitive function and onemust not set attributes on primitive functions.value()detects this and produces an error suggestion to usereduce = structure("+", init = 42) instead. The latterstill gave the same error, which is now fixed.
This is the fourth rollout out of several towards a near-future majorrelease. This has been made possible due to a multi-year effort ofinternal re-designs, work with package maintainers, release, and repeat.This release fixes a few more regressions introduced infuture 1.40.0 (2025-04-10).
Now argumentworkers forplan(multisession) defaults toavailableCores(constraints = "connections-16"). This willmake the default forplan(multisession) work also oncomputers with a large number of CPU cores (e.g. 192 and 256 cores)while leaving 16 connection slots available for other needs.
Futures now muffle anypackageStartupMessageconditions produced when pre-loading and pre-attaching packages, e.g. asspecified by thepackages argument or those needed byglobal objects.
The pre-validation of the cluster worker allotted to a futurewhen launched was unnecessarily expensive due to a thinko sincefuture 1.40.0 (2025-04-10), e.g. it would take ~0.1-0.2seconds for a multisession future, whereas after the fix it is effectly0.0 seconds.
Callingresolved() on a lazyClusterFuture would collect the result for the firstresolved future in order to free up one worker slot. Now thisis only done if all slots are occupied. The net benefit is that lazycluster futures will be launched faster, unless all workers arebusy.
Cluster and multisession workers initiate more things whencreated, e.g. pre-loading of packages and memoization of availablecores. Previously, such steps were performed only when the first futurewas evaluated on a worker.
If a multicore future that was terminated abruptly (e.g. viatools::pskill() or by the operating system), then it wasnot detected as such. Instead it resulted in an unexpected error thatcould not be recovered from. Now it is detected and aFutureInterruptError is signaled, which can then be handledand the future may bereset().
Calls toresolved() were not registered byFutureJournal.
Future backend factory was created via the calling environmentrather than via the namespace environment where it lives.
This is the third rollout out of several towards a near-future majorrelease that I am really excited about. This has been made possible dueto a multi-year effort of internal re-designs, work with packagemaintainers, release, and repeat. This release fixes a few regressionsintroduced infuture 1.40.0 (2025-04-10), despitepassingall unit,regression, and system tests.
"device", is opened by, for instance, aplot()call without explicitly opening a graphics device. The reason for thischeck is that we rarely want to plot to thedefault graphicsdevice in parallel processing, which typically ends up plotting to aRplots.pdf file that is local to the parallel worker. Ifthat is truly wanted, please open a graphics devices explicitly(e.g. pdf() orpng()) before plotting.Alternatively, explicitly set R optiondevice inside thefuture expression.makeClusterFuture() for creating a cluster ofstateless parallel workers for processing via the future framework. Thisrequires R (>= 4.4.0) [2024-04-24]. Please make sure to readhelp("makeClusterFuture", package = "future") to learnabout potential pitfalls. The plan is to support more corner cases infuture releases, and when not possible, add more mechanisms fordetecting non-supported cases and given an informative error.Settingoptions(warn = 2) on a parallel worker wasignored - warnings were not escalated to errors on the worker, and wasinstead relayed as-is in the parent R session, unlessoptions(warn = 2) was also set in the parent. Nowoptions(warn = 2) on a worker causes warnings to beescalated immediately to errors on the worker, which therefore alsoterminates the future.
future() argumentsstdout andconditions were not applied when packages that werespecified via argumentpackages were loaded and attached.This prevented us from excluding, for instance,packageStartupMessage:s, causing them to be displayed insequential and multicore processing.
When the usingcluster andmultisessionbackends, one could, in some cases, end up with warnings on “package maynot be available when loading” that are produced byserialize(). These type of warnings are nowsuppressed.
Now the cluster future backend tries even harder to shut downparallel cluster workers when shutting down the backend. If it fails tocommunicate with one or more of the parallel workers, it will now closeany socket connections that remain open towards such clusternodes.
The built-in checks for added, removed, or modified graphicaldevices introduced infuture 1.40.0 (2025-04-10), couldproduce false positives, complaining about “A future expression mustclose any opened devices and must not close devices it did not open.Details: 1 devices differ: index=2, before=‘NA’, after=’‘“. The problemwas that it did not prune the empty ’after’ before the check.
Themulticore backend did not relayimmediateCondition:s in a near-live fashion, but only whenthe results of the futures where collected.
Thesequential,cluster,multisession, andmulticore backends relayedimmediateCondition:s, but did not record them properly inthe future object.
This is the second rollout out of three-four major updates, which isnow possible due to a multi-year effort of internal re-designs, workwith package maintainers, release, and repeat. This release fixes tworegressions introduced infuture 1.40.0 (2025-04-10),despite passingallunit, regression, and system tests of the Future API that we havebuilt up over the years. On the upside, fixing these issues led to agreatly improved static-code analyzer for automatically finding globalvariables in future expressions. Also, with this release, we can nowmove on top releasing modern versions of future backendsfuture.callr andfuture.mirai thatsupport interrupting futures and near-live progress updates using theprogressr package. In addition, map-reduce packagessuch asfuture.apply,furrr, anddoFuture can be updated to take advantage of earlyexiting on errors via cancellation of futures.
future() does a better job in identifying globalvariables in the future expression. This is achieved by the static-codeanalysis now walks the abstract syntax tree (AST) of the futureexpression using a strategy that better emulates how the R engineidentifies global variables at run-time.
Addcancel() for canceling one or more futures. Bydefault, it attempts to interrupt any running futures. This replaces theinterrupt() method introduced in the previous version,which now has been removed.
Nowprint() forFuture reports also onthe current state of the future, e.g. ‘created’, ‘running’, ‘finished’,and ‘interrupted’.
Nowprint(plan()) reports on the number of created,launched, and finished futures since the future backend was set. It alsoreports on the total and average runtime of all finished futures thusfar.
Globals in the environment of an anonymous function were lostsince v1.40.0 (2025-04-10). This was partly resolved by updates to thefuture package and partly by updates to theglobals package. This regression has now beenfixed.
Multisession workers stopped inheriting the R package librarypath of the main R session in v1.40.0 (2025-04-10). This regression hasnow been fixed.
In rare cases, a future backend might fail to launch a future andat the same time fail to handle such errors. That would result inhard-to-understand, obscure errors. In case the future backend does notdetect this itself, such errors are now caught by thefuture package and resignaled as informative errors ofclassFutureLaunchError. By always handling launch errors,we assure that futures failing to launch can always be reset andrelaunched again, possible on alternative backend.
When a future fails to launch due to issues with the parallelworker, querying it withvalue() produces aFutureLaunchError. When this happened forcluster ormultisession futures,resolved() would return FALSE and not TRUE as expected. Inaddition, theFutureLaunchError would be lost, resulting insuch futures being stuck in an unresolved state, and theFutureLaunchError error never being signaled.
Shutdown ofcluster andmultisessionworkers could fail if one of the the workers was already terminated,e.g. interrupted or crashed. Now the shutdown of each worker isindependent of the others, lowering the risk of leaving stray PSOCKworkers behind.
The built-in validation that futures do not leave behind strayconnections could, in some cases, result inError in vapply(after, FUN = as.integer, FUN.VALUE = NA_integer_): values must be length 1, but FUN(X[[9]]) result is length 0when there were such stray connections.
interrupt() introduced in previous version has beenremoved. Usecancel() instead. The default forcancel() is to interrupt as well. One reason for the changeis that the word “interrupt” conveys themechanism, whereas the“cancel” conveys theintent, which is the preferred style.Another reason was thatinterrupt() masked ditto of thepopularrlang package, and vice versa - the choicecancel() has fewer name clashes.This is the first rollout out of three major updates, which is nowpossible due to a multi-year effort of internal re-designs, work withpackage maintainers, release, and repeat. This release comes with alarge redesign of how future backends are implemented internally. Onegoal is to lower the threshold for implementing exciting, new features,that has been on hold for too long. Some of these features are availablealready in this release, and more are to come in near-future releases.Another goal is to make it straightforward to implement a newbackend.
This update is fully backward compatible with previous versions.Developers and end-users can expect business as usual. Like allreleases, this version has beenvalidated thoroughlyvia reverse-dependency checks,future.tests checks, andmore.
Nowwith() can be used to evaluate R expressions,including futures, using a temporary future plan. For example,with(plan(multisession), { expr }) evaluates{ expr } using multisession futures, before reverting backto plan set previously by the user. To do the same inside a function,setwith(plan(multisession), local = TRUE), which usesmultisession futures until the function exits.
Addinterrupt(), which interrupts a future, if theparallel backend supports it, otherwise it is silently ignored. It canalso be used on a container (i.e. lists,listenv:s andenvironment) of futures. Interrupts are enabled by default formulticore andmultisession futures. Interruptsare disabled by default forcluster futures, because thereparallel workers may be running on remote machines where the overhead ofinterrupting such workers might be too large. To override the defaults,specifyplan() argumentinterrupts,e.g. plan(cluster, workers = hosts, interrupts = TRUE).
Addreset(), which resets a future that hascompleted, failed, or been interrupted. The future is reset back to alazy, vanilla future that can be relaunched.
value() on containers gained argumentreduce, which specifies a function for reducing the values,e.g. values(fs, reduce = `+`). Optional attributeinit controls the initial value. Note that attributes mustnot be set on primitive functions. As a workaround, usereduce = structure("+", init = 42).
value() on containers gained argumentinorder, which can be used control whether standard outputand conditions are relayed in order ofx, or as soon as afuture inx is resolved. It also controls the order of howvalues are reduced.
value() gained argumentdrop to turnresolved futures into minimal, invalid light-weight futures after theirvalues have been returned. This reduces the memory use. This isparticularly useful when usingreduce in combination withinorder = FALSE. For instance, if you have a list offuturesfs, and you know that you will not need to querythe futures for their values more than once, then it is memory efficientand more performant to usev <- value(fs, reduce = `+`, inorder = FALSE, drop = TRUE).
value() on containers cancels non-resolved futuresif an error is detected in one of the futures.
Addminifuture(), which is likefuture(), but with different default arguments resulting inless overhead with the added burden of having to specify globals andpackages, not having conditions and standard output relayed, andignoring random number generation.
Printingplan() will output details on the futurebackend, e.g. number of workers, number of free workers, backendsettings, and summary of resolved and non-resolved, activefutures.
Interrupted futures are now handled and produce an informativeerror.
Timeout errors triggered bysetTimeLimit() are nowrelayed.
Failures to launch a future is now detected, handled, and relayedas an error with details on why it failed.
Failed workers are automatically detected and relaunched, ifsupported by the parallel backend. For instance, if acluster worker is interrupted, or crashes for otherreasons, it will be relaunched. This works for both local and remoteworkers.
A future must close any connections or graphical devices itopens, and must never close ones that it did not open. Nowvalue() produces a warning if such misuse is detected. Thismay be upgrade to an error in future releases. The default behavior canbe controlled via an R option. Reverse dependency checks spotted oneCRAN package, out of 426, that left stray connections behind.
All parallel backends now prevent nested parallelization, unlessexplicitly allowed, e.g. settings recognized byparallelly::availableCores() or set by the futureplan(). Previously, this had to be implemented by eachbackend, but now it’s handled automatically by the futureframework.
Add new FutureBackend API for writing future backends. Please usewith care, because there will be further updates in the next few releasecycles.
The maximum total size of objects send to and from the worker cannow be configured per backend,e.g. plan(multisession, maxSizeOfObjects = 10e6) willproduce an error if the total size of globals exceeds 10 MB.
Backendssequential andmulticore nolonger has a limit on the maximum size of globals, i.e. they now defaulttomaxSizeOfObjects = +Inf. Backendscluster andmultisession also default tomaxSizeOfObjects = +Inf, unless R optionfuture.globals.maxSize (sic!) is set.
Now ‘interrupt’ conditions are captured during the evaluation ofthe future, and results in the evaluation being terminated with aFutureInterruptError. Not all backends manage to catchinterrupts, leading to the parallel R workers to terminate, resulting ina regularFutureError. Previously, interrupts would resultin non-deterministic behavior and errors depending of futurebackend.
Timeout errors triggered bysetTimeLimit() waslikely to render the future and the corresponding workerinvalid.
Identified and fixed one reason for whycluster andmultisession futures could result in errors on “Unexpectedresult (of class ‘NULL’ != ‘FutureResult’) retrieved forMultisessionFuture future … This suggests that the communication with‘RichSOCKnode’ #1 on host ‘localhost’ (R Under development (unstable)(2025-03-23 r88038), platform x86_64-pc-linux-gnu) is out ofsync.”
Switching plan while having active futures would likely result inthe active futures becoming corrupt, resulting in unpredictable errorswhen querying the future by, for instance,value(), butalsoresolved(), which should never produce an error. Nowsuch futures become predictable, interrupted futures.
future::makeClusterPSOCK() still works, whereasparallelly::makeClusterPSOCK() is the preferred use. Thelong-term goal is to clean out these re-exports. Starting with thisrelease, thefuture package no longer re-exportsautoStopCluster(),makeClusterMPI(),makeNodePSOCK().cleanup attribute,cf. attr(cluster, "cleanup").parallelly::serializedSize().resolved() forClusterFuture:s wouldproduceError: 'inherits(future, "Future")' is not TRUEinstead of an intended, informative error message that the connection tothe parallel worker is broken.workers argument. Forexample,plan(cluster, workers = cl), whereclis acluster object, would come with an extra overhead,because theworkers object was unnecessarily transferred tothe cluster nodes.plan(multisession, workers = I(n)), and same forcluster, preserves the “AsIs” class attribute on theworkers argument so that it is propagated toparallelly::makeClusterWorkers().future.* options.getExpression() on ‘cluster’ future could under somecircumstances calllocal() on the global search path ratherthanbase::local() as intended. For example, if a packagethat exports its ownlocal() function was attached, thenthat would be called instead, often leading to a hard-to-troubleshooterror.Add prototype of an internal event-logging framework for thepurpose of profiling futures and their backends.
Add optionfuture.globalenv.onMisuse for optionallyasserting that a future expression does not result in variables beingadded to the global environment.
Add optionfuture.onFutureCondition.keepFuture forcontrolling whetherFutureCondition objects should keep acopy of theFuture object or not. The default is to keep acopy, but if the future carries large global objects, then theFutureCondition will also be large, which can result inmemory issues and slow downs.
The ‘multiprocess’ strategy, which has been deprecated sincefuture 1.20.0 [2020-10-30] is now defunct. Please use ‘multisession’(recommended) or ‘multicore’ instead.
Add optional assertion of the internal Futurestatefield.
remote(). Note thatplan(remote, ...) has been deprecated sincefuture 1.24.0 [2022-02-19] and defunct sincefuture 1.30.0 (2022-12-15).Deprecatedplan(multiprocess, ...) now equalsplan(sequential), while still producing one warning eachtime a future is created.
Argumentlocal is defunct and has been removed.Previously onlylocal = FALSE was defunct.
Remove defunct argumentvalue from allresolve() methods.
Remove defunct functionstransparent() andTransparentFuture().
futureOf() usedlistenv::map(), whichis deprecated inlistenv (>= 0.9.0) in favor oflistenv::mapping().
Starting with R (>= 4.2.0), the internal functionmyInternalIP() no longer detected when an attempted systemcall failed, resulting in an obscure error instead of falling back toalternatives. This was because errors produced bysystem2()no longer inherits from classsimpleError.
plan(cluster, ..., persistent = TRUE) instead. Note thatpersistent = TRUE will eventually also become deprecatedand defunct, but by then we will have an alternative solutionavailable.R CMD check may produce “checking for detritus in the tempdirectory … NOTE” and how to avoid them.plan(sequential) and whengetOption("warn") == 0. This bug was introduced infuture 1.26.0 [2022-05-27].Using the deprecatedplan(multiprocess) will nowtrigger a deprecation warningeach time amultiprocess future is created. This means that there couldbe a lot of warnings produced. Note thatmultiprocess hasbeen deprecated sincefuture 1.20.0 [2020-10-30].Please usemultisession (recommended) ormulticore instead.
Removingvalues(), which has been defunct sincefuture 1.23.0. Usevalue()instead.
source(..., local = TRUE) is preferred oversource() when used inside futures.do.call(plan, args = list(multisession, workers = 2))would ignore theworkers argument, and any otherarguments.Previously deprecated use oflocal = FALSE withfutures is now defunct.
The R option to temporarily allowplan(transparent)although it was declared defunct has now been removed;plan(transparent), together with functionstransparent() andTransparentFuture() are nowformally defunct.
Using argumentpersistent with multisession futuresis now defunct. Previously onlypersistent = TRUE wasdefunct.
Use CSS style to align image to the right instead of non-HTML5attributealign="right".
Avoid nested<em> tags in HTML-generated helppages.
workers = 1 can now be overridden byspecifyingworkers = I(1).Some warnings and errors showed the wrong call.
print() forFutureResult would reportcaptured conditions all with classlist, instead of theircondition classes.
R CMD check --as-cran on R-devel and MS Windowswould trigger a NOTE on “Check: for detritus in the temp directory” and“Found the following files/directories: ‘Rscript1349cb8aeeba0’ …”. Therewere two package tests that explicitly created PSOCK cluster withoutstopping them. A third test launched multisession future withoutresolving it, which prevented the PSOCK worker to terminate. This wasnot detected in R 4.2.0. It is not a problem on macOS and Linux, becausethere background workers are automatically terminated when the main Rsession terminates.R options and environment variables are now reset on the workersafter future is resolved as they were after any packages required by thefuture has been loaded and attached. Previously, they were reset to whatthey were before these were loaded and attached. In addition, onlypre-existing R options and environment variables are reset. Any new onesadded are not removed for now, because we do not know which added Roptions or environment variables might have been added from loading apackage and that are essential for that package to work.
If it was changed while evaluating the future expression, thecurrent working directory is now reset when the future has beenresolved.
futureSessionInfo() gained argumentanonymize. If TRUE (default), host and user names areanonymized.
futureSessionInfo() now also report on the main Rsession details.
The bug fix infuture 1.22.0 that addressed theissue where objecta infuture(fcn(), globals = list(a = 42, fcn = function() a))would not be found has been redesigned in a more robust way.
Use of packages such asdata.table andff in cluster and multisession futures broke infuture 1.25.0. Fordata.table, we saw“Error in setalloccol(ans) : verbose must be TRUE or FALSE”. Forff, we saw “Error in splitted$path[nopath] <-getOption(”fftempdir”) : replacement has length zero”. See ‘SignificantChanges’ for why and how this was fixed.
The deprecation warning for usinglocal = FALSE wassilenced for sequential futures sincefuture1.25.0.
futureCall() ignored argumentsstdout,conditions,earlySignal,label,andgc.
Strategy ‘transparent’ was deprecated infuture1.24.0 and is now defunct. Useplan(sequential, split = TRUE) instead.
Strategy ‘multiprocess’ was deprecated infuture1.20.0, and ‘remote’ was deprecated infuture 1.24.0.Since then, attempts to use them inplan() would produce adeprecation warning, which was limited to one per R session. Startingwith this release, this warning is now produced whenever usingplan() with these deprecated future strategies.
Nowf <- future(..., stdout = structure(TRUE, drop = TRUE))will cause the captured standard output to be dropped from the futureobject as soon as it has been relayed once, for instance, byvalue(f). Similarly,conditions = structure("conditions", drop = TRUE) willdrop captured non-error conditions as soon as they have been relayed.This can help decrease the amount of memory used, especially if thereare many active futures.
Nowresolve() respects optionfuture.wait.interval. Previously, it was hardcoded to pollfor results every 0.1 seconds.
value() will only attempt to recover UTF-8 symbolsin the captured standard output if the future was evaluated on an MSWindows that does not support capturing of UTF-8 symbols. Support forUTF-8 capturing on also MS Windows was added in R 4.2.0, but ittypically requires an up-to-date MS Windows 10 or MS Windows Server2022.future.wait.interval wasdecreased from 0.2 seconds to 0.01 seconds. This controls the pollingfrequency for finding an available worker when all workers are currentlybusy. Starting with this release, this option also controls the pollingfrequency ofresolve().plan(multicore, workers = 2) andplan(sequential, split = TRUE) introduced breaking sideeffects to the futures evaluated.future(..., seed = TRUE) forwards the RNG state inthe calling R session. Previously, it would leave it intact.plan() andtweak() preserve calls inarguments,e.g. plan(multisession, workers = 2, rscript_startup = quote(options(socketOptions="no-delay"))),andtweak(..., abc = quote(x == y)).nbrOfFreeWorkers() would produce “Error:‘is.character(name)’ is not TRUE” forplan(multisession, workers = 1).
Internal calls toFutureRegistry(action = "collect-first") andFutureRegistry(action = "collect-last") could signal errorsearly when pollingresolved().
Strategy ‘remote’ is deprecated in favor of ‘cluster’. Theplan() function will give an informative deprecationwarning when ‘remote’ is used. For now, this warning is given only onceper R session.
Strategy ‘transparent’ is deprecated in favor of ‘sequential’with argumentsplit = TRUE set. Theplan()function will give an informative deprecation warning when ‘transparent’is used. For now, this warning is given only once per Rsession.
plan() now produces a one-time warning if a‘transparent’ strategy is set. The warning reminds the user that‘transparent’ should only be used for troubleshooting purposes and neverbe used in production. These daysplan(sequential, split = TRUE) together withdebug() is probably a better approach for troubleshooting.The long-term plan is to deprecate the ‘transparent’ strategy.
Support forpersistent = TRUE with multisessionfutures is defunct.
\u2713) would berelayed as<U+2713> (8 characters). The reason forthis is a limitation in R itself on MS Windows. Now,value() attempts to recover such MS Windows output to UTF-8before relaying it. There is an option for disabling this newfeature.quit() must not be used inforked R processes.future(..., seed) will set the random seed as lateas possible just before the future expression is evaluated. Previouslyit was done before package dependencies where attached, which could leadto non-reproduce random numbers in case a package dependency wouldupdate the RNG seed when attached.values(), which has been deprecated sincefuture 1.20.0, is now defunct. Usevalue()instead.
Support forpersistent = TRUE with multisessionfutures is defunct. If still needed, a temporary workaround is to usecluster futures. However, it is likely that support forpersistent will eventually be deprecated for all futurebackends.
Argumentvalue ofresolve(), deprecatedsincefuture 1.15.0, is defunct in favor of argumentresult.
parallel::makeCluster(..., type = "FORK"). This test isdisabled on macOS, where it appears that the main R session becomesunstable after the FORK node is terminated.A lazy future remains a generic future until it is launched,which means it is not assigned a future backend class untillaunched.
Argumentseed forfutureAssign() andfutureCall() now defaults to FALSE just like forfuture().
R_FUTURE_* environment variables are now only readwhen thefuture package is loaded, where they set thecorrespondingfuture.* option. Previously, some of theseenvironment variables were queried by different functions as a fallbackto when an option was not set. By only parsing them when the package isloaded, it decrease the overhead in functions, and it clarifies thatoptions can be changed at runtime whereas environment variables shouldonly be set at startup.
The overhead of initiating futures have been significantlyreduced. For example, the roundtrip time forvalue(future(NULL)) is about twice as fast for‘sequential’, ‘cluster’, and ‘multisession’ futures. For ‘multicore’futures the roundtrip speedup is about 20%. The speedup comes frompre-compiling the R expression that will be used to resolve the futureexpression into R expression templates which then can quickly compiledfor each future. This speeds up the creation of these expression by ~10times, compared when re-compiling them each time.
The default timeout forresolved() was decreasedfrom 0.20 to 0.01 seconds for cluster/multisession and multicorefutures, which means they will spend less time waiting for results whenthey are not available.
Analogously to how globals may be scanned for “non-exportable”objects when optionfuture.globals.onReference is set to"error" or"warning",value()will now check for similar problems in the resolved value object. Anexample of this isf <- future(xml2::read_xml("<body></body>")),which will result in an invalidxml_document object if runin parallel, because such objects cannot be transferred between Rprocesses.
In addition to specify which condition classes to be captured andrelayed, it is now possible to also specify condition classes to beignored. For example,conditions = structure("condition", exclude = "message")captures all conditions but message conditions.
Now cluster futures usehomogeneous = NULL as thedefault instead ofhomogeneous = TRUE. The new default willresult in theparallelly package trying to inferwhether TRUE or FALSE should be used based on theworkersargument.
Now the the post-mortem analysis report of multicore and clusterfutures in case their results could not be retrieved include informationon globals and their sizes, and if some of them are non-exportable. Asimilar, detailed report is also produced when a cluster future fails toset up and launch itself on a parallel worker.
if optionfuture.fork.multithreading.enable isFALSE,RcppParallel, in addition toOpenMP, is forced to run with a single threadedwhenever running in a forked process (=‘multicore’ futures). This isdone by setting environment variableRCPP_PARALLEL_NUM_THREADS to 1.
AddfutureSessionInfo() to get a quick overview ofthe future framework, its current setup, and to run simple tests onit.
Nowplan(multicore) warns immediately if multicoreprocessing, that is, forked processing, is not supported, e.g. whenrunning in the RStudio Console.
plan(multiprocess, workers = n) did not warn about‘multiprocess’ being deprecated when argumentworkers wasspecified.
getGlobalsAndPackages() could throw a false error on“Did you mean to create the future within a function? Invalid futureexpression tries to use global... variables that do notexist:... is solely part of aformula or used in some S4 generic functions.
When enabled, optionfuture.globals.onReferencecould falsely alert on ‘Detected a non-exportable reference(externalptr) in one of the globals (<unknown>) used in the futureexpression’ in globals, e.g. when usingfuture.apply orfurrr map-reduce functions when using a ‘multisession’backend.
future(fcn(), globals = list(a = 42, fcn = function() a))would fail with “Error in fcn() : object ‘a’ not found” when usingsequential or multicore futures. This affected also map-reduce callssuch asfuture.apply::future_lapply(1, function(x) a, future.globals = list(a = 42)).
Resolving a ‘sequential’ future without globals would result ininternal several...future.* objects being written to thecalling environment, which might be the global environment.
Environment variableR_FUTURE_PLAN would propagatedown with nested futures, forcing itself onto also nested future plans.Now it is unset in nested futures, resulting in a sequential futurestrategy unless another was explicitly set byplan().
Transparent futures no longer warn aboutlocal = FALSE being deprecated. Althoughlocal = FALSE is being deprecated, it is still usedinternally by ‘transparent’ futures for a while longer. Please do notuse ‘transparent’ futures in production code and never in apackage.
remote() could produce an error on “object‘homogeneous’ not found”.
nbrOfFreeWorkers() for ‘cluster’ futures assumedthat the current plan is set to cluster too.
In order to handle them conditionally higher up in the callchain, warnings and errors produced from using the random numbergenerator (RNG) in a future without declaring the intention to use oneare now of classRngFutureWarning andRngFutureError, respectively. Both of these classesinherits fromRngFutureCondition.
Now run-time errors from resolving a future take precedence overRngFutureError:s. That is,future({ rnorm(1); log("a") }, seed = FALSE) will signalan error ‘log(“a”)’ instead of an RNG error when optionfuture.rng.onMisuse is set to"error".
nbrOfFreeWorkers() to query how many workers arefree to take on futures immediately. Until all third-party futurebackends have implemented this, some backends might produce an errorsaying it is not yet supported.future(..., seed = TRUE) with ‘sequential’ futureswould set the RNG kind of the parent process. Now it behaves the sameregardless of future backend.
SignalingimmediateCondition:s with ‘multicore’could result inError in save_rds(obj, file) : save_rds() failed to rename temporary save file '/tmp/RtmpxNyIyK/progression21f3f31eadc.rds.tmp' (NA bytes; last modified on NA) to '/tmp/RtmpxNyIyK/progression21f3f31eadc.rds' (NA bytes; last modified on NA).There was an assertion at the end of the internalsave_rds() function that incorrectly assumed that thetarget file should exist. However, the file might have already beenprocessed and removed by the future in the main R session.
value() with both a run-time error and an RNGmistake would signal the RNG warning instead of the run-time error whenthe for-internal-use-only argumentsignal was set toFALSE.
Due to a mistake introduced infuture 1.20.0,the package would end up assigning a.packageVersion objectto the global environment when loaded.
future::plan("multisession") would produce ‘Error in if(debug) mdebug(“covr::package_coverage() workaround …”) : argument isnot interpretable as logical’ if and only if thecovrpackage was loaded.Strategy ‘multiprocess’ is deprecated in favor of either‘multisession’ or ‘multicore’, depending on operating system and Rsetup. Theplan() function will give an informativedeprecation warning when ‘multiprocess’ is used. This warning is givenonly once per R session.
Launching R or Rscript with command-line option--parallel=n, where n > 1, will now use ‘multisession’as future strategy. Previously, it would use ‘multiprocess’, which isnow deprecated.
Support forlocal = FALSE is deprecated. For thetime being, it remains supported for ‘transparent’ futures and ‘cluster’futures that usepersistent = TRUE. However, note thatpersistent = TRUE will also deprecated at some point inthe future. These deprecations are required in order to furtherstandardize the Future API across various types of parallelbackends.
Now multisession workers inherit the package library path fromthe main R sessionwhen they are created, that is, when callingplan(multisession). To avoid this, useplan(multisession, rscript_libs = NULL), which is anargument passed down tomakeClusterPSOCK(). With thisupdate, ‘sequential’, ‘multisession’, and ‘multicore’ futures see theexact same library path.
Several functions for managingparallel-styleprocessing have been moved to a newparallelly package.Specifically, functionsavailableCores(),availableWorkers(),supportsMulticore(),as.cluster(),autoStopCluster(),makeClusterMPI(),makeClusterPSOCK(), andmakeNodePSOCK() have been moved. None of them are specificto futures per se and are likely useful elsewhere too. Also, having themin a separate, standalone package will speed up the process of releasingany updates to these functions. The code base of thefuture package shrunk about 10-15% from this migration.For backward compatibility, the migrated functions remain in thispackage as re-exports.
Setting up a future strategy with argumentsplit = TRUE will cause the standard output and non-errorconditions to be split (“tee:d”) on the worker’s end, while stillrelaying back to the main R session as before. This can be useful whendebugging withbrowse() ordebug(),e.g. plan(sequential, split = TRUE). Without it, debugoutput is not displayed.
Now multicore futures relayimmediateCondition:s ina near-live fashion.
It is now possible to pass any arguments thatmakeClusterPSOCK() accepts in the call toplan(cluster, ...) andplan(multisession, ...). For instance, to set theworking directory of the cluster workers to a temporary folder, passargumentrscript_startup = "setwd(tempdir())". Anotherexample isrscript_libs = c(libs, "*") to prepend thelibrary path on the worker with the paths inlibs.
plan() andtweak() check for even morearguments that must not be set by either of them. Specifically, attemptsto adjust the following arguments offuture() will resultin an error:conditions,envir,globals,packages,stdout, andsubstitute in addition to already validatedlazy andseed.
tweak() now returns a wrapper function that callsthe original future strategy function with the modified defaults.Previously, it would make a copy of the original function with modifiedargument defaults. This new approach will make it possible to introducenew future arguments that can be modified bytweak() andplan() without having to update every future backendpackage, e.g. the newsplit = TRUE argument.
Add a ‘Best Practices for Package Developers’ vignette.
Add a ‘How the Future Framework is Validated’ vignette.
substitute = TRUE.Since last version,future 1.19.1,future(..., conditions = character(0L)) would no longeravoid intercepting conditions as intended; instead, it muffles allconditions. From now on, useconditions = NULL.
Relaying ofimmediateCondition:s was not near-livefor multisession and cluster if the underlying PSOCK cluster useduseXDR=FALSE for communication.
print() for Future would also print any attributesof its environment.
The error message produced bynbrOfWorkers() wasincomplete.
Renamed environment variableR_FUTURE_MAKENODEPSOCK_tries used bymakeClusterPSOCK() toR_FUTURE_MAKENODEPSOCK_TRIES.
The Mandelbrot demo would produce random numbers withoutdeclaring so.
Strategy ‘multiprocess’ is deprecated in favor of either‘multisession’ or ‘multicore’, depending on operating system and Rsetup.
values() is deprecated. Usevalue()instead.
All backward compatible code for the legacy, defunct, internalFuture elementvalue is now removed. Using orrelying on it is an error.
... as a globals, rather than viaarguments, in higher-level map-reduce APIs such asfuture.apply andfurrr, arguments in... could produce an error on “unused argument”.Futures detect when random number generation (RNG) was used toresolve them. If a future uses RNG without parallel RNG was requested,then an informative warning is produced. To request parallel RNG,specify argumentseed,e.g. f <- future(rnorm(3), seed = TRUE) ory %<-% { rnorm(3) } %seed% TRUE. Higher-level map-reduceAPIs provide similarly named “seed” arguments to achieve the same. To,escalate these warning to errors, set optionfuture.rng.onMisuse to"error". To silencethem, set it to"ignore".
Now, all non-captured conditions are muffled, if possible. Forinstance,future(warning("boom"), conditions = c("message")) willtruly muffle the warning regardless of backend used. This was needed tofix below bug.
makeClusterPSOCK() will now retry to create acluster node up totries (default: 3) times before givingup. If argumentport species more than one port(e.g. port = "random") then it will also attempt find avalid random port up totries times before giving up. Thepre-validation of the random port is only supported in R (>= 4.0.0)and skipped otherwise.
makeClusterPSOCK() skips shell quoting of theelements inrscript if it inherits fromAsIs.
makeClusterPSOCK(), or actuallymakeNodePSOCK(), gained argumentquiet, whichcan be used to silence output produced bymanual = TRUE.
If multithreading is disabled but multicore futures fail toacknowledge the setting on the current system, then an informativeFutureWarning is produced by such futures.
NowavailableCores() better supports Slurm.Specifically, if environment variableSLURM_CPUS_PER_TASKis not set, which requires that option--slurm-cpus-per-task=n is specified andSLURM_JOB_NUM_NODES=1, then it falls back to usingSLURM_CPUS_ON_NODE, e.g. when using--ntasks=n.
NowavailableCores() andavailableWorkers() supports LSF/OpenLava. Specifically,they acknowledge environment variableLSB_DJOB_NUMPROC andLSB_HOSTS, respectively.
plan(multisession),plan(cluster, workers = <number>), andmakeClusterPSOCK() which they both use internally, sets uplocalhost workers twice as fast compared to versions sincefuture 1.12.0, which brings it back to par with abare-boneparallel::makeCluster(..., setup_strategy = "sequential")setup. The slowdown was introduced infuture 1.12.0(2019-03-07) when protection against leaving stray R processes behindfrom failed worker startup was implemented. This protection now makesuse of memoization for speedup.Sequential and multicore backends, but not multisession, wouldproduce errors on “‘…’ used in an incorrect context” in cases where... was part of argumentglobals and not theevaluation environment.
Contrary to other future backends, any conditions produced whileresolving a sequential future usingfuture(..., conditions = character()) would be signaled,although the most reasonable expectation would be that they aresilenced. Now, all non-captured conditions are muffled, ifpossible.
Optionfuture.rng.onMisuse was not passed down tonested futures.
Disabling multithreading in forked processes by setting R optionfuture.fork.multithreading.enable or environment variableR_FUTURE_FORK_MULTITHREADING_ENABLE toFALSEwould cause multicore futures to always return value1L.This bug was introduced infuture 1.17.0(2020-04-17).
getGlobalsAndPackages() did not always return aglobals element that was of classFutureGlobals.
getGlobalsAndPackages(..., globals) wouldrecalculatetotal_size even when it was already calculatedor known to be zero.
getGlobalsAndPackages(Formula::Formula(~ x)) wouldproduce “the condition has length > 1” warnings (which will becomeerrors in future R versions).
persistent = TRUE with multisession futuresis deprecated.print() onRichSOCKcluster givesinformation not only on the name of the host but also on the version ofR and the platform of each node (“worker”), e.g. “Socket cluster with 3nodes where 2 nodes are on host ‘localhost’ (R version 4.0.0(2020-04-24), platform x86_64-w64-mingw32), 1 node is on host ‘n3’ (Rversion 3.6.3 (2020-02-29), platform x86_64-pc-linux-gnu)”.
Error messages from cluster future failures are now moreinformative than “Unexpected result (of class ‘NULL’ !=‘FutureResult’)”. For example, if thefuture package isnot installed on the worker, then the error message clearly says so.Even, if there is an unexpected result error from a PSOCK clusterfuture, then the error produced give extra information on node where itfailed, e.g. “Unexpected result (of class ‘NULL’ != ‘FutureResult’)retrieved for ClusterFuture future (label = ‘ClusterFutureworker (‘RichSOCKnode’ #1 on host ‘n3’ (R version 3.6.3 (2020-02-29),platform x86_64-pc-linux-gnu)) is out of sync.”
It is now possible to set environment variables on workers beforethey are launched bymakeClusterPSOCK() by specify them asas"<name>=<value>" as part of therscript vector argument,e.g. rscript = c("ABC=123", "DEF='hello world'", "Rscript").This works because elements inrscript that match regularexpression[[:alpha:]_][[:alnum:]_]*=.* are no longer shellquoted.
makeClusterPSOCK() now returns a cluster that inaddition to inheriting fromSOCKcluster it will alsoinherit fromRichSOCKcluster.
MademakeClusterPSOCK() andmakeNodePSOCK() agile to the name change fromparallel:::.slaveRSOCK() toparallel:::.workRSOCK() in R (>= 4.1.0).
makeClusterPSOCK(..., rscript) will not try tolocaterscript[1] if argumenthomogeneous isFALSE (or inferred to be FALSE).
makeClusterPSOCK(..., rscript_envs) would result ina syntax error when starting the workers due to non-ASCII quotationmarks if optionuseFancyQuotes was not set toFALSE.
plan(list(...)) would produce ‘Error inUseMethod(“tweak”) : no applicable method for ’tweak’ applied to anobject of class “list”’ if a non-function object named ‘list’ was on thesearch path.
plan(x$abc) with x <- list(abc = sequential)would produce ‘Error in UseMethod(“tweak”) : no applicable method for’tweak’ applied to an object of class “c(‘FutureStrategyList’,‘list’)”’.
TESTS:R_FUTURE_FORK_ENABLE=false R CMD check ...would produce ‘Error: connections left open: …’ when checking the‘multiprocess’ example.
Support forpersistent = TRUE with multisessionfutures is deprecated. If still needed, a temporary workaround is to usecluster futures. However, it is likely that support forpersistent will eventually be deprecated for all futurebackends.
Optionsfuture.globals.method,future.globals.onMissing, andfuture.globals.resolve are deprecated and produce warningsif set. They may only be used for troubleshooting purposes because theymay affect how futures are evaluated, which means that reproducibilitycannot be guaranteed elsewhere.
values() tovalue() to clean upand simplify the API.makeClusterPSOCK() gained argumentrscript_envs for setting environment variables in workerson startup,e.g. rscript_envs = c(FOO = "3.14", "BAR").
Now the result of a future holds session details in case an erroroccurred while evaluating the future.
_R_CHECK_LIMIT_CORES_ set. Tobetter emulate CRAN submission checks, thefuturepackage will, when loaded, set this environment variable to ‘TRUE’ ifunset and ifR CMD check is running. Note thatfuture::availableCores() respects_R_CHECK_LIMIT_CORES_ and returns at most2L(two cores) if detected.Any globals namedversion andhas_future would be overwritten with “garbage” valuesinternally.
Disabling of multi-threading when using ‘multicore’ futures didnot work on all platforms.
values() S3 methods have been renamed tovalue() since they are closely related to the originalpurposevalue(). Thevalues() methods willcontinue to work but will soon be formally deprecated and later be madedefunct and finally be removed. Please replace allvalues()withvalue() calls.oplan <- plan(new_strategy) returns the list ofall nested strategies previously set, instead of just the strategy ontop of this stack. This makes it easier to temporarily use another plan.For the old behavior, useoplan <- plan(new_strategy)[[1]].Nowvalue() detects if afuture(..., seed = FALSE) call generated random numbers,which then might give unreliable results because non-parallel safe,non-statistically sound random number generation (RNG) was used. Ifoptionfuture.rng.onMisuse is"warning", awarning is produced. If"error", an error is produced. If"ignore" (default), the mistake is silently ignored. Usingseed = NULL is likeseed = FALSE but withoutperforming the RNG validation.
For convenience, argumentseed offuture() may now also be an ordinary single integer randomseed. If so, a L’Ecuyer-CMRG RNG seed is created from this seed. Ifseed = TRUE, then a L’Ecuyer-CMRG RNG seed based on thecurrent RNG state is used. Useseed = FALSE when it isknown that the future does not use RNG.
ClusterFuture:s now relayimmediateCondition:s back to the main process momentarilyafter they are signaled and before the future is resolved.
future.fork.multithreading.enable or environment variableR_FUTURE_FORK_MULTITHREADING_ENABLE toFALSE.This requires thatRhpcBLASctl package is installed.Parallelization via multi-threaded processing (done in native code bysome packages and externally library) while at the same time usingforked (aka “multicore”) parallel processing is unstable in some cases.Note that this is not only true when usingplan(multicore)but also when using, for instance,parallel::mclapply().This is in beta so the above names and options might change later.Evaluation of futures could fail if the global environmentcontainedfunctions with the same names as a small set of baseR functions, e.g. raw(),list(), andoptions().
future(alist(a =)) would produce “Error inobjectSize_list(x, depth = depth - 1L) : argument”x_kk” is missing, withno default”
Future andFutureResult objects with aninternal version 1.7 or older have been deprecated since 1.14.0 (July2019) and are now defunct.
Defunct hidden argumentprogress ofresolve(), and hidden arguments/fieldscondition andcalls ofFutureResult are now gone.
makeClusterPSOCK()draws a random port from (when argumentport is notspecified) can now be controlled by environment variableR_FUTURE_RANDOM_PORTS. The default range is still11000:11999 as with theparallelpackage.resolved() infuture 1.15.0 would cause lazy futures to block if allworkers were occupied.resolved() will now launch lazy futures.Now the “visibility” of future values is recorded and reflectedbyvalue().
Now optionfuture.globals.onReference defaults toenvironment variableR_FUTURE_GLOBALS_ONREFERENCE.
?makeClusterPSOCKwith instructions on how to troubleshoot when the setup of local andremote clusters fail.values() would resignalimmediateCondition:s despite those should only be signaledat most once per future.
makeClusterPSOCK() could produce warnings like“cannot open file‘/tmp/alice/Rtmpi69yYF/future.parent=2622.a3e32bc6af7.pid’: No suchfile”, e.g. when launching R workers running in Dockercontainers.
Package would set or update the RNG state of R(.Random.seed) when loaded, which could affect RNGreproducibility.
Package could set.Random.seed to NULL, instead ofremoving it, which in turn would produce a warning on “‘.Random.seed’ isnot an integer vector but of type ‘NULL’, so ignored” when the nextrandom number generated.
Now a future assignment to list environments produce moreinformative error messages if attempting to assign to more than oneelement.
makeClusterMPI() did not work for MPI clusters withcomm other than1.
Argumentvalue ofresolve() isdeprecated. Useresult instead.
Use of internal argumentevaluator tofuture() is now defunct.
All types of conditions are now captured and relayed. Previously,only conditions of classmessage andwarningwere relayed.
If one of the futures in a collection produces an error, thenvalues() will signal that error as soon as it is detected.This means that while callingvalues() guarantees toresolve all futures, it does not guarantee that the result from allfutures are gathered back to the master R session before the error isrelayed.
values() now relaysstdout and signalas soon as possible as long as the standard output and the conditionsare relayed in their original order.
If a captured condition can be “muffled”, then it will bemuffled. This helps to prevent conditions from being handled twice bycondition handlers when futures are evaluated in the main R session,e.g. plan(sequential). Messages and warnings were alreadymuffled in the past.
Forked processing is considered unstable when running R fromcertain environments, such as the RStudio environment. Because of this,‘multicore’ futures have been disabled in those cases sincefuture 1.13.0. This change caught several RStudio usersby surprise. Starting withfuture 1.14.0, aninformative one-time-per-session warning will be produced when attemptsto use ‘multicore’ is made in non-supported environments such asRStudio. This warning will also be produced when using ‘multiprocess’,which will fall back to using ‘multisession’ futures. The warning can bedisabled by setting R optionfuture.supportsMulticore.unstable, or environment variableFUTURE_SUPPORTSMULTICORE_UNSTABLE to"quiet".
Now optionfuture.startup.script falls back toenvironment variableR_FUTURE_STARTUP_SCRIPT.
Conditions inheritingimmediateCondition aresignaled as soon as possible. Contrary to other types of conditions,these will be signaled only once per future, despite beingcollected.
Early signaling did not take place forresolved()forClusterFuture andMulticoreFuture.
When early signaling was enabled, functions such asresolved() andresolve() would relay capturedconditions multiple times. This would, for instance, result in the samemessages and warnings being outputted more than once. Now it is onlyvalue() that will resignal conditions.
The validation of connections failed to detect when theconnection had been serialized (= aNIL external pointer)on some macOS systems.
Argumentprogress ofresolve() is nowdefunct (was deprecated sincefuture 1.12.0). Optionfuture.progress is ignored. This will make room for otherprogress-update mechanisms that are in the works.
Usage of internal argumentevaluator tofuture() is now deprecated.
Removed defunct argumentoutput fromFutureError().
FutureResult fields/argumentsconditionandcalls are now defunct. Useconditionsinstead.
Future andFutureResult objects with aninternal version 1.7 or older are deprecated and will eventually becomedefunct. Future backends that implement their ownFutureclasses should update to implement aresult() methodinstead of avalue() method for theirFutureclasses. All future backends available on CRAN and Bioconductor havealready been updated accordingly.
help("supportsMulticore") for more details, e.g. how tore-enable process forking. Note that parallelization via ‘multisession’is unaffected and will still work as before. Also, when forkedprocessing is disabled, or otherwise not supported, usingplan("multiprocess") will fall back to using ‘multisession’futures.Forked processing can be disabled by setting R optionfuture.fork.enable to FALSE (or environment variableR_FUTURE_FORK_ENABLE=false). When disabled, ‘multicore’futures fall back to a ‘sequential’ futures even if the operating systemsupports process forking. If set of TRUE, ‘multicore’ will not fall backto ‘sequential’. If NA, or not set (the default), a set ofbest-practices rules will decide whether forking is enabled or not. Seehelp("supportsMulticore") for more details.
NowavailableCores() also recognizes PBS environmentvariableNCPUS, because the PBSPro scheduler does not setPBS_NUM_PPN.
If, optionfuture.availableCores.custom is set to afunction, thenavailableCores() will call that function andinterpret its value as number of cores. Analogously, optionfuture.availableWorkers.custom can be used to specify ahostnames of a set of workers thatavailableWorkers() sees.These new options provide a mechanism for anyone to customizeavailableCores() andavailableWorkers() incase they do not (yet) recognize, say, environment variables that arespecific the user’s compute environment or HPC scheduler.
makeClusterPSOCK() gained support for argumentrscript_startup for evaluating one or more R expressions inthe background R worker prior to the worker event loop launching. Thisprovides a more convenient approach than having to use, say,rscript_args = c("-e", sQuote(code)).
makeClusterPSOCK() gained support for argumentrscript_libs to control the R package library search pathon the workers. For example, toprepend the folder~/R-libs on the workers, userscript_libs = c("~/R-libs", "*"), where"*"will be resolved to the current.libPaths() on theworkers.
Debug messages are now prepended with a timestamp.
makeClusterPSOCK() did not shell quote the Rscriptexecutable when running its pre-tests checking whether localhost Rscriptprocesses can be killed by their PIDs or not.value ofresolve() has beenrenamed toresult to better reflect that not only valuesare collected when this argument is used. Argumentvaluestill works for backward compatibility, but will eventually be formallydeprecated and then defunct.IfmakeClusterPSOCK() fails to create one of manynodes, then it will attempt to stop any nodes that were successfullycreated. This lowers the risk for leaving R worker processesbehind.
Future results now hold the timestamps when the evaluation of thefuture started and finished.
Functions no longer produce “partial match of ‘condition’ to‘conditions’” warnings withoptions(warnPartialMatchDollar = TRUE).
When future infix operators (%conditions%,%globals%,%label%,%lazy%,%packages%,%seed%, and%stdout%)that are intended for future assignments were used in the wrong context,they would incorrectly be applied to the next future created. Nowthey’re discarded.
makeClusterPSOCK() infuture (>=1.11.1) produced warnings when argumentrscript hadlength(rscript) > 1.
Validation of L’Ecuyer-CMRG RNG seeds failed in recent Rdevel.
Withoptions(OutDec = ","), the default value ofseveral argument would resolve toNA_real_ rather than anumeric value resulting in errors such as “is.finite(alpha) is notTRUE”.
Argumentprogress ofresolve() is nowdeprecated.
Argumentoutput ofFutureError() is nowdefunct.
FutureError no longer inheritssimpleError.
makeClusterPSOCK() fails to connect to a worker,it produces an error with detailed information on what could havehappened. In rare cases, another error could be produced when generatingthe information on what the workers PID is.The defaults of several arguments ofmakeClusterPSOCK() andmakeNodePSOCK() can nowbe controlled via environment variables in addition to R options thatwas supported in the past. An advantage of using environment variablesis that they will be inherited by child processes, also nestedones.
The printing of future plans is now less verbose when theworkers argument is a complex object such as a PSOCKcluster object. Previously, the output would include verbose output ofattributes, etc.
R CMD check is running or not. If it is, then afew future-specific environment variables are adjusted such that thetests play nice with the testing environment. For instance, it sets thesocket connection timeout for PSOCK cluster workers to 120 seconds(instead of the default 30 days!). This will lower the risk for more andmore zombie worker processes cluttering up the test machine (e.g. CRANservers) in case a worker process is left behind despite the main Rprocesses is terminated. Note that these adjustments are appliedautomatically to the checks of any package that depends on, or imports,thefuture package.makeClusterPSOCK() would fail to connect to aworker, for instance due to a port clash, then it would leave the Rworker process running - also after the main R process terminated. Whenthe worker is running on the same machine,makeClusterPSOCK() will now attempt to kill such stray Rprocesses. Note thatparallel::makePSOCKcluster() still hasthis problem.The future call stack (“traceback”) is now recorded when theevaluation of a future produces an error. Usebacktrace()on the future to retrieve it.
NowfutureCall() defaults toargs = list() making is easier to call functions that donot take arguments,e.g. futureCall(function() 42).
plan() gained argument.skip = FALSE.When TRUE, setting the same future strategy as already set will beskipped, e.g. callingplan(multisession) consecutively willhave the same effect as calling it just once.
makeClusterPSOCK() produces more informative errormessages whenever the setup of R workers fails. Also, its verbosemessages are now prefixed with[local output] to helpdistinguish the output produced by the current R session from thatproduced by background workers.
It is now possible to specify what type of SSH clientsmakeClusterPSOCK() automatically searches for and in whatorder,e.g. rshcmd = c("<rstudio-ssh>", "<putty-plink>").
NowmakeClusterPSOCK() preserves the global RNGstate (.Random.seed) also when it draws a random portnumber.
makeClusterPSOCK() gained argumentrshlogfile.
Cluster futures provide more informative error messages when thecommunication with the worker node is out of sync.
Argumentstdout was forced to TRUE when usingsingle-core multicore or single-core multisession futures.
When evaluated in a local environment,futureCall(..., globals = "a") would set the value ofglobala to NULL, regardless if it exists or not and whatits true value is.
makeClusterPSOCK(..., rscript = "my_r") would insome cases fail to find the intendedmy_rexecutable.
ROBUSTNESS: A cluster future, including a multisession one, couldretrieve results from the wrong workers if a new set of cluster workershad been set up after the future was created/launched but before theresults were retrieved. This could happen because connections in R areindexed solely by integers which are recycled when old connections areclosed and new ones are created. Now cluster futures assert that theconnections to the workers are valid, and if not, an informative errormessage is produced.
Callingresult() on a non-resolvedUniprocessFuture would signal evaluation errors.
future::future_lapply(). Please use theone in thefuture.apply package instead.Add support for manually specifying globals in addition to thosethat are automatically identified via argumentglobals or%globals%. Two examples areglobals = structure(TRUE, add = list(a = 42L, b = 3.14))andglobals = structure(TRUE, add = c("a", "b")).Analogously, attributeignore can be used to excludeautomatically identified globals.
The error reported when failing to retrieve the results of afuture evaluated on a localhost cluster/multisession worker or aforked/multicore worker is now more informative. Specifically, itmentions whether the worker process is still alive or not.
AddmakeClusterMPI(n) for creating MPI-basedclusters of a similar kind asparallel::makeCluster(n, type = "MPI") but that alsoattempts to workaround issues whereparallel::stopCluster()causes R to stall.
makeClusterPSOCK() andmakeClusterMPI()gained argumentautoStop for controlling whether thecluster should be automatically stopped when garbage collected ornot.
BETA: Nowresolved() forClusterFutureis non-blocking also for clusters of typeMPIcluster ascreated byparallel::makeCluster(..., type = "MPI").
plan(multiprocess) would not initiate theworkers. Instead workers would be set up only when the first future wascreated.value() is called. This new behavior can be controlled bythe argumentstdout tofuture() or byspecifying the%stdout% operator if a future assignment isused.R optionwidth is passed down so that standardoutput is captured consistently across workers and consistently with themaster process.
Now morefuture.* options are passed down so thatthey are also acknowledged when using nested futures.
Add vignette on ‘Outputting Text’.
CLEANUP: Only the core parts of the API are now listed in thehelp index. This was done to clarify the Future API. Help for non-coreparts are still via cross references in the indexed API as well viahelp().
When using forced, nested ‘multicore’ parallel processing, suchas,plan(list(tweak(multicore, workers = 2), tweak(multicore, workers = 2))),then the child process would attempt to resolve futures owned by theparent process resulting in an error (on ‘bad error message’).
When usingplan(multicore), if a forked worker wouldterminate unexpectedly, it could corrupt the master R session such thatany further attempts of using forked workers would fail. A forked workercould be terminated this way if the user pressedCtrl-C (theworker receives aSIGINT signal).
makeClusterPSOCK() produced a warning whenenvironment variableR_PARALLEL_PORT was set torandom (e.g. as on CRAN).
Printing aplan() could produce an error when thedeparsed call used to set up theplan() was longer than 60characters.
future::future_lapply() is defunct (gives an errorif called). Please use the one in thefuture.applypackage instead.
Argumentoutput ofFutureError() isformally deprecated.
Removed allFutureEvaluationCondition classes andrelated methods.
getGlobalsAndPackages() gained argumentmaxSize.
makeClusterPSOCK() now produces a more informativewarning if environment variableR_PARALLEL_PORT specifies anon-numeric port.
Nowplan() gives a more informative error message incase it fails, e.g. when the internal future validation fails andwhy.
AddedUnexpectedFutureResultError to be used bybackends for signaling in a standard way that an unexpected result wasretrieved from a worker.
When the communication between an asynchronous future and abackground R process failed, further querying of the futurestate/results could end up in an infinite waiting loop. Now the failedcommunication error is recorded and re-signaled if any further queryingattempts.
Internal, seldom usedmyExternalIP() failed torecognize IPv4 answers from some of the lookup servers. This could inturn produce another error.
In R (>= 3.5.0), multicore futures would produce multiplewarnings originating from querying whether background processes havecompleted or not. These warnings are now suppressed.
More errors related to orchestration of futures are of classFutureError to make it easier to distinguish them fromfuture evaluation errors.
Add support for a richer set of results returned by resolvedfutures. Previously only the value of the future expression, which couldbe a captured error to be resignaled, was expected. Now aFutureResult object may be returned instead. Although notsupported in this release, this update opens up for reporting onadditional information from the evaluation of futures, e.g. capturedoutput, timing and memory benchmarks, etc. Before that can take place,existing future backend packages will have to be updatedaccordingly.
backtrace() returns only the last call that producedthe error. It is unfortunately not possible to capture the call stackthat led up to the error when evaluating a future expression.
value() forMulticoreFuture would notproduce an error when a (forked) background R workers would terminatebefore the future expression is resolved. This was a limitationinherited from theparallel package. Now an informativeFutureError message is produced.
value() forMulticoreFuture would notsignal errors unless they inherited fromsimpleError - nowit’s enough for them to inherits fromerror.
value() forClusterFuture no longerproduces aFutureEvaluationError, butFutureError, if the connection to the R worker has changed(which happens if something as drastic ascloseAllConnections() have been called.)
futureCall(..., globals = FALSE) would produce“Error: second argument must be a list”, because the explicit argumentswhere not exported. This could also happen when specifying globals byname or as a named list.
Nested futures were too conservative in requiring globalvariables to exist, even when they were false positives.
future::future_lapply() is formally deprecated.Please use the one in thefuture.apply packageinstead.
Recently introducedFutureEvaluationConditionclasses are deprecated, because they no longer serve a purpose sincefuture evaluation conditions are now signaled as-is.
future_lapply() has moved to thefuture.apply package available on CRAN.Argumentworkers of future strategies may now alsobe a function, which is called without argument when the future strategyis set up and used as is. For instance,plan(multiprocess, workers = halfCores) wherehalfCores <- function() { max(1, round(availableCores()/ 2)) }will use half of the number of available cores. This is useful whenusing nested future strategies with remote machines.
On Windows,makeClusterPSOCK(), and thereforeplan(multisession) andplan(multiprocess),will use the SSH client distributed with RStudio as a fallback ifneitherssh norplink is available on thesystemPATH.
Nowplan() makes sure thatnbrOfWorkers() will work for the new strategy. This willhelp catch mistakes such asplan(cluster, workers = cl)wherecl is a basic R list rather than acluster list early on.
Added%packages% to explicitly control packages tobe attached when a future is resolved,e.g. y %<-% { YT[2] } %packages% "data.table". Note,this is only needed in cases where the automatic identification ofglobal and package dependencies is not sufficient.
Added condition classesFutureCondition,FutureMessage,FutureWarning, andFutureError representing conditions that occur while afuture is setup, launched, queried, or retrieved. They donotrepresent conditions that occur while evaluating the future expression.For those conditions, new classesFutureEvaluationCondition,FutureEvaulationMessage,FutureEvaluationWarning, andFutureEvaluationError exists.
if (runif(1) < 1/2) x <- 0; y <- 2 * x.externalptr)can not be exported, but there are exceptions. By setting optionsfuture.globals.onReference to"warning", awarning is produced informing the user about potential problems. If"error", an error is produced. Because there might be falsepositive, the default is"ignore", which will cause abovescans to be skipped. If there are non-exportable globals and these testsare skipped, a run-time error may be produced only when the futureexpression is evaluated.The total size of global variables was overestimated, anddramatically so if defined in the global environment and there were arelarge objects there too. This would sometimes result in a false errorsaying that the total size is larger than the allowed limit.
An assignment such asx <- x + 1 where theleft-hand side (LHS)x is a global failed to identifyx as a global because the right-hand side (RHS)x would override it as a local variable. Updates to theglobals package fixed this problem.
makeClusterPSOCK(..., renice = 19) would launch eachPSOCK worker vianice +19 resulting in the error “nice:‘+19’: No such file or directory”. This bug was inherited fromparallel::makePSOCKcluster(). Now usingnice --adjustment=19 instead.
Protection against passing future objects to other futures didnot work for future strategy ‘multicore’.
future_lapply() has moved to the newfuture.apply package available on CRAN. Thefuture::future_lapply() function will soon be deprecated,then defunct, and eventually be removed from thefuturepackage. Please update your code to make use offuture.apply::future_lapply() instead.
Dropped defunct ‘eager’ and ‘lazy’ futures; use ‘sequential’instead.
Dropped defunct argumentscluster andmaxCores; useworkers instead.
In previous version of thefuture package theFutureError class was used to represent both orchestrationerrors (nowFutureError) and evaluation errors (nowFutureEvaluationError). Any usage of classFutureError for the latter type of errors is deprecated andshould be updated toFutureEvaluationError.
Nowplan() accepts also strings such as"future::cluster".
Nowbacktrace(x[[ER]]) works also fornon-environmentx:s, e.g. lists.
When measuring the size of globals by scanning their content, forcertain types of classes the inferred lengths of these objects wereincorrect causing internal subset out-of-range issues.
print() forFuture would output oneglobal per line instead of concatenating the information withcommas.
getGlobalsAndPackages().future_lapply() would give “Error in objectSize.env(x,depth = depth - 1L): object ‘nnn’ not found” when for instance ‘nnn’ ispart of an unresolved expression that is an argument value.tweak(), and henceplan(), generates amore informative error message if a non-future function is specified bymistake, e.g. callingplan(cluster) with thesurvival package attached afterfutureis equivalent to callingplan(survival::cluster) whenplan(future::cluster) was intended.nbrOfWorkers() gave an error withplan(remote). Fixed by making the ‘remote’ future inheritcluster (as it should).quit(), but that appeared to havecorrupted the main R session when running on Solaris.Formally defunct ‘eager’ and ‘lazy’ futures; use ‘sequential’instead.
Dropped previously defunct%<=% and%=>% operators.
makeClusterPSOCK() now defaults to use the WindowsPuTTY software’s SSH clientplink -ssh, ifsshis not found.
Argumenthomogeneous ofmakeNodePSOCK(), a helper function ofmakeClusterPSOCK(), will default to FALSE also if thehostname is a fully qualified domain name (FQDN), that is, it “containsperiods”. For instance,c('node1', 'node2.server.org') willusehomogeneous = TRUE for the first worker andhomogeneous = FALSE for the second.
makeClusterPSOCK() now asserts that each clusternode is functioning by retrieving and recording the node’s sessioninformation including the process ID of the corresponding Rprocess.
Nested futures sets optionmc.cores to preventspawning of recursive parallel processes by mistake. Because ‘mc.cores’controlsadditional processes, it was previously set to zero.However, since some functions such asmclapply() does notsupport that, it is now set to one instead.
makeClusterPSOCK() gained more detaileddescriptions on arguments and what their defaults are.future_lapply() with multicore / multisessionfutures, would use a suboptimal workload balancing where it split up thedata in one chunk too many. This is no longer a problem because of howargumentworkers is now defined for those type of futures(see note on top).
future_lapply(), as well as lazy multicore and lazysequential futures, did not respect optionfuture.globals.resolve, but was hardcoded to always resolveglobals (future.globals.resolve = TRUE).
When globals larger than the allowed size (optionfuture.globals.maxSize) are detected an informative errormessage is generated. Previous version introduced a bug causing theerror to produce another error.
Lazy sequential futures would produce an error when resolved ifrequired packages had been detached.
print() would not display globals gathered for lazysequential futures.
Added package tests for globals part of formulas part of otherglobals, e.g. purrr::map(x, ~ rnorm(.)), which requiresglobals (>= 0.10.0).
Now package tests withparallel::makeCluster() notonly test fortype = "PSOCK" clusters but also"FORK" (when supported).
TESTS: Cleaned up test scripts such that the overall processingtime for the tests was roughly halved, while preserving the same testcoverage.
future_lapply() is now tonotgenerate RNG seeds (future.seed = FALSE). If proper randomnumber generation is needed, usefuture.seed = TRUE. Formore details, see help page.future() andfuture_lapply() gainedargumentpackages for explicitly specifying packages to beattached when the futures are evaluated. Note that the defaultthroughout thefuture package is that all globals andall required packages are automatically identified and gathered, so inmost cases those do not have to be specified manually.
The default values for argumentsconnectTimeout andtimeout ofmakeNodePSOCK() can now becontrolled via global options.
Nowfuture_lapply() guarantees that the RNG state ofthe calling R process after returning is updated compared to what it wasbefore and in the exact same way regardless offuture.seed(except FALSE),future.scheduling and future strategy used.This is done in order to guarantee that an R script callingfuture_lapply() multiple times should be numericallyreproducible given the same initial seed.
It is now possible to specify a pre-generated sequence of.Random.seed seeds to be used for eachFUN(x[[i]], ...) call infuture_lapply(x, FUN, ...).
future_lapply() scans global variables for non-resolvedfutures (to resolve them) and calculate their total size once.Previously, each chunk (a future) would redo this.Nowfuture_lapply(X, FUN, ...) identifies globalobjects amongX,FUN and...recursively until no new globals are found. Previously, only the firstlevel of globals were scanned. This is mostly thanks to a bug fix inglobals 0.9.0.
A future that used a global objectx of a class thatoverrideslength() would produce an error iflength(x) reports more elements than what can besubsetted.
nbrOfWorkers() gave an error withplan(cluster, workers = cl) wherecl is acluster object created byparallel::makeCluster(), etc. This prevented for instancefuture_lapply() to work with such setups.
plan(cluster, workers = cl) wherecl <- makeCluster(..., type = MPI") would give aninstant error due to an invalid internal assertion.
Previously deprecated argumentsmaxCores andcluster are now defunct.
Previously deprecated assignment operators%<=%and%=>% are now defunct.
availableCores(method = "mc.cores") is now defunctin favor of"mc.cores+1".
plan(), e.g. plan(cluster) will set up workerson all cluster nodes. Previously, this only happened when the firstfuture was created.Renamed ‘eager’ futures to ‘sequential’,e.g. plan(sequential). The ‘eager’ futures will bedeprecated in an upcoming release.
Added support for controlling whether a future is resolvedeagerly or lazily when creating the future,e.g. future(..., lazy = TRUE),futureAssign(..., lazy = TRUE), andx %<-% { ... } %lazy% TRUE.
future(),futureAssign() andfutureCall() gained argumentseed, whichspecifies a L’Ecuyer-CMRG random seed to be used by the future. The seedfor future assignment can be specified via%seed%.
futureAssign() now passes all additional argumentstofuture().
Addedfuture_lapply() which supports load balancing(“chunking”) and perfect reproducibility (regardless of type of loadbalancing and how futures are resolved) via initial randomseed.
AddedavailableWorkers(). By default it returnslocalhost workers according toavailableCores(). Inaddition, it detects common HPC allocations given in environmentvariables set by the HPC scheduler.
The default forplan(cluster) is nowworkers = availableWorkers().
Nowplan() stops any clusters that were implicitlycreated. For instance, a multisession cluster created byplan(multisession) will be stopped whenplan(eager) is called.
makeClusterPSOCK() treats workers that refer to alocal machine by its local or canonical hostname as “localhost”. Thisavoids having to launch such workers over SSH, which may not besupported on all systems / compute cluster.
Optionfuture.debug = TRUE also reports on totalsize of globals identified and for cluster futures also the size of theindividual global variables exported.
Optionfuture.wait.timeout (replacesfuture.wait.times) specifies the maximum waiting time for afree workers (e.g. a core or a compute node) before generating a timeouterror.
Optionfuture.availableCores.fallback, whichdefaults to environment variableR_FUTURE_AVAILABLECORES_FALLBACK can now be used to specifythe default number of cores / workers returned byavailableCores() andavailableWorkers() whenno other settings are available. For instance, ifR_FUTURE_AVAILABLECORES_FALLBACK=1 is set system wide in anHPC environment, then all R processes that usesavailableCores() to detect how many cores can be used willrun as single-core processes. Without this fallback setting, and withoutother core-specifying settings, the default will be to use all cores onthe machine, which does not play well on multi-user systems.
plan(lazy) are now deprecated.Instead, useplan(eager) and thenf <- future(..., lazy = TRUE) orx %<-% { ... } %lazy% TRUE. The reason behind this isthat in some cases code that uses futures only works under eagerevaluation (lazy = FALSE; the default), or vice verse.By removing the “lazy” future strategy, the user can no longer overridethelazy = TRUE / FALSE that the developer is using.Creation of cluster futures (including multisession ones) wouldtime out already after 40 seconds if all workers were busy. New defaulttimeout is 30 days (optionfuture.wait.timeout).
nbrOfWorkers() gave an error forplan(cluster, workers) whereworkers was acharacter vector or acluster object of theparallel package. Because of this,future_lapply() gave an error with such setups.
availableCores(methods = "_R_CHECK_LIMIT_CORES_")would give an error if not runningR CMD check.
AddedmakeClusterPSOCK() - a version ofparallel::makePSOCKcluster() that allows for more flexiblecontrol of how PSOCK cluster workers are set up and how they arelaunched and communicated with if running on external machines.
Added genericas.cluster() for coercing objects tocluster objects to be used as inplan(cluster, workers = as.cluster(x)). Also added ac() implementation for cluster objects such that multiplecluster objects can be combined into a single one.
AddedsessionDetails() for gathering details of thecurrent R session.
plan() andplan("list") now prints moreuser-friendly output.
On Unix, internalmyInternalIP() tries morealternatives for finding the local IP number.
%<=% is deprecated. Use%<-%instead. Same for%=>%.values() for lists and list environments of futureswhere one or more of the futures resolved to NULL would give anerror.
value() forClusterFuture would givecryptic error message “Error in stop(ex) : bad error message” if thecluster worker had crashed / terminated. Now it will instead give anerror message like “Failed to retrieve the value ofClusterFuture from cluster node #1 on ‘localhost’. Thereason reported was”error reading from connection”.
Argumentuser toremote() was ignored(since 1.1.0).
workers = "localhost" they (again) use the exact same Rexecutable as the main / calling R session (in all other cases it useswhateverRscript is found in thePATH). Thiswas already indeed implemented in 1.0.1, but with the added support forreverse SSH tunnels in 1.1.0 this default behavior was lost.REMOTE CLUSTERS: It is now very simple to usecluster() andremote() to connect to remoteclusters / machines. As long as you can connect via SSH to thosemachines, it works also with these future. The new code completelyavoids incoming firewall and incoming port forwarding issues previouslyneeded. This is done by using reverse SSH tunneling. There is also noneed to worry about internal or external IP numbers.
Added optional argumentlabel to all futures,e.g. f <- future(42, label = "answer") andv %<-% { 42 } %label% "answer".
Added argumentuser tocluster() andremote().
Now allFuture classes supportsrun()for launching the future andvalue() callsrun() if the future has not been launched.
MEMORY: Nowplan(cluster, gc = TRUE) causes thebackground R session to be garbage collected immediately after the valueis collected. Since multisession and remote futures are special cases ofcluster futures, the same is true for these as well.
ROBUSTNESS: Now the default future strategy is explicitly setwhen no strategies are set, e.g. when used nested futures. Previously,only mc.cores was set so that only a single core was used, but now alsoplan("default") set.
WORKAROUND:resolved() on cluster futures wouldblock on Linux until future was resolved. This is due to a bug in R. Theworkaround is to use round the timeout (in seconds) to an integer, whichseems to always work / be respected.
Global variables part of subassignments in future expressions arerecognized and exported (iff found), e.g. x$a <- value,x[["a"]] <- value, andx[1,2,3] <- value.
Global variables part of formulae in future expressions arerecognized and exported (iff found),e.g. y ~ x | z.
As an alternative to the default automatic identification ofglobals, it is now also possible to explicitly specify them either bytheir names (as a character vector) or by their names and values (as anamed list), e.g. f <- future({ 2*a }, globals = c("a"))orf <- future({ 2*a }, globals = list(a = 42)). Forfuture assignments one can use the%globals% operator,e.g. y %<-% { 2*a } %globals% c("a").
ROBUSTNESS: For the special case where ‘remote’ futures useworkers = "localhost" they now use the exact same Rexecutable as the main / calling R session (in all other cases it useswhateverRscript is found in thePATH).
FutureError now extendssimpleError andno longer the error class of captured errors.
Sincefuture 0.13.0, a globalpkgwould be overwritten by the name of the last package attached infuture.
Futures that generatedR.oo::Exception errors, theytriggered another internal error.
Add support forremote(..., myip = "<external>"), which now queries aset of external lookup services in case one of them fails.
Addmandelbrot() function used in demo to the APIfor convenience.
ROBUSTNESS: If.future.R script, which is sourcedwhen thefuture package is attached, gives an error,then the error is ignored with a warning.
TROUBLESHOOTING: If the future requires attachment of packages,then each namespace is loaded separately and before attaching thepackage. This is done in order to see the actual error message in casethere is a problem while loading the namespace. Withrequire()/library() this error message isotherwise suppressed and replaced with a generic one.
Falsely identified global variables no longer generate an errorwhen the future is created. Instead, we leave it to R and the evaluationof the individual futures to throw an error if the a global variable istruly missing. This was done in order to automatically handle futureexpressions that use non-standard evaluation (NSE),e.g. subset(df, x < 3) wherex is falselyidentified as a global variable.
Dropped support for system environment variableR_FUTURE_GLOBALS_MAXSIZE.
DEMO: Now the Mandelbrot demo tiles a single Mandelbrot regionwith one future per tile. This better illustrates parallelism.
Documented R options used by thefuturepackage.
Custom futures based on a constructor function that is definedoutside a package gave an error.
plan("default") assumed that thefuture.plan option was a string; gave an error if it was afunction.
Various future options were not passed on to futures.
A startup.future.R script is no longer sourced ifthefuture package is attached by a futureexpression.
Added remote futures, which are cluster futures with convenientdefault arguments for simple remote access to R, e.g.plan(remote, workers = "login.my-server.org").
Now.future.R (if found in the current directory orotherwise in the user’s home directory) is sourced when thefuture package is attach (but not loaded). This helpsseparating scripts from configuration of futures.
Added support forplan(cluster, workers = c("n1", "n2", "n2", "n4")),whereworkers (also forClusterFuture()) is aset of host names passed toparallel::makeCluster(workers).It can also be the number of localhost workers.
Added command line option--parallel=<p>,which is long for-p <p>.
Now command line option-p <p> also set thedefault future strategy to multiprocessing (if p >= 2 and eagerotherwise), unless another strategy is already specified via optionfuture.plan or system environment variableR_FUTURE_PLAN.
NowavailableCores() also acknowledges environmentvariableNSLOTS set by Sun/Oracle Grid Engine(SGE).
MEMORY: Added argumentgc = FALSE to all futures.When TRUE, the garbage collector will run at the very end in the processthat evaluated the future (just before returning the value). This mayhelp lowering the overall memory footprint when running multipleparallel R processes. The user can enable this by specifyingplan(multiprocess, gc = TRUE). The developer can controlthis usingfuture(expr, gc = TRUE) orv %<-% { expr } %tweak% list(gc = TRUE).
plan(list("eager")),whereas it did work withplan("eager") andplan(list(eager)).AddednbrOfWorkers().
Added informativeprint() method for theFuture class.
values() passes arguments... tovalue() of each future.
AddedFutureError class.
maxCores andcluster toworkers. If using the old argument names a deprecationwarning will be generated, but it will still work until made defunct ina future release.resolve() for lists and environments did not workproperly when the set of futures was not resolved in order, which couldhappen with asynchronous futures.Add support toplan() for specifying differentfuture strategies for the different levels of nested futures.
Addbacktrace() for listing the trace theexpressions evaluated (the calls made) before a condition wascaught.
Add transparent futures, which are eager futures with earlysignaling of conditioned enabled and whose expression is evaluated inthe calling environment. This makes the evaluation of such futures assimilar as possible to how R evaluates expressions, which in turnsimplifies troubleshooting errors, etc.
Add support for early signaling of conditions. The default is (asbefore) to signal conditions when the value is queried. In addition,they may be signals as soon as possible, e.g. when checking whether afuture is resolved or not.
Signaling of conditions when callingvalue() is nowcontrolled by argumentsignal (previouslyonError).
NowUniprocessFuture:s captures the call stack forerrors occurring while resolving futures.
ClusterFuture() gained argumentpersistent = FALSE. Withpersistent = TRUE,any objects in the cluster R session that was created during theevaluation of a previous future is available for succeeding futures thatare evaluated in the same session. Moreover, globals are stillidentified and exported but “missing” globals will not give an error -instead it is assumed such globals are available in the environmentwhere the future is evaluated.
OVERHEAD: Utility functions exported byClusterFuture are now much smaller; previously they wouldexport all of the package environment.
f <- multicore(NA, maxCores = 2) would end up inan endless waiting loop for a free core ifavailableCores()returned one.
ClusterFuture() would ignorelocal = TRUE.
Added multiprocess futures, which are multicore futures ifsupported, otherwise multisession futures. This makes it possible to useplan(multiprocess) everywhere regardless of operatingsystem.
Future strategy functions gained class attributes such that it ispossible to test what type of future is currently used, e.g.inherits(plan(), "multicore").
ROBUSTNESS: It is only the R process that created a future thatcan resolve it. If a non-resolved future is queried by another Rprocess, then an informative error is generated explaining that this isnot possible.
ROBUSTNESS: Nowvalue() for multicore futuresdetects if the underlying forked R process was terminated beforecompleting and if so generates an informative error messages.
resolve() gained argumentrecursive.
Added optionfuture.globals.resolve for controllingwhether global variables should be resolved for futures or not. If TRUE,then globals are searched recursively for any futures and if found such“global” futures are resolved. If FALSE, global futures are not located,but if they are later trying to be resolved by the parent future, thenan informative error message is generated clarifying that only the Rprocess that created the future can resolve it. The default is currentlyFALSE.
FIX: Exports of objects available in packages already attached bythe future were still exported.
FIX: NowavailableCores() returns3L(=2L+1L) instead of2L if_R_CHECK_LIMIT_CORES_ is set.
Add multisession futures, which analogously to multicore ones,use multiple cores on the local machine with the difference that theyare evaluated in separate R session running in the background ratherthan separate forked R processes. A multisession future is a specialtype of cluster futures that do not require explicit setup of clusternodes.
Add support for cluster futures, which can make use of a clusterof nodes created byparallel::makeCluster().
AddfutureCall(), which is for futures whatdo.call() is otherwise.
Standardized how options are named,i.e. future.<option>. If you used any future optionspreviously, make sure to check they follow the above format.
globals = TRUE).Now%<=% can also assign to multi-dimensionallist environments.
Addfutures(),values() andresolved().
Addresolve() to resolve futures in lists andenvironments.
NowavailableCores() also acknowledges the number ofCPUs allotted by Slurm.
CLEANUP: Now the internal future variable created by%<=% is removed when the future variable isresolved.
futureOf(envir = x) did not work properly whenx was a list environment.ROBUSTNESS: Now values of environment variables are trimmedbefore being parsed.
ROBUSTNESS: Add reproducibility test for random number generationusing Pierre L’Ecuyer’s RNG stream regardless of how futures areevaluated, e.g. eager, lazy and multicore.
findGlobals(..., method = "ordered") inglobals (> 0.5.0) such that a global variablepreceding a local variable with the same name is properly identified andexported/frozen.Globals that were copies of package objects were not exported tothe future environments.
Thefuture package had to be attached orfuture::future() had to be imported, if%<=% was used internally in another package. Similarly,it also had to be attached if multicore futures where used.
eager() andmulticore() gained argumentglobals, whereglobals = TRUE will validatethat all global variables identified can be located already before thefuture is created. This provides the means for providing the same testson global variables with eager and multicore futures as with lazyfutures.lazy(sum(x, ...), globals = TRUE) now properly passes... from the function from which the future is setup. Ifnot called within a function or called within a function without... arguments, an informative error message is thrown.plan("default") resets to the default strategy,which is synchronous eager evaluation unless optionfuture_plan or environment variableR_FUTURE_PLAN has been set.
availableCores("mc.cores") returnsgetOption("mc.cores") + 1L, because optionmc.cores specifies “allowed number ofadditional Rprocesses” to be used in addition to the main R process.
plan(future::lazy) and similar gave errors.multicore() gained argumentmaxCores,which makes it possible to use for instanceplan(multicore, maxCores = 4L).
AddavailableMulticore() [from (in-house)async package].
demo("mandelbrot", package = "future").ROBUSTNESS:multicore() blocks until one of the CPUcores is available, iff all are currently occupied by other multicorefutures.
old <- plan(new) now returns the oldplan/strategy (was the newly set one).
Eager and lazy futures now records the result internally suchthat the expression is only evaluated once, even if their error valuesare requested multiple times.
Eager futures are always created regardless of error ornot.
AllFuture objects are environments themselves thatrecord the expression, the call environment and optionalvariables.
lazy() “freezes” global variables at the time when thefuture is created. This way the result of a lazy future is more likelyto be the same as an ‘eager’ future. This is also how globals are likelyto be handled by asynchronous futures.plan() records the call.demo("mandelbrot", package = "future"), which canbe re-used by other future packages.Addedplan().
Added eager future - useful for troubleshooting.