Movatterモバイル変換

targets 1.11.4

• Tone down progress bar output for medium-overhead scenarios.
• Speed uptar_meta() default settings fortar_read() etc. (for million-target pipelines).
• Choose the"terse" reporter by default if the callingsession is non-interactive. This will hopefully avoid problems on CRANfor packages that usetargets with the defaultsettings.
• Improve reporter deprecation messages (#1493,@dakvid).
• Clarify scope oftar_renv() (#1506,@valentingar).
• Handle errors inrstudioapi::isAvailable() (#1519,@dipterix).
• Improve error message when metadata file is corrupted (#1523,@dakvid).

targets 1.11.3

Bug fixes

• Useqmethod = "escape" to avoidhttps://github.com/Rdatatable/data.table/issues/3509 (#1480,@koefoeden).
• Ensureerror = "trim" does not hang when the erroredtarget has a long chain of reverse dependencies (#1481,@koefoeden).
• Manually remove class"rlib_error_package_not_found"from errors (#1484,@malcolmbarrett). This and #1354 areunfortunate consequences of #997.

Other changes

• CallsuppressPackageStartupMessages() once for thewhole pipeline. Repeated target-specific calls may be slow, and themessages themselves are cumbersome. This is an appropriatetradeoff.
• Ensure the progress bar from the balanced reporter does not chop upmessages fromtar_debug_instructions().
• Remove ANSI escape sequences from warnings and error messages.
• Usecli::cli_text() instead ofcli::cli_progress_output() (#1478,@dipterix).
• Minor speedups in the beginning and end oftar_make()(#1482).
• Cache_targets/objects/ time stamps only for localbuilders mentioned in the metadata, as opposed to everything in thatdirectory (#1482).
• Instrument pre-processing overhead with progress bars (#1482).

targets 1.11.2

• Documentation fix: ifformat is"file" andrepository is not"local", then the local fileis no longer deleted after upload (#1467).
• Improve legend labels in graphs.
• Repair mermaid.js graphs with disconnected edges (#1472).

targets 1.11.1

• Bugfix:rstudio_available() returnsFALSEwithout error ifrstudioapi is not installed.
• Add a new"terse" reporter, which is the"balanced" reporter without the progress bar. Make"terse" the default reporter

targets 1.11.0

Deprecated features

• Deprecate thepriority argument oftar_target(). Because of #1458, custom priorities no longerhave an effect on execution order. However, up-to-date parallelizedpipelines with 100000+ targets can now be checked around 10 timesfaster, so the tradeoff is worth it. And as a workaround, you can sendhigh-priority targets to one or more specialcrewcontrollers in a controller group (details:https://books.ropensci.org/targets/crew.html#heterogeneous-workers).

Changes to default behavior

• Keepformat = "file" files on disk even for non-localrepositories (#1467).

Changes to default settings

• Intar_option_get(), setrepository_metato"local" by default, regardless ofrepository (#1427).
• Intar_option_get(), setstorage = "worker",retrieval = "auto", andmemory = "auto" by default (#1426). Formemory,"auto" is now equivalent to"transient" most of the time, but it is equivalent to"persistent" for non-dynamic targets that other targetsdynamically branch over. Forretrieval, the"auto" setting is new. It is equivalent to"worker" for most cases, but it aligns with"main" for dynamic branches that branch over non-dynamictargets. All this is to avoid re-reading the upstream target from diskevery time a branch needs to run.
• Set the new “balanced” reporter to be the default reporter fortar_make() andtar_outdated().
• Set the defaultgarbage_collection argument oftar_option_get() to 0 (#1464).

Efficiency improvements

• Speed up checking up-to-date targets in large dynamic branchingpipelines (#1458, #1460). The speedup is over 10-fold or more in somecases.
• Maintain a persistent text connection when appending to a metadatatext file (#1415).
• Avoid superfluous garbage collection whencrewcontrollers are saturated.
• Set defaults forstorage,retrieval, andmemory that balance resource tradeoffs for the most commonpipelines (#1426).
• Garbage collection only runs intargets:::target_run()(#1464). There is no longer a separategc() call on themain process.
• Shave off overhead fromstore_sync_file_meta() in thegeneral case.

Other changes

• Upload workspaces to the cloud iftar_option_get("repository_meta") is"aws" or"gcp". Download them withtar_workspace_download() and delete them withtar_destroy(destroy = "all") ortar_destroy(destroy = "cloud").
• Deep-copy settings when resolvingformat = "auto"(#1425,@paulseamer).
• Addstore_read_path.tar_auto() (#1429,@paulseamer).
• Improve error message that explainsiteration = "group"branching problems.
• Allow more special characters in recorded warnings and errormessages.
• Callcli::style_reset() at the end of non-silentreporters (#1450,@r2evans).
• Exclude lists of target definitions from the globals in thedependency graph (#1431).
• Nomenclature change: drop the term “dynamic file” in favor of “filetarget”.
• Internally choose a default level separation in thevisNetwork graph based on the number of hierarchical levelsand the maximum number of vertices per level (#1432).
• Intar_visnetwork(), choose the colors of the edgesbased on the origin vertices, not the destination vertices (#1433).
• In the"verbose" and"timestamp"reporters, print “dispatched pattern” messages, and print the totalcomputation and storage summed over all the branches.
• Create a new"balanced" reporter with acli progress bar (#1442).
• Deprecate reporters"forecast","forecast_interactive","verbose_positives",and"timestamp_positives" (#1442).
• Ensure colors printed to the console are preserved when forwardedfrom thecallr process (#1442).
• Addtar_option_with()(https://github.com/ropensci/tarchetypes/issues/215,@noamross).
• Useprettyunits to print elapsed times and filesizes.
• Shorten and simplify thetar_make() error message.
• Minor bugfix: add a newon_worker argument totarget_run() andbuilder_unload_value() so thelatter only removes the target value if the target was actually run on aworker.

targets 1.10.1

• Restore explicit references to “self” inR6classes.
• Performcrew task retries.
• Try to handleNA buckets instore_delete_objects.tar_aws() andstore_delete_objects.tar_gcp().

targets 1.10.0

Invalidating changes

These changes invalidate certain targets in a pipeline and cause themto rerun on the nexttar_make().

• Exclude function signatures fromtar_repository_cas()output strings to reduce the size of pipeline metadata (#1390).
• Exclude function signatures fromtar_format() outputstrings to reduce the size of pipeline metadata (#1390).

Summary of performance gains

tar_make() andtar_outdated() run muchfaster in this release. Extensive profiling was done on a real-worldsimulation pipeline with 66002 up-to-date targets. Fortar_make() using all the default settings:

And fortar_outdated() using all the defaultsettings

To take advantage of these speed gains for an existing pipeline, youmay have to runtar_make() to convert the time stamps andfile sizes to a new format. This initialtar_make() isslow, but subsequenttar_make() calls should be much fasterthan before the upgrade.

Other/specific changes

• Speed uptar_make() andtar_outdated() byavoiding excessive buffering and disk writes for metadata and reporterswhen the pipeline is just skipping targets.
• Use a more lookup-efficient data structure fortar_runtime$file_info (#1398).
• Fall back on vector aggregation without names (#1401,@guglicap).
• Speed up representation of file sizes in metadata (#1408).
• Add a new"forecast_interactive" reporter totar_outdated() to choose"forecast" forinteractive sessions and"silent" for non-interactiveones.
• Add a newseconds_reporter_outdated argument totar_config_set() with a default of 1 to control the timeinterval of the reporter oftar_outdated() and otherpassive algorithm functions.
• Remove target descriptions from the default labels of graphvisualizations.

targets 1.9.1

Bug fixes

• Allow branch references to contain multi-elementpathvectors with cloud metadata (#1382,@n8layman).
• Avoid partial matches in internal code (#1384,@olivroy).
• Add error handling around calls tops::ps_disk_partitions() andps::ps_fs_mount_point().
• Do not store_targets/objects/ paths in metadata forCAS repositories (#1391).

Compatibility

• Ensure compatibility withigraph >= 2.1.2.

targets 1.9.0

Improvements

• Un-break workflows that useformat = "file_fast"(#1339,@koefoeden).
• Fix deadlock inerror = "trim" (#1340,@koefoeden).
• Remove tailored debugging message (#1341,@koefoeden).
• Store warnings while writing to storage (#1345,@Aariq).
• Allowgarbage_collection to be a non-negative integerto control the frequency of garbage collection in a performant,convenient, unified way (#1351).
• Deprecate thegarbage_collection argument oftar_make(),tar_make_future(), andtar_make_clusterm() (#1351).
• Instrumenttarget_run(),target_prepare(),andtarget_conclude() usingautometric.
• Avoid sending problematic error classes such as"vctrs_error_subscript_oob" torlang::abort()(#1354,@Jiefei-Wang).
• Reduce memory consumption by ~23% in large pipelines by avoiding theaccumulation of promise objects (#1352).
• Avoidstore_assert_format() andstore_convert_object() isstorage is"none".
• Add alist() method totar_repository_cas() to make it easier and more efficientto specify custom CAS repositories (#1366).
• Improve speed and reduce memory consumption by avoiding deep copiesof inner environments of target definition objects (#1368).
• Reduce memory consumption by storing buds and branches aslightweight references whenmemory is"transient" (#1364).
• Replace thememory class with the newlookup class.
• Implementmemory = "auto" to select transient memoryfor dynamic branches and persistent memory for other targets(#1371).
• Omit whole pattern targets from branch subpipelines when possible.Should reduce memory consumption in some cases.
• Omit whole stem targets from branch subpipelines whenretrieval is"main" and only a bud is actuallyused. The same cannot be done with branches because each branch may needto be (un)marshaled individually.
• Compress branches into references whenretrieval is"worker" and the whole pattern is part of thesubpipeline.
• Avoid duplicated branch aggregation: just send the branches over thenetwork.
• Back-compatibly switchformat = "qs" fromqs toqs2 (#1373).
• Addtar_unblock_process().

Potentially invalidatingchanges

• Add"keepNA" and"keepInteger" to.deparseOpts() (#1375). This may cause existing pipelinesto rerun, but it makes add-ons liketarchetypes::tar_map()much easier to use.

targets 1.8.0

• Wraptar_watch() UI module inbslib::page() (#1302,@kwbyron-lilly).
• Removecallr_function intar_make_as_job()argument list.
• Ensurestorage = "worker" is respected when the processof storing an object generates an error (#1304,@multimeric).
• Default to the_targets.R pattern intar_branches() (#1306,@multimeric,@mattwarkentin).
• Remove superfluous functions and globals from metadata withtar_prune() (#1312,@benzipperer).
• Change the defaultworkspace_on_error option toTRUE (#1310,@hadley).
• Enhance and organize theerror = "stop" errormessage.
• Avoid saving a file in_targets/objects forerror = "null". Instead, switch to a special"null" storage format class iferror is"null" the target throws an error. This should allow usersto more freely create new formats withtar_format() withoutworrying about how to handleNULL objects created byerror = "null".
• Implementformat = "auto" (#1311,@hadley).
• Replacepingr dependency withbase::socketConnection() for local URL utilities (#1317,#1318,@Adafede).
• Implementtar_repository_cas(),tar_repository_cas_local(), andtar_repository_cas_local_gc() for content-addressablestorage (#1232, #1314,@noamross).
• Addtar_format_get() to make implementing CAS systemseasier.
• Implementerror = "trim" intar_target()andtar_option_set() (#1310, #1311,@hadley).
• Use the file system type to decide whether to trust time stamps(#1315,@hadley,@gaborcsardi).
• Deprecateformat = "file_fast" in favor of the above(#1315).
• Deprecatetrust_object_timestamps in favor of the moreunifiedtrust_timestamps intar_option_set()(#1315).
• Print storage size of each target in verbose reporters (#1337,@psychelzh).
• Combine help files oftar_target() andtar_target_raw(). Same withtar_load() andtar_load_raw().
• Add asubstitute argument totar_format()to make it easier to write custom storage formats withoutmetaprogramming.

targets 1.7.1

• Usebslib intar_watch().
• Speed uptarget_upstream_edges() andpipeline_upstream_edges() by avoiding data frames until thelast minute (17% speedup for certain kinds of large pipelines).
• Automatically setas_job toFALSE intar_make() ifrstudioapi and/or RStudio is notavailable.

targets 1.7.0

Invalidating changes

• Usesecretbase::siphash13() instead ofdigest(algo = "xxhash64", serializationVersion = 3) sohashes of in-memory objects no longer depend on serialization version 3headers (#1244,@shikokuchuo). Unfortunately, pipelinesbuilt with earlier versions oftargets will need torerun.

Other improvements

• Ensure patterns marshal properly (#1266, #1264,https://github.com/njtierney/geotargets/issues/52,@Aariq,@njtierney).
• Inform and prompt the user when the pipeline was built with an oldversion oftargets and changes to the package will causethe current work to rerun (#1244). For thetar_make*()functions,utils::menu() prompts the user to give people achance to downgrade if necessary.
• For type safety in the internal database class, read all columns ascharacter vectors indata.table::fread(), then convert themto the correct types afterwards.
• Add a newtar_resources_custom_format() function whichcan pass environment variables to customize the behavior of customtar_format() storage formats (#1263, #1232,@Aariq,@noamross).
• Only marshal dependencies if actually sending the target to aparallel worker.

targets 1.6.0

• Modernizeextras intar_renv().
• tar_target() gains adescription argumentfor free-form text describing what the target is about (#1230, #1235,#1236,@tjmahr).
• tar_visnetwork(),tar_glimpse(),tar_network(),tar_mermaid(), andtar_manifest() now optionally show target descriptions(#1230, #1235, #1236,@tjmahr).
• tar_described_as() is a new wrapper aroundtidyselect::any_of() to select specific subsets of targetsbased on the description rather than the name (#1136, #1196,@noamross,@mattmoo).
• Fix the documentation of thenames argument (nudgeusers towardtidyselect expressions).
• Make assertions on the pipeline process more robust (to check if twoprocesses are trying to access the same data store).

targets 1.5.1

• Avoidarrow-related CRAN check NOTE.
• use_targets() only writes the_targets.Rscript. Therun.sh andrun.R scripts aresuperseded by theas_job argument oftar_make(). Users not using the RStudio IDE can calltar_make() withcallr_function = callr::r_bgto run the pipeline as a background process.tar_make_clustermq() andtar_make_future() aresuperseded in favortar_make(use_crew = TRUE), so templatefiles are no longer written for the former automatically.

targets 1.5.0

Invalidating changes

Because of the changes below, upgrading to this version oftargets will unavoidably invalidate previously builttargets in existing pipelines. Your pipeline code should still work, butany targets you ran before will most likely need to rerun after theupgrade.

• Intar_seed_create(), usesecretbase::sha3(x = TARGET_NAME, bits = 32L, convert = NA)to generate target seeds that are more resistant to overlapping RNGstreams (#1139,@shikokuchuo). The previous approachused a less rigorous combination ofdigest::digest(algo = "sha512") anddigets::digest2int().

Other improvements

• Update the documentation of thedeployment argument oftar_target() to reflect the advent ofcrew(#1208,@psychelzh).
• Unsetcli.num_colors on exit intar_error() andtar_warning() (#1210,@dipterix).
• Do not try to accessseconds_timeout if thecrew controller is actually a controller group (#1207,https://github.com/wlandau/crew.cluster/discussions/35,@stemangiola,@drejom).
• tar_make() gains anas_job argument tooptionally run atargets pipeline as an RStudio job.
• Bump requiredigraph version to 2.0.0 becauseigraph::get.edgelist() was deprecated in favor ofigraph::as_edgelist().
• Do not dispatch targets to backloggedcrew controllers(or controller groups) (#1220). Use the newpush_backlog()andpop_backlog()crew methods to make thissmooth.
• Make the debugger message more generic (#1223,@eliocamp).
• Throw an early and informative error fromtar_make() ifthere is already atargets pipeline running on a localprocess on the same local data store. The local process is detectedusing the process ID and time stamp fromtar_process()(with a 1.01-second tolerance for the time stamp).
• Removepkgload::load_all() warning (#1218). Tried using.__DEVTOOLS__ but it interferes with reversedependencies.
• Add documentation and an assertion intar_target_raw()to let users know thatiteration = "group" is invalid fordynamic targets (ones withpattern = map(...) etc.; #1226,@bmfazio).

targets 1.4.1

• Print “errored pipeline” when at least one target errors.
• Bump minimumclustermq version to 0.9.2.
• Repair thetar_debug_instructions() tips for whencommands are long.
• Do not look for dependencies of primitive functions (#1200,@smwindecker,@joelnitta).

targets 1.4.0

Invalidating changes

Because of the changes below, upgrading to this version oftargets will unavoidably invalidate previously builttargets in existing pipelines. Your pipeline code should still work, butany targets you ran before will most likely need to rerun after theupgrade.

• Use SHA512 during the creation of target-specific pseudo-randomnumber generator seeds (#1139). This change decreases the risk ofoverlapping/correlated random number generator streams. See the “RNGoverlap” section of thetar_seed_create() help file fordetails and justification. Unfortunately, this change will invalidateall currently built targets because the seeds will be different. Toavoid rerunning your whole pipeline, setcue = tar_cue(seed = FALSE) intar_target().
• For cloud storage: instead of the hash of the local file, use theETag for AWS S3 targets and the MD5 hash for GCP GCS targets (#1172).Sanitize withtargets:::digest_chr64() in both cases beforestoring the result in the metadata.
• For a cloud target to be truly up to date, the hash in the metadatanow needs to match thecurrent object in the bucket, not theversion recorded in the metadata (#1172). In other words,targets now tries to ensure that the up-to-date dataobjects in the cloud are in their newest versions. So if you roll backthe metadata to an older version, you will still be able to accesshistorical data versions with e.g. tar_read(), but thepipeline will no longer be up to date.

Other changes to seeds

• Add a new exported functiontar_seed_create() whichcreates target-specific pseudo-random number generator seeds.
• Add an “RNG overlap” section in thetar_seed_create()help file to justify and defend howtargets andtarchetypes approach pseudo-random numbers.
• Add functiontar_seed_set() which sets a seed and setsall the RNG algorithms to their defaults in the R installation of theuser. Each target now usestar_seed_set() function to setits seed before running its R command (#1139).
• Deprecatetar_seed() in favor of the newtar_seed_get() function.

Other cloud storageimprovements

• For all cloud targets, check hashes in batched LIST requests insteadof individual HEAD requests (#1172). Dramatically speeds up the processof checking if cloud targets are up to date.
• For AWS S3 targets,tar_delete(),tar_destroy(), andtar_prune() now useefficient batched calls todelete_objects() instead ofcostly individual calls todelete_object() (#1171).
• Add a newverbose argument totar_delete(),tar_destroy(), andtar_prune().
• Add a newbatch_size argument totar_delete(),tar_destroy(), andtar_prune().
• Add new argumentspage_size andverbose totar_resources_aws() (#1172).
• Add a newtar_unversion() function to remove versionIDs from the metadata of cloud targets. This makes it easier to interactwith just the current version of each target, as opposed to the versionID recorded in the local metadata.

Other improvements

• Migrate to the changes inclustermq 0.9.0 (@mschubert).
• In progress statuses, change “started” to “dispatched” and change“built” to “completed” (#1192).
• Deprecatetar_started() in favor oftar_dispatched() (#1192).
• Deprecatetar_built() in favor oftar_completed() (#1192).
• Console messages from reporters say “dispatched” and “completed”instead of “started” and “built” (#1192).
• Thecrew scheduling algorithm no longer waits onsaturated controllers, and targets that are ready are greedilydispatched tocrew even if all workers are busy (#1182,#1192). To appropriately set expectations for users, reporters print“dispatched (pending)” instead of “dispatched” if the task load isbacklogged at the moment.
• In thecrew scheduling algorithm, waiting for tasks isnow a truly event-driven process and consumes 5-10x less CPU resources(#1183). Only the auto-scaling of workers uses polling (with aninexpensive default polling interval of 0.5 seconds, configurablethroughseconds_interval in the controller).
• Simplify stored target tracebacks.
• Print the traceback on error.

targets 1.3.2

• Try to fix function help files for CRAN.

targets 1.3.1

• Addtar_config_projects() andtar_config_yaml() (#1153,@psychelzh).
• Apply error modes tobuilder_wait_correct_hash() intarget_conclude.tar_builder() (#1154,@gadenbuie).
• Remove duplicated error message frombuilder_error_null().
• Allowtar_meta_upload() andtar_meta_download() to avoid errors if one or more metadatafiles do not exist. Add a new argumentstrict to controlerror behavior.
• Add new argumentsmeta,progress,process, andcrew to control individualmetadata files intar_meta_upload(),tar_meta_download(),tar_meta_sync(), andtar_meta_delete().
• Avoid newly deprecated arguments and functions increw0.5.0.9003 (https://github.com/wlnadau/crew/issues/131).
• Allowtar_read() etc. inside a pipeline whenever ituses a different data store (#1158,@MilesMcBain).
• Setseed = FALSE infuture::future()(#1166,@svraka).
• Add a newphysics argument totar_visnetwork() andtar_glimpse() (#925,@Bdblodgett-usgs).

targets 1.3.0

Invalidating changes

Because of these changes, upgrading to this version oftargets will unavoidably invalidate previously builttargets in existing pipelines. Your pipeline code should still work, butany targets you ran before will most likely need to rerun after theupgrade.

• In thehash_deps() method of the metadata class,exclude symbols which are not actually dependencies, rather than justgiving them empty strings. This change decouples the dependency hashfrom the hash of the target’s command (#1108).

Cloud metadata

• Continuously upload metadata files to the cloud duringtar_make(),tar_make_clustermq(), andtar_make_future() (#1109). Upload them to the repositoryspecified in therepository_metatar_option_set() option, and use the bucket and prefix setin theresourcestar_option_set() option.repository_meta defaults to the existingrepositorytar_option_set() option.
• Add new functionstar_meta_download(),tar_meta_upload(),tar_meta_sync(), andtar_meta_delete() to directly manage cloud metadata outsidethe pipeline (#1109).

Other changes

• Fix solution of #1103 so the copy fallback actually runs (@jds485, #1102, #1103).
• Switch back totempdir() for #1103.
• Movepath_scratch_dir_network() tofile.path(tempdir(), "targets") and make suretar_destroy("all") andtar_destroy("cloud")delete it.
• Displaytar_mermaid() subgraphs with transparent fillsand black borders.
• Allowdatabase$get_data() to work with listcolumns.
• Disallow functions that access the local data store (includingmetadata) from inside a target while the pipeline is running (#1055,#1063). The only exception to this is local file targets such astarchetypes literate programming target factories liketar_render() andtar_quarto().
• In thehash_deps() method of the metadata class, use anew customsort_chr() function which temporarily sets theLC_COLLATE locale to"C" for sorting. Thisensures lexicographic comparisons are consistent across platforms(#1108).
• Intar_source(), use thefile argument andkeep.source = TRUE to help with interactive debugging(#1120).
• Deprecatedseconds_interval intar_config_get(),tar_make(),tar_make_clustermq() andtar_make_future().Replace it withseconds_meta (to control how often metadatagets saved) andseconds_reporter (to control how often toprint messages to the R console) (#1119).
• Respectseconds_meta andseconds_reporterfor writing metadata and console messages even for currently buildingtargets (#1055).
• Retry all cloud REST API calls with HTTP error codes (429, 500-599)with the exponential backoff algorithm fromgoogleAuthR(#1112).
• Forformat = "url", only retry on the HTTP error codesabove.
• Make cloud temp file instances unique in order to avoid fileconflicts with the same target.
• Un-deprecateseconds_interval andseconds_timeout fromtar_resources_url(), andimplementmax_tries arguments intar_resources_aws() andtar_resources_gcp()(#1127).
• Usefile andkeep.source inparse() incallr utils and targetMarkdown.
• Automatically convert"file_fast" format to"file" format for cloud targets.
• Intar_prune() andtar_delete(), do nottry to delete pattern targets which have no cloud storage.
• Add new argumentsseconds_timeout,close_connection,s3_force_path_style totar_resources_aws() to support the analogous arguments inpaws.storage::s3() (#1134,@snowpong).

targets 1.2.2

• Fix a documentation issue for CRAN.

targets 1.2.1

• Addtar_prune_list() (#1090,@mglev1n).
• Wrapfile.rename() intryCatch() and fallback on a copy-then-remove workaround (@jds485, #1102, #1103).
• Stage temporary cloud upload/download files intools::R_user_dir(package = "targets", which = "cache")instead oftempdir().tar_destroy(destroy = "cloud") andtar_destroy(destroy = "all") remove any leftover files fromfailed uploads/downloads (@jds485, #1102, #1103).
• Usepaws.storage instead of all ofpaws.

targets 1.2.0

crew integration

• Do not assume S3 classes when validatingcrewcontrollers.
• Suggest a crew controller in the_targets.R file fromuse_targets().
• Maketar_crew() compatible withcrew >=0.3.0.
• Rename argumentterminate toterminate_controller intar_make().
• Add argumentuse_crew intar_make() andadd an option intar_config_set() to make itconfigurable.
• Write progress data and metadata intarget_prepare().

Other improvements

• Allow users to set the defaultlabel andlevel_separation arguments throughtar_config_set() (#1085,@Moohan).

targets 1.1.3

• Decide onnanonext usage intime_seconds_local() at runtime and not installation time.That way, ifnanonext is removed aftertargetsis installed, functions intargets still work. Fixes theCRAN issues seen intarchetypes,jagstargets,andgittargets.

targets 1.1.2

• Removecrew-related startup messages.

targets 1.1.1

• Pre-computecli colors and bullets to improveperformance in RStudio.
• UsepackageStartupMessage() for package startupmessages.

targets 1.1.0

Bug fixes

• Send targets to the appropriate controller in a controller groupwhencrew is used.

General improvements

• Callgc() more appropriately whengarbage_collection isTRUE intar_target().
• Addgarbage_collection arguments totar_make(),tar_make_clustermq(), andtar_make_future() to add optional garbage collection beforetargets are sent to workers. This is different and independent from thegarbage_collection argument oftar_target().In high-performance computing scenarios, the former controls whathappens on the main controlling process, whereas the latter controlswhat happens on the worker.
• Addgarbage_collection andseconds_interval arguments totar_make(),tar_make_clustermq(),tar_make_future(), andtar_config_set().
• Downsize thetar_runtime object.
• Remove the 100 Kb file size cutoff for determining whether to trustthe file timestamp or recompute the hash when checking if a file is upto date (#1062). Instate the"file_fast" format and thetrust_object_timestamps option intar_option_set() as safer alternatives.
• Consolidate store constructors.
• Allowcrew controller groups (#1065,@mglev1n).
• Expose more exponential backoff configuration parameters throughtar_backoff(). Thebackoff argument oftar_option_set() now accepts output fromtar_backoff(), and supplying a numeric is deprecated.
• Fix the exponential backoff rules in thecrewscheduling algorithm.
• Implementtar_resources_network() to configure retriesand timeouts for internal HTTP/HTTPS requests in specialized targetswithformat = "url",repository = "aws", andrepository = "gcp". Also applies to syncing target filesacross network file systems in the case ofstorage = "worker" orformat = "file", whichpreviously had a hard-codedseconds_interval = 0.1 andseconds_timeout = 60.
• Deprecateseconds_interval andseconds_timeout intar_resources_url() infavor of the new equivalent arguments oftar_resources_network()
• Safely withhold a target from itscrew controller whenthe controller is saturated (#1074,@mglev1n).
• Use exponential backoff when appending a target back to the queue inthe case of a saturatedcrew controller.
• Use native retries inpaws.common (@DyfanJones).

Speedups

• Cache info about all of_targets/objects/ intar_callr_inner_try() and update the cache as targets aresaved to_targets/objects/ to avoid the overhead ofrepeated calls tofile.exists() andfile.info() (#1056).
• Trust the timestamps by default when checking whether files in_targets/objects/ are up to date (#1062).tar_option_set(trust_object_timestamps = FALSE) ignores thetimestamps and recomputes the hashes.
• Write to_targets/meta/meta and_targets/meta/progress in timed batches instead of line byline (#1055).
• Reporters now print progress messages in timed batches instead ofline by line (#1055).
• The summary and forecast reporters are much faster because theyavoid going through data frames.
• Avoidtempfile() when working with the scratchdirectory.
• Usenanonext::mclock() instead ofproc.time() when there is no risk of forked processes.
• Replacewithr with slightly faster/leaner base Ralternatives.
• Efficiently catch changes to the working directory instead ofoverburdening the pipeline with calls tosetwd()(#1057).
• Invoketar_options methods in the internals instead oftar_option_get().
• Avoidgsub() instore_init().
• Avoid repeated calls tometa$get_record() inbuilder_should_run().
• Mock the store object when creating a record from a metadatarow.
• Avoidcli::col_none() to reduce the number of ANSIcharacters printed to the R console.

targets 1.0.0

targets is moving to version 1.0.0 because it issignificantly more mature than previous versions. Specifically,

1. tar_make() now integrates withcrew, whichwill significantly improve the waytargets doeshigh-performance computing going forward.
2. All other functionality intargets has stabilized.There is still room for smaller new features, but none as large ascrew integration, none that will fundamentally change howthe package operates.

Major improvements

• Support distributed computing through thecrew packageintar_make() (#753).crew itself is still inits early stages and currently lacks the launcher plugins to match theclustermq andfuture backends, but long-term,crew will be the predominant high-performance computingbackend.

Minor improvements

• Add a newstore_copy_object() to the store class toenable"fst_dt" and other formats to make deep copies whenneeded (#1041,@MilesMcBain).
• Add a newcopy argument to allowtar_format() formats to set thestore_copy_object() method (#1041,@MilesMcBain).
• Shorten the output string returned bytar_format() whendefault methods are used.
• Add achange_directory argument totar_source() (#1040,@dipterix).
• Informat = "url" targets, implement retries andtimeouts when connecting to URLs. The default timeout is 10 seconds, andthe default retry interval is 1 second. Both are configurable viatar_resources_url() (#1048).
• Useparallelly::freePort() intar_random_port().
• Rename a target and a function in thetar_script()example pipeline (#1033,@b-rodrigues).
• Edit the description.

targets 0.14.3

• Handle encoding errors while trying to process error and warningmessages (#1019,@adrian-quintario).
• Fix S3 generic/method consistency.

targets 0.14.2

• Forward user-level custom error conditions to the top of thepipeline (#997,@alexverse).
• Link to the help page of the manual.

targets 0.14.1

• Fix the command inserted for debug mode (#975).
• Set empty chunk options to ensure Target Markdown compatibility withthe special “setup” chunk (#973,@KaiAragaki).
• Only store the first 50 warnings in the metadata, and cap the textof the warning messages at 2048 characters (#983,@thejokenott).
• Enhance thetar_destroy() help file (#988,@Sage0614).
• Implementdestroy = "user" intar_destroy().

targets 0.14.0

• Move#!/bin/sh line to the top of SLURMclustermq template file (#944, #955,@GiuseppeTT).
• Add new functiontar_path_script().
• Renametar_store() totar_path_store()with deprecation.
• Renametar_path() totar_path_target()with deprecation.
• Add new functiontar_path_script_support().
• Make Target Markdown target scripts dynamically locate their supportscripts so the appropriate scripts can be found even when they aregenerated from one directory and sourced from another (#953, #957,@TylerGrantSmith).
• Allow user-side control of the seeds at the pipeline level.tar_option_set() now supports aseed argument,and target-specific seeds are determined bytar_option_get("seed") and the target name.tar_option_set(seed = NA) disables seed-setting behaviorbut forcibly invalidates all the affected targets except whenseed isFALSE in the target’star_cue() (#882,@sworland-thyme,@joelnitta).
• Implement aseed argument intar_cue() tocontrol whether targets update in response to changing orNA seeds (#882,@sworland-thyme,@joelnitta).
• Reduce the number of per-target AWS/GCP storage API calls.Previously there were 3 API calls per target, including 2 HEAD requests.Now there is just 1 for a typical target (unless dependencies have to bedownloaded). Relies on S3 strong read-after-write consistency(#958).
• Update thetar_github_actions() workflow file to use@v2 (#960,@kulinar).
• Print helpful hints while debugging a target interactively(#961).
• Only attempt to debug a target whencallr_function isNULL (#961).
• Make formats"feather","parquet","file", and"url" work witherror = "null" (#969).
• Declare formats"keras" and"torch"superseded bytar_format(). Documented in thetar_target() help file.
• Declare formats"keras" and"torch"incompatible witherror = "null". Documented in thetar_target() help file and in a warning thrown bytar_target() viatar_target_raw().
• Add aconvert argument totar_format() toallow customstore_convert_object() methods (#970).

targets 0.13.5

• Useany_of() instead ofall_of() in teststo ensure compatibility withtidyselect 1.1.2.9000 (#928,@hadley).
• Make therun.R fromuse_targets()executable (#929,@petrbouchal).
• Add#!/usr/bin/env Rscript to the top ofrun.R fromuse_targets() (#929,@petrbouchal).

targets 0.13.4

• Implement custom alternative toskip_on_cran() to avoidhttps://github.com/r-lib/testthat/issues/1470#issuecomment-1248145555.
• Skip more tests on CRAN.

targets 0.13.3

Enhancements

• Print “no targets found” when there are no targets in the pipelineto check or build, or if thenames argument oftar_make() does not identify any such targets in thepipeline (#923,@llrs).
• Ignore.packageName,.__NAMESPACE__., and.__S3MethodsTable__. when importing objects from packageswith theimports option oftar_option_set().
• Import datasets from packages in theimports option oftar_option_set() (#926,@joelnitta).
• Print target-specific elapsed runtimes in the verbose and timestampreporters.
• Improve error messages in functions liketar_read() andtar_load() when the data store is missing.

targets 0.13.2

Bug fixes

• Do not incorrectly reference feather resources for parquetstorage.

Enhancements

• Simplify and improve error handling.
• In thecommand column oftar_manifest()output, separate lines with “” instead of “\n” so the text output isstraightforward to work with.
• Add adrop_missing argument totar_manifest() to hide/show columns with allNA values.
• Do not set Parquet version.

targets 0.13.1

• Fix reverse dependency checks.

targets 0.13.0

Bug fixes

• Do not bootstrap the junction of a stem unless the target isbranched over (#858,@dipterix).
• For non-“file” AWS targets, immediately delete the scratch fileafter the target is uploaded (#889,@stuvet).

New features

• Allow extra arguments topaws functions via... intar_resources_aws() (#855,@michkam89).
• Addtar_source() to conveniently source R scripts(e.g. in_targets.R).

Enhancements

• Color ordinarytargets messages the default themecolor, and color warnings and errors red (#856,@gorkang).
• Automatically supply job names in the scripts generated byuse_targets().
• Inherit resources one-by-one in nested fashion fromtar_option_get("resources") (#892). See the revised"Resources" section of thetar_resources()help file for details.

targets 0.12.1

New features

• Add argumentslegend andcolor to furtherconfiguretar_mermaid() (#848,@noamross).
• For HPC schedulers like SLURM and SGE,use_targets()now creates ajob.sh script to run the pipeline as acluster job (#839).

Enhancements

• Use lapply() to source scripts inuse_targets(). Avoidsdefining a global variable for the file.
• Recursively find scripts to source in theuse_targets()_targets.R file.
• Refactor error printing.

targets 0.12.0

Bug fixes

• Fixtar_mermaid() graph ordering.
• Hash the node names and quote the label names oftar_mermaid() graphs to avoid JavaScript keywords.
• Remove superfluous line breaks in the node labels of graphvisuals.
• Fix metadata migration to version >= 0.10.0 (#812,@tjmahr).
• data.table::fread() with encoding equal togetOption("encoding") if available (#814,@svraka). Only works withUTF-8 and latin1 because that is whatdata.tablesupports.
• Force add files in GitHub Actions workflow job (#815,@tarensanders).

New features

• use_targets() now writes a_targets.R filetailored to the project in the current working directory (#639,@noamross).
• Move the olduse_targets() touse_targets_rmd().

Enhancements

• Load packages when loading data for downstream targets in thepipeline (#713).
• Handle edge case whengetOption("OutDec") is not"." to prevent time stamps from being corrupted (#433,@jarauh).
• Added helper functiontar_load_everything() to quicklyload all targets (#823,@malcolmbarrett)

targets 0.11.0

Bug fixes

• Print out the relevant target names if targets have conflictingnames.
• Catch all the target warnings instead of just reporting the lastone.
• Allow 200 group URL status codes instead of just 200 (#797,@petrbouchal).

New features

• Add Google Cloud Storage viatar_target(..., repository = "gcp") (#720,@markedmondson1234). Specialthanks to@markedmondson1234 for the cloudstorage utilities inR/utils_gcp.R
• mermaid.js static graphs withtar_mermaid() (#775,@yonicd).
• Implementtar_target(..., error = "null")to allowerrored targets to returnNULL and continue (#807,@zoews). Errors are stillregistered, those targets are not up to date, and downstream targetshave an easier time continuing on.
• Implementtar_assert_finite().
• tar_destroy(),tar_delete(), andtar_prune() now attempt to delete cloud data for theappropriate targets (#799). In addition,tar_exist_objects() andtar_objects() nowreport about target data in the cloud when applicable. Add a newcloud argument to each function to optionally suppress thisnew behavior.
• Add azoom_speed argument totar_visnetwork() andtar_glimpse() (#749,@dipterix).
• Report the total runtime of the pipeline in the"verbose","verbose_positives","timestamp", and"timesamp_positives"reporters.

Enhancements

• Allow target name character strings to have attributes (#758,@psanker).
• Sort metadata rows when the pipeline finishes so thatversion-controlling the metadata is easier (#766,@jameelalsalam).

Deprecations

• Deprecate the"aws_*" storage format values in favor ofa newrepository argument (#803). In other words,tar_target(..., format = "aws_qs") is nowtar_target(..., format = "qs", repository = "aws"). Andinternally, storage classes with multiple inheritance are createddynamically as opposed to having hard-coded source files. All this pavesthe way to add new cloud storage platforms without combinatorialchaos.

targets 0.10.0

Bug fixes

• Add class"tar_nonexportable" toformat = "aws_keras" andformat = "aws_torch"stores.
• Export S3 methods of generictar_make_interactive_load_target().

New features

• Allow entirely custom storage formats throughtar_target(format = tar_format(...)) (#736).
• Add a new functiontar_call() to return thetargets function currently running (from_targets.R or a target).
• Add a new functiontar_active() to tell whether thepipeline is currently running. Detects if it is called fromtar_make() or similar function.

Enhancements

• AddSys.getenv("TAR_PROJECT") to the output oftar_envvars().
• Set thestore field oftar_runtime priorto sourcing_targets.R sotar_store() works intarget scripts.
• Explicitly export all the environment variables fromtar_envvars() to targets run on parallel workers.
• Allowformat = "file" targets to returncharacter(0) (#728,@programLyrique).
• Automatically remove non-targets from the target list and improvetarget list error messages (#731,@billdenney).
• Link to resources on deploying to RStudio Connect (#745,@ian-flores).

targets 0.9.1

• Mask pointers in function dependencies (#721,@matthiaskaeding)

targets 0.9.0

Highlights

• Track the version ID of AWS S3-backed targets if the bucket isversion-enabled (#711). If you put your targets in AWS and the metadataand code under version control, you cangit checkout adifferent branch of your code and all you targets will stay up todate.
• Refactor the AWS path format internally. It now consists ofarbitrarily extensible key-value pairs so more AWS S3 functionality maybe added more seamlessly going forward (#711).
• Switch the AWS S3 backend topaws (#711).

New features

• Add aregion argument totar_resources_aws() to allow the user to explicitly declarea region for each AWS S3 buckets (@caewok, #681). Different buckets can nowhave different regions. This feature required modifying the metadatapath for AWS storage formats. Before, the first element of the path wassimply the bucket name. Now, it is internally formatted like"bucket=BUCKET:region=REGION", whereBUCKET isthe user-supplied bucket name andREGION is theuser-supplied region name. The newtargets isback-compatible with the old metadata format, but if you run thepipeline withtargets >= 0.8.1.9000 and then downgradetotargets <= 0.8.1, any AWS targets will break.
• Add new reporterstimestamp_positives" and"verbose_positives" that omit messages for skipped targets(@psanker,#683).
• Implementtar_assert_file().
• Implementtar_reprex() for creating easier reproducibleexamples of pipelines.
• Implementtar_store() to get the path to the store ofthe currently running pipeline (#714,@MilesMcBain).
• Automatically write a_targets/user/ folder toencouragegittargets users to put custom files there fordata version control.

Bug fixes

• Make suretar_path() uses the current store path of thecurrently running pipeline instead oftar_config_get("store") (#714,@MilesMcBain).

Enhancements

• Refactor the automatic.gitignore file inside the datastore to allow the metadata to be committed to version control moreeasily (#685, #711).
• Document target name requirements intar_target() andtar_target_raw() (@tjmahr, #679).
• Catch and relay any the error if a target cannot be checked intarget_should_run.tar_builder(). These kinds of errorssometimes come up with AWS storage.
• Fix the documentation of the reporters.
• Only write_targets/.gitignore for new data stores sothe user can delete the.gitignore file without itmysteriously reappearing (#685).

targets 0.8.1

New features

• Add argumentsstrict andsilent to allowtar_load() andtar_load_raw() to bypasstargets that cannot be loaded.

Enhancements

• Improvetidyselect docs intar_make()(#640,@dewoller).
• Use namespaced call totar_dir() intar_test() (#642,@billdenney).
• Improvetar_assert_target_list() error message (@kkami1115, #654).
• Throw an informative error if a target name starts with a dot (@dipterix, #662).
• Improve help files oftar_destroy() and related cleanupfunctions (@billdenney, #675).

targets 0.8.0

Bug fixes

• Hash the correct files intar_target(target_name, ..., format = "aws_file").Previously,_targets/objects/target_name was also hashed ifit existed.

New features

• Implement a newtar_config_unset() function to deleteone or more configuration settings from the YAML configurationfile.
• Implement theTAR_CONFIG environment variable to setthe default file path of the YAML configuration file with projectsettings (#622,@yyzeng,@atusy,@nsheff,@wdkrnls). IfTAR_CONFIG is notset, the file path is still_targets.yaml.
• Restructure the YAML configuration file format to handleconfiguration information for multiple projects (using theconfig package) and support theTAR_PROJECTenvironment variable to select the current active project for a given Rsession. The old single-project format is gracefully deprecated (#622,@yyzeng,@atusy,@nsheff,@wdkrnls).
• Implementretrieval = "none" andstorage = "none" to anticipate loading/saving targets fromother languages, e.g. Julia (@MilesMcBain).
• Add a newtar_definition() function to get the targetdefinition object of the current target while that target is running ina pipeline.
• If called inside an AWS target,tar_path() now returnsthe path to the staging file instead of_targets/objects/target_name. This ensures you can stillwrite totar_path() instorage = "none"targets and the package will automatically hash the right file andupload it to the cloud. (This behavior does not apply to formats"file" and"aws_file", where it is nevernecessary to setstorage = "none".)

Enhancements

• Useeval(parse(text = ...), envir = tar_option_set("envir")instead ofsource() in the_targets.R file forTarget Markdown.
• Allow feather and parquet formats to accept objects of classRecordBatch andTable (@MilesMcBain).
• Letknitr load the Target Markdown engine (#469,@nviets,@yihui). Minimumknitr version is now1.34.
• In thetar_resources_future() help file, encourage theuse ofplan to specify resources.

targets 0.7.0

Bug fixes

• Ensureerror = "continue" does not cause erroredtargets to haveNULL values.
• Relay output and messages in Target Markdown interactive mode (usingthe R/defaultknitr engine).

New features

• Expose thepoll_connection,stdout, andstderr arguments ofcallr::r_bg() intar_watch() (@mpadge).
• Add new helper functions to list targets in each progress category:tar_started(),tar_skipped(),tar_built(),tar_canceled(), andtar_errored().
• Add new helper functionstar_interactive(),tar_noninteractive(), andtar_toggle() todifferentially suppress code in non-interactive and interactive mode inTarget Markdown (#607,@33Vito).

Enhancements

• Handlefuture errors within targets (#570,@stuvet).
• Handle storage errors within targets (#571,@stuvet).
• In Target Markdown in non-interactive mode, suppress messages if themessageknitr chunk option isFALSE (#574,@jmbuhr).
• In Target Markdown, iftar_interactive is not set,choose interactive vs non-interactive mode based onisTRUE(getOption("knitr.in.progress")) instead ofinteractive().
• Convert errors loading dependencies into errors running targets(@stuvet).

targets 0.6.0

Bug fixes

• Allowtar_poll() to lose and then regain connection tothe progress file.
• Make sure changes to thetar_group column ofiteration = "group" data frames do not invalidate slices(#507,@lindsayplatt).

New features

• In Target Markdown, add a newtar_interactive globaloption to select interactive mode or non-interactive mode (#469).
• Highlight a graph neighborhood when the user clicks a node. Controlthe neighborhood degree with new argumentsdegree_from anddegree_to oftar_visnetwork() andtar_glimpse() (#474,@rgayler).
• Make the target script path configurable intar_config_set() (#476).
• Add atar_script chunk option in Target Markdown tocontrol where the{targets} language engine writes thetarget script and helper scripts (#478).
• Add new argumentsscript andstore tochoose custom paths to the target script file and data store forindividual function calls (#477).
• Allow users to set an alternative path to the YAML configurationfile for the current R session (#477). Most users have no reason to setthis path, it is only for niche applications like Shiny apps withtargets backends. Unavoidably, the path gets reset to_targets.yaml when the session restarts.
• Add new_targets.yaml config optionsreporter_make,reporter_outdated, andworkers to control function argument defaults shared acrossmultiple functions called outside_targets.R (#498,@ianeveperry).
• Addtar_load_globals() for debugging, testing,prototyping, and teaching (#496,@malcolmbarrett).
• Add structure to theresources argument oftar_target() to avoid conflicts among formats and HPCbackends (#489). Includes user-side helper functions liketar_resources() andtar_resources_aws() tobuild the required data structures.
• Log skipped targets in_targets/meta/progress anddisplay then intar_progress(),tar_poll(),tar_watch(),tar_progress_branches(),tar_progress_summary(), andtar_visnetwork()(#514). Instead of writing each skip line separately to_targets/meta/progress, accumulate skip lines in a queueand then write them all out in bulk when something interesting happens.This avoids a lot of overhead in certain cases.
• Add ashortcut argument totar_make(),tar_make_clustermq(),tar_make_future(),tar_outdated(), andtar_sitrep() to moreefficiently skip parts of the pipeline (#522, #523,@jennysjaarda,@MilesMcBain,@kendonB).
• Supportnames andshortcut in graph dataframes and graph visuals (#529).
• Moveallow andexclude to the networkbehind the graph visuals rather than the visuals themselves (#529).
• Add a new “progress” display to thetar_watch() app toshow verbose progress info and metadata.
• Add a newworkspace_on_error argument oftar_option_set() to supersedeerror = "workspace". Helps control workspace behaviorindependently of theerror argument oftar_target() (#405, #533, #534,@mattwarkentin,@xinstein).
• Implementerror = "abridge" intar_target() and related functions. If a target errors outwith this option, the target itself stops, any currently running targetskeeps, and no new targets launch after that (#533, #534,@xinstein).
• Add a menu prompt totar_destroy() which can besuppressed withTAR_ASK = "false" (#542,@gofford).
• Support functionstar_older() andtar_newer() to help users identify and invalidate targetsat regular times or intervals.

Deprecations

• In Target Markdown, deprecate thetargets chunk optionin favor oftar_globals (#469).
• Deprecateerror = "workspace" intar_target() and related functions. Usetar_option_set(workspace_on_error = TRUE) instead (#405,#533,@mattwarkentin,@xinstein).

Performance

• Reset the backoff upper bound when concluding a target or shuttingdown aclustermq worker (@rich-payne).
• Set more aggressive default backoff bound of 0.1 seconds (previous:5 seconds) and set a more aggressive minimum of 0.001 seconds (previous:0.01 seconds) (@rich-payne).
• Speed up the summary and forecast reporters by only printing to theconsole every quarter second.
• Avoid superfluous calls tostore_sync_file_meta.default() on small files.
• Intar_watch(), take several measures to avoid longcomputation times rendering the graph:
  • Expose argumentsdisplay anddisplays totar_watch() so the user can select which display showsfirst.
  • Make"summary" the default display instead of"graph".
  • Setoutdated toFALSE by default.

Enhancements

• Simplify the Target Markdown example.
• Warn about unnamed chunks in Target Markdown.
• Redesign option system to be more object-oriented and rigorous. Alsoexport most options to HPC workers (#475).
• Simplify config system to let API function arguments take control(#483).
• Intar_read() for targets withformat = "aws_file", download the file back to the path theuser originally saved it when the target ran.
• Replace theTAR_MAKE_REPORTER environment variable withtargets::tar_config_get("reporter_make").
• Useeval(parse(text = readLines("_targets.R")), envir = some_envir)and related techniques instead of the less controllablesource(). Expose anenvir argument to manyfunctions for further control over evaluation ifcallr_function isNULL.
• Dropout.attrs when hashing groups of data frames toextend #507 toexpand.grid() (#508).
• Increase the number of characters in errors and warnings up to2048.
• Refactor assertions to automatically generate better messages.
• Export assertions, conditions, and language utilities in packagesthat build on top oftargets.
• ChangeGITHUBPAT toGITHUB_TOKEN in thetar_github_actions() YAML file (#554,@eveyp).
• Support theeval chunk option in Target Markdown (#552,@fkohrt).
• Record time stamps in the metadatatime column for allbuilder targets, regardless of storage format.

targets 0.5.0

Bug fixes

• Export in-memory config settings from_targets.yaml toparallel workers.

New features

• Add a limited-scopeexclude argument totar_watch() andtar_watch_server() (#458,@gorkang).
• Write a.gitignore file to ignore everything in_targets/meta/ except.gitignore and_targets/meta/meta.
• Target Markdown: addknitr engines for pipelineconstruction and prototyping from within literate programming documents(#469,@cderv,@nviets,@emilyriederer,@ijlyttle,@GShotwell,@gadenbuie,@tomsing1). Huge thanks to@cderv on this one foranswering my deluge of questions, helping me figure out what was and wasnot possible inknitr, and ultimately circling me back to asuccessful approach.
• Add an RStudio R Markdown template for Target Markdown (#469).
• Implementuse_targets(), which writes the TargetMarkdown template to the project root (#469).
• Implementtar_unscript() to clean up scripts written byTarget Markdown.

Enhancements

• Enable priorities intar_make() andtar_manifest().
• Show the priority in the print method of stem and patterntargets.
• Throw informative errors if the secondary arguments topattern = slice() orpattern = sample() areinvalid.
• Intar_target_raw(), assert that commands have length 1when converted to expressions.
• Handle errors and post failure artifacts in the Github Actions YAMLfile.
• Rewrite the documentation on invalidation rules intar_cue() (@maelle).
• Dropdplyr groups and"grouped_df" classintar_group() (tarchetypes discussion #53,@kendonB).
• Assign branch names to dynamic branching return values produced bytar_read() andtar_read_raw().

targets 0.4.2

Bug fixes

• Do not use time stamps to monitor the config file(e.g. _targets.yaml). Fixes CRAN check errors from version0.4.1.

targets 0.4.1

• Fix CRAN test error on Windows R-devel.
• Do not inheritroxygen2 docstrings fromshiny.
• Handle more missingSuggests: packages.
• Unset the config lock before readingtargets.yaml inthecallr process.

targets 0.4.0

Bug fixes

• Avoidfile.rename() errors when migrating stagedtemporary files (#410).
• Return correct error messages from feather and parquet formats(#388). Now callingassert_df() fromstore_assert_format() instead ofstore_cast_object(). And now those last two functions arenot called at all if the target throws an error.
• Retry writing lines to database files so Windows machines can runtar_poll() at the same time as the pipeline (#393).
• Rename file written bytar_renv() to_targets_packages.R (#397).
• Ensure metadata is loaded to compute labels properly whenoutdated = FALSE intar_visnetwork().

New features

• Implementtar_timestamp() andtar_timestamp_raw() to get the last modified timestamp of atarget’s data (#378).
• Implementtar_progress_summary() to compactly summarizeall pipeline progress (#380).
• Add acharacters argument oftar_traceback() to cap the traceback line lengths(#383).
• Add new “summary” and “about” views totar_watch()(#382).
• Implementtar_poll() to repeatedly poll runtimeprogress in the R console (#381).tar_poll() is alightweight alternative totar_watch().
• Change the color of the “dormant” status in the graph.
• Add atar_envvar() function to list values of specialenvironment variables supported intargets. The help fileexplains each environment variable in detail.
• Support extra project-level configuration settings with_targets.yaml (#297). New functionstar_config_get() andtar_config_set() interactwith the_targets.yaml file. Currently only supports thestore field to set the data store path to something otherthan_targets/.

Performance

• Shut down superfluous persistent workers earlier in dynamicbranching and when all remaining targets havedeployment = "main" (#398, #399, #404,@pat-s).

Enhancements

• Attempt to print only the useful part of the traceback intar_traceback() (#383).
• Add a line break at the end of the “summary” reporter so warnings donot mangle the output.
• Intar_watch(), useshinybusy instead ofshinycssloaders and keep current output on display whilenew output is rendering (#386,@rcorty).
• Right-align the headers and counts in the “summary” and “forecast”reporters.
• Add a timestamp to the “summary” reporter.
• Make the reporters show when a target ends (#391,@mattwarkentin).
• Make the reporters show when a pattern ends if the pattern built atleast one target and none of the targets errored or canceled.
• Use words “start” and “built” in reporters.
• Use the region of the AWS S3 bucket instead of the localAWS_DEFAULT_REGION environment variable(check_region = TRUE; #400,@tomsing1).
• Intar_meta(), returnPOSIXct times in thetime zone of the calling system (#131).
• Throw informative error messages when a target’s name or command ismissing (#413,@liutiming).
• Bring back ALTREP inqs::qread() now thatqs 0.24.1 requiresstringfish >= 1.5.0(#147,@glep).
• Relax dynamic branching checks sopattern = slice(...)can take multiple indexes (#406, #419,@djbirke,@alexgphayes)

targets 0.3.1

Bug fixes

• queue$enqueue() is nowqueue$prepend() andalways appends to the front of the queue (#371).

Enhancements

• Throw a warning ifdevtools::load_all() or similar isdetected inside_targets.R (#374).

CRAN

• Skipfeather andparquet tests onCRAN.

targets 0.3.0

Bug fixes

• Fix the “write target at cursor” RStudio addin and move cursorbetween the parentheses.

New features

• Add abackoff option intar_option_set()to set the maximum upper bound (seconds) for the polling interval(#333).
• Add a newtar_github_actions() function to write aGitHub Actions workflow file for continuous deployment of data analysispipelines (#339,@jaredlander).
• Add a newTAR_MAKE_REPORTER environment variable toglobally set the reporter of thetar_make*() functions(#345,@alexpghayes).
• Support new storage formats “feather”, “parquet”, “aws_feather”, and“aws_parquet” (#355,@riazarbi).

Performance

• Implement an exponential backoff algorithm for polling the priorityqueue intar_make_clustermq() andtar_make_future() (#333).
• Intar_make_future(), try to submit a target every timea worker is polled.
• Intar_make_future(), poll workers in order of targetpriority.
• Avoid the time delay in exiting on error (fromhttps://github.com/r-lib/callr/issues/185).
• Clone target objects for the pipeline and scrape moretargets internal objects out of the environment in order toavoid accidental massive data transfers to workers.

Enhancements

• Userlang::check_installed() insideassert_package() (#331,@malcolmbarrett).
• Allowtar_destroy(destroy = "process").
• Intar_watch(), increase defaultsecondsto 15 (previously 5).
• Intar_watch(), debounce instead of throttleinputs.
• Intar_watch(), add an action button to refresh theoutputs.
• Always deduplicate metadata aftertar_make(). Will helpcompute a cache key on GitHub Actions and similar services.
• Deprecatetar_deduplicate() due to the item above.
• Reorder information in timestamped messages.
• Document RNG seed generation intar_target_raw(),tar_meta(), andtar_seed() (#357,@alexpghayes).
• Switch meaning of%||% and%|||% toconform to historical precedent.
• Only show a command line spinner ifreporter = "silent"(#364,@matthiasgomolka).
• Target and pipeline objects no longer have anenvirelement.

targets 0.2.0

Bug fixes

• Intar_load(), subset metadata to avoid accidentalattempts to load global objects intidyselect calls.
• Do not register a pattern as running unless an actual branch isabout to start (#304).
• Use a name spec invctrs::vec_c() (#320,@joelnitta).

New features

• Add a newnames argument totar_objects()andtar_workspaces() withtidyselectfunctionality.
• Record info on the main process (PID, R version,targets version) in_targets/meta/process andwrite new functionstar_process() andtar_pid() to retrieve the data (#291, #292).
• Add a newtargets_only argument totar_meta().
• Add new functionstar_helper() andtar_helper_raw() to write general-purpose R scripts, usingtidy evaluation for as a template mechanism (#290, #291, #292,#306).
• Export functions to check the existence of various pieces of localstorage:tar_exist_meta(),tar_exist_objects(),tar_exist_progress(),tar_exist_progress(),tar_exist_script()(#310).
• Add a newsupervise argument totar_watch().
• Add a newcomplete_only argument totar_meta() to optionally return only complete rows (noNA values).
• Catchcallr errors and refer users to the debuggingchapter of the manual.

Enhancements

• Improve error messages of invalid arguments (#298,@brunocarlin). Removespartial argument matching in most cases.
• By default, locally enablecrayon if an only if thecalling process is interactive (#302,@ginolhac). Can still be disabled withoptions(crayon.enabled = FALSE) in_targets.R.
• Improve error handling and message forformat = "url"when the HTTP response status code is not 200 (#303,@petrbouchal).
• Add moreextras packages totar_renv() (tosupporttar_watch()).
• Show informative message instead of error intar_watch() if_targets.R does not exist.
• Clear up the documentation of thenames argument oftar_load() (#314,@jameelalsalam).
• Do not overridenobody in customcurlhandles (#315,@riazarbi).
• Rename “running” to “started” in the progress metadata. This avoidsthe implicit claim thattargets is somehow activelymonitoring each job, e.g. through a connection or heartbeat (#318).
• Seterrormode = "warn" ingetVDigest() forfiles to work around https://github.com/eddelbuettel/digest/issues/49for network drives on Windows.targets already runs thosefile checks anyway. (#316,@boshek).
• If a package fails to load, print the library pathstargets tried to load from.

targets 0.1.0

Bug fixes

• tar_test() now skips all tests on Solaris in order tofix the problems shown on the CRAN check page.
• Enableallow andexclude to work onimports intar_visnetwork() andtar_glimpse().
• PutvisNetwork legends on right to avoid crowding thegraph.

Performance

• Callforce() on subpipeline objects to eliminatehigh-memory promises in target objects. Allows targets to be deployed toworkers much faster whenretreival is"main"(#279).

New features

• Add a new box to thetar_watch() app to tabulateprogress on dynamic branches (#273,@mattwarkentin).
• Storetype,parent, andbranches in progress data fortar_watch()(#273,@mattwarkentin).
• Add afields argument intar_progress()and default to"progress" for back compatibility (#273,@mattwarkentin).
• Add a newtar_progress_branches() function to tabulatebranch progress (#273,@mattwarkentin).
• Add new “refresh” switch totar_watch() to toggleautomatic refreshing and force a refresh.

Enhancements

• Exclude.Random.seed by default intar_visnetwork().
• Spelling: “cancelled” changed to “canceled”.
• Enhance controls and use of space in thetar_watch()app.
• Centralize internal path management utilities.

Configuration

• Skipclustermq tests on Solaris.

targets 0.0.2

CRAN response

• Avoid starting the description with the package name.
• Removeif(FALSE) blocks from help files to fix“unexecutable code” warnings (tar_glimpse(),tar_visnetwork(), andtar_watch()).
• Remove commented code in the examples (tar_edit(),tar_watch_ui(), andtar_watch_server()).
• Ensure that all examples, tests, and vignettes do not write to theuser’s home file space. (Fixed an example oftar_workspace().)

Enhancements

• Use JOSS paper inCITATION.

targets 0.0.1

Enhancements

• Accept lists of target objects at the end of_targets.R(#253).
• Deprecatetar_pipeline() andtar_bind()because of the above (#253).
• Always show a special message when the pipeline finishes (#258,@petrbouchal).
• DisablevisNetwork stabilization (#264,@mattwarkentin).
• Use defaultvisNetwork font size.
• Relay errors as condition messages iferror is"continue" (#267,@liutiming).

targets 0.0.0.9003

Bug fixes

• Ensure pattern-only pipelines can be defined so they can be combinedagain later withtar_bind() (#245,@yonicd).
• Implement safeguards aroundigraph topologicalsort.

Enhancements

• Topologically sort the rows oftar_manifest() (#263,@sctyner).

Breaking changes

• Make patterns composable (#212,@glep,@djbirke).
• Allow workspaces to load nonexportable objects (#214).
• Make workspace files super light by saving only a reference to therequired dependencies (#214).
• Add a newworkspaces argument totar_option_set() to specify which targets will save theirworkspace files duringtar_make() (#214).
• Changeerror = "save" toerror = "workspace" to so it is clearer that savingworkspaces no longer duplicates data (#214).
• Renamewhat todestroy intar_destroy().
• Removetar_undebug() because is redundant withtar_destroy(destroy = "workspaces").

New features

• Make patterns composable (#212).
• Add new dynamic branching patternshead(),tail(), andsample() to provide functionalityequivalent todrake’smax_expand (#56).
• Add a newtar_pattern() function to emulate dynamicbranching outside a pipeline.
• Add a newlevel_separation argument totar_visnetwork() andtar_glimpse() to controlthe aspect ratio (#226).
• Track functions from multiple packages with theimportsargument totar_option_set() (#239).
• Add color for “built” progress ifoutdated isFALSE intar_visnetwork().
• Tweak colors intar_visnetwork() to try to account forcolor blindness.

Enhancements

• Return full patterns fromtar_manifest().
• Record package load errors in progress and metadata (#228,@psychelzh).
• tar_renv() now invokes_targets.R througha background process just liketar_outdated() etc. so itcan account for more hidden packages (#224,@mattwarkentin).
• Setdeployment equal to"main" for alltargets intar_make(). This ensurestar_make()does not waste time waiting for nonexistent files to ship over anonexistent network file system (NFS).tar_make_clustermq()ortar_make_future() could use NFS, so they still leavedeployment alone.

targets 0.0.0.9002

Breaking changes

• Add a newsize field to the metadata to allowtargets to make better judgments about when to rehash files(#180). We now compare hashes to check file size differences instead ofdoing messy floating point comparisons with ad hoc tolerances. It breaksback compatibility with old projects, but the error message isinformative, and this is all still before the first officialrelease.
• Change “local” to “main” and “remote” to “worker” in thestorage,retrieval, anddeployment settings (#183,@mattwarkentin).
• Ensure function dependencies are sorted before computing thefunction hash (GitHub commitf15face7d72c15c2d1098da959492bdbfcddb425).
• Movegarbage_collection to a target-level setting,i.e. argument totar_target() andtar_option_set() (#194). Previously was an argument to thetar_make*() functions.
• Allowtar_name() andtar_path() to runoutside the pipeline with debugging-friendly default return values.

Bug fixes

• Stop sending target return values over the network whenstorage is"remote" (#182,@mattwarkentin).
• Shorten lengths of warnings and error messages to 128 characters(#186,@gorkang).
• Restrict in-memory metadata to avoid incorrectly recycling deletedtargets (#191).
• Marshal nonexportable dependencies before sending them to workers.Transport data throughtarget$subpipeline rather thantarget$cache to make that happen (#209,@mattwarkentin).

New features

• Add a new functiontar_bind() to combine pipelineobjects.
• Addtar_seed() to get the random number generator seedof the target currently running.

Enhancements

• Allow target-specificfuture::plan()s through theresources argument oftar_target() (#198,@mattwarkentin).
• Uselibrary() instead ofrequire() incommand_load_packages().
• Evaluate commands directly intargets$cache$targets$envir to improve convenience ininteractive debugging (ls() just works now.) This isreasonably safe now that the cache is populated at the last minute andcleared as soon as possible (#209, #210).

targets 0.0.0.9000

• First version.