Functions available in Arrow dplyr queries

Thearrow package contains methods for 37dplyr table functions, many ofwhich are "verbs" that do transformations to one or more tables.The package also has mappings of 223 R functions to the correspondingfunctions in the Arrow compute library. These allow you to write code insideofdplyr methods that call R functions, including many in packages likestringr andlubridate, and they will get translated to Arrow and runon the Arrow query engine (Acero). This document lists all of the mappedfunctions.

dplyr verbs

Most verb functions return anarrow_dplyr_query object, similar in spiritto adbplyr::tbl_lazy. This means that the verbs do not eagerly evaluatethe query on the data. To run the query, call eithercompute(),which returns anarrowTable, orcollect(), which pulls the resultingTable into an Rtibble.

• anti_join(): thecopy argument is ignored

• arrange()

• collapse()

• collect()

• compute()

• count()

• distinct():.keep_all = TRUE returns a non-missing value if present, only returning missing values if all are missing.

• explain()

• filter()

• full_join(): thecopy argument is ignored

• glimpse()

• group_by()

• group_by_drop_default()

• group_vars()

• groups()

• inner_join(): thecopy argument is ignored

• left_join(): thecopy argument is ignored

• mutate()

• pull(): thename argument is not supported; returns an R vector by default but this behavior is deprecated and will return an ArrowChunkedArray in a future release. Provideas_vector = TRUE/FALSE to control this behavior, or setoptions(arrow.pull_as_vector) globally.

• relocate()

• rename()

• rename_with()

• right_join(): thecopy argument is ignored

• select()

• semi_join(): thecopy argument is ignored

• show_query()

• slice_head(): slicing within groups not supported; Arrow datasets do not have row order, so head is non-deterministic;prop only supported on queries wherenrow() is knowable without evaluating

• slice_max(): slicing within groups not supported;with_ties = TRUE (dplyr default) is not supported;prop only supported on queries wherenrow() is knowable without evaluating

• slice_min(): slicing within groups not supported;with_ties = TRUE (dplyr default) is not supported;prop only supported on queries wherenrow() is knowable without evaluating

• slice_sample(): slicing within groups not supported;replace = TRUE and theweight_by argument not supported;n only supported on queries wherenrow() is knowable without evaluating

• slice_tail(): slicing within groups not supported; Arrow datasets do not have row order, so tail is non-deterministic;prop only supported on queries wherenrow() is knowable without evaluating

• summarise(): window functions not currently supported; arguments.drop = FALSE and.groups = "rowwise" not supported

• tally()

• transmute()

• ungroup()

• union()

• union_all()

Function mappings

In the list below, any differences in behavior or support between Acero andthe R function are listed. If no notes follow the function name, then youcan assume that the function works in Acero just as it does in R.

Functions can be called either aspkg::fun() or justfun(), i.e. bothstr_sub() andstringr::str_sub() work.

In addition to these functions, you can call any of Arrow's 281 computefunctions directly. Arrow has many functions that don't map to an existing Rfunction. In other cases where there is an R function mapping, you can stillcall the Arrow function directly if you don't want the adaptations that the Rmapping has that make Acero behave like R. These functions are listed in theC++ documentation, andin the function registry in R, they are named with anarrow_ prefix, suchasarrow_ascii_is_decimal.