Movatterモバイル変換

Any R object

ADBI::DBIConnection object,as returned bydbConnect().

Other parameters passed on to methods.

An object inheriting fromDBIObject, i.e.DBIDriver,DBIConnection, or aDBIResult.

a character string containing SQL.

boolean input to modifystatement to align withAWS Athena UNLOAD,default is set toFALSE.

An R object whose SQL type we want to determine.

A character vector to quote as string.

Athena catalog, default set to NULL to return all tables from all Athena catalogs

Athena schema, default set to NULL to return all tables from all Athena schemas.Note: The use of DATABASE and SCHEMA is interchangeable within Athena.

The table name, passed on todbQuoteIdentifier(). Options are:

• a character string with the unquoted DBMS table name,e.g."table_name",

• a call toId() with components to the fully qualified table name,e.g.Id(schema = "my_schema", table = "table_name")

• a call toSQL() with the quoted and fully qualified table namegiven verbatim, e.g.SQL('"my_schema"."table_name"')

Deletes S3 files linking to AWS Athena table

Allows for S3 files to be deleted without the prompt check. It is recommend to leave this set toFALSEto avoid deleting other S3 files when the table's definition points to the root of S3 bucket.

If set toTRUE will print out AWS Athena statistics of query.

re-formats AWS Athena partitions format. So that each column represents a partitionfrom the AWS Athena table. Default set toFALSE to prevent breaking previous package behaviour.

Any R object

A object inheriting fromDBI::DBIDriver orDBI::DBIConnection.

An R object whose SQL type we want to determine.

Other arguments passed on to methods.

A connection object, as returned bydbConnect().

Attributes to filter by.

A connection object, as returned bydbConnect().

The maximum number of rows to display.

Parameters specifying the object.

An object inheriting fromDBI::DBIResult.

Other arguments passed on to methods.

maximum number of records to retrieve per fetch. Usen = -1orn = Infto retrieve all pending records. Some implementations may recognize otherspecial values.

An object inheriting fromDBI::DBIResult,DBI::DBIConnection, orDBI::DBIDriver.

AnAthenaConnection object, produced byDBI::dbConnect()

A character string specifying a table name. Names will beautomatically quoted so you can use any sequence of characters, notjust any valid bare table name.

A data.frame to write to the database.

Allows overwriting the destination table. Cannot beTRUE ifappend is alsoTRUE.

Allow appending to the destination table. Cannot beTRUE ifoverwrite is alsoTRUE. Existing Athena DDL file type will be retainedand used when uploading data to AWS Athena. If parameterfile.type doesn't match AWS Athena DDL file type a warning message will be creatednotifying user andRAthena will use the file type for the Athena DDL. When appending to an Athena DDL that has been created outside ofRAthena.RAthena can support the following SerDes and Data Formats.

• csv/tsv:LazySimpleSerDe

• parquet:Parquet SerDe

• json:JSON SerDe Libraries

EitherTRUE,FALSE,NA or a string.

IfTRUE, always translate row names to a column called "row_names".IfFALSE, never translate row names. IfNA, translaterownames only if they're a character vector.

A string is equivalent toTRUE, but allows you to override thedefault name.

For backward compatibility,NULL is equivalent toFALSE.

Additional field types used to override derived types.

Partition Athena table (needs to be a named list or vector) for example:c(var1 = "2019-20-13")

s3 bucket to store Athena table, must be set as a s3 uri for example ("s3://mybucket/data/").By default, the s3.location is set to s3 staging directory fromAthenaConnection object.Note:When creating a table for the first times3.location will be formatted from"s3://mybucket/data/" to the followingsyntax"s3://{mybucket/data}/{schema}/{table}/{parition}/" this is to support tables with the same name but existing in differentschemas. If schema isn't specified inname parameter then the schema fromdbConnect is used instead.

What file type to store data.frame on s3, RAthena currently supportsc("tsv", "csv", "parquet", "json"). Default delimited file type is "tsv", in previous versionsofRAthena (=< 1.6.0) file type "csv" was used as default. The reason for the change is that columns containingArray/JSON format cannot be written toAthena due to the separating value ",". This would cause issues with AWS Athena.Note: "parquet" format is supported by thearrow package and it will need to be installed to utilise the "parquet" format."json" format is supported byjsonlite package and it will need to be installed to utilise the "json" format.

FALSE | TRUE To determine if to compress file.type. If file type isc("csv", "tsv") then "gzip" compression is used, for file type "parquet""snappy" compression is used. CurrentlyRAthena doesn't support compression for "json" file type.

Split the data frame by max number of rows i.e. 100,000 so that multiple files can be uploaded into AWS S3. By default when compressionis set toTRUE and file.type is "csv" or "tsv" max.batch will split data.frame into 20 batches. This is to help theperformance of AWS Athena when working with files compressed in "gzip" format.max.batch will not split the data.framewhen loading file in parquet format. For more information please go tolink

Other arguments used by individual methods.

Method to read and write tables to Athena, currently default to"data.table". The file_parser alsodetermines the data format returned for example"data.table" will returndata.table and"vroom" will returntibble.

The R type that 64-bit integer types should be mapped to (default:"integer64").Inbuiltbigint conversion typesc("integer64", "integer", "numeric", "character").

The R type that "binary/varbinary" types should be mapped to (default"raw").Inbuilt binary conversion typesc("raw", "character").

Attempt to converts AWS Athena data types arrays, json usingjsonlite:parse_json (default:"auto").Inbuilt json conversion typesc("auto", "character").Custom Json parsers can be provide by using a function with data frame parameter.

Number of queries to be cached. Currently only support caching up to 100 distinct queries (default:0).

Clears all previous cached query metadata

Maximum number of requests to attempt (default:5).

This method is deprecated please use verbose instead.

set AWS Athena unload functionality globally (default:FALSE)

Clear down⁠AWS Athena⁠⁠AWS S3⁠ resource (s3_staging_dir location).This is useful for users that don't have the⁠AWS IAM role⁠ permissions deletefroms3_staging_dir (default:TRUE)

print package info messages (default:TRUE)

The name of a profile to use. If not given, then the default profile is used.To set profile name, theAWS Command Line Interface (AWS CLI) will need to be configured.To configure AWS CLI please refer to:Configuring the AWS CLI.

Default region when creating new connections. Please refer tolink forAWS region codes (region code example: Region = EU (Ireland) region_name = "eu-west-1")

The Amazon Resource Name (ARN) of the role to assume (such asarn:aws:sts::123456789012:assumed-role/role_name/role_session_name)

An identifier for the assumed role session. By defaultRAthena creates a session namesprintf("RAthena-session-%s", as.integer(Sys.time()))

The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role.This setting can have a value from 1 hour to 12 hours. By default duration is set to 3600 seconds (1 hour).

If set toTRUE environmental variablesAWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY andAWS_SESSION_TOKEN will be set.

AdbConnect object, as returned bydbConnect()

SQL code to be sent to AWS Athena

returning format for explain queries, default set to"text". Other formats can be found:https://docs.aws.amazon.com/athena/latest/ug/athena-explain-statement.html

return plan for explain queries, default set toNULL. Other type can be found:https://docs.aws.amazon.com/athena/latest/ug/athena-explain-statement.html

other parameters, currently not implemented

R object to be transformed into athena equivalent

AdbConnect object, as returned bydbConnect()

SQL code to be sent to AWS Athena

other parameters, currently not implemented

A object inheriting fromDBI::DBIDriver.

AWS access key ID

AWS secret access key

AWS temporary session token

The catalog_name to which the connection belongs

The schema_name to which the connection belongs

The name of thework group to run Athena queries , Currently defaulted toNULL.

Amount of time took when checking query execution status. Default set to a random interval between 0.5 - 1 seconds.

Athena encryption at restlink.Supported Amazon S3 Encryption Optionsc("NULL", "SSE_S3", "SSE_KMS", "CSE_KMS"). Connection will default to NULL,usually changing this option is not required.

AWS Key Management Service,please refer tolink for more information around the concept.

The name of a profile to use. If not given, then the default profile is used.To set profile name, theAWS Command Line Interface (AWS CLI) will need to be configured.To configure AWS CLI please refer to:Configuring the AWS CLI.

The Amazon Resource Name (ARN) of the role to assume (such asarn:aws:sts::123456789012:assumed-role/role_name/role_session_name)

An identifier for the assumed role session. By defaultRAthena creates a session namesprintf("RAthena-session-%s", as.integer(Sys.time()))

The duration, in seconds, of the role session. The value can range from 900 seconds (15 minutes) up to the maximum session duration setting for the role.This setting can have a value from 1 hour to 12 hours. By default duration is set to 3600 seconds (1 hour).

The location in Amazon S3 where your query results are stored, such ass3://path/to/query/bucket/

Default region when creating new connections. Please refer tolink forAWS region codes (region code example: Region = EU (Ireland) region_name = "eu-west-1")

Use this Botocore session instead of creating a new default one.

The R type that 64-bit integer types should be mapped to,default isbit64::integer64, which allows the full range of 64 bitintegers.

The R type that "binary/varbinary" types should be mapped to,default israw. If the mapping fails R will resort to⁠character`` type. To ignore data type conversion set to ⁠character'.

Attempt to converts AWS Athena data types arrays, json usingjsonlite:parse_json. If the mapping fails R will resort tocharacter type.Custom Json parsers can be provide by using a function with data frame parameter.To ignore data type conversion set to"character".

Sets the timezone for the connection. The default isUTC.IfNULL then no timezone is set, which defaults to the server's time zone.⁠AWS Athena⁠ accepted time zones:https://docs.aws.amazon.com/athena/latest/ug/athena-supported-time-zones.html.

Stops AWS Athena process when R gets a keyboard interrupt, currently defaults toTRUE

Optional to get AWS Athena Schema from AWS Glue Catalogue and display it in RStudio's Connections Tab.Default set toTRUE. For large⁠AWS Glue Catalogue⁠ it is recommended to setrstudio_conn_tab=FALSE to ensure a fast connection.

(character/list) The complete URL to use for the constructed client. Normally,botocore will automatically construct the appropriate URL to use when communicating with aservice. You can specify a complete URL (including the "http/https" scheme) to override thisbehaviour. Ifendpoint_override is a character then AWS Athena endpoint is overridden. To overrideAWS S3 or AWS Glue endpoints a named list needs to be provided. The list can only have the following names⁠c('athena', 's3', glue')⁠for examplelist(glue = "https://glue.eu-west-1.amazonaws.com")

Passes parameters toboto3.session.Session andclient.

• boto3.session.Session

  • botocore_session (botocore.session.Session): Use this Botocore session insteadof creating a new default one.

• boto3.client

  • config (botocore.client.Config) – Advanced client configuration options. If region_nameis specified in the client config, its value will take precedence over environment variablesand configuration values, but not over a region_name value passed explicitly to the method.Seebotocore config documentationfor more details.

  • api_version (string) – The API version to use. By default, botocore will use the latestAPI version when creating a client. You only need to specify this parameter if you want touse a previous API version of the client.

  • use_ssl (boolean) – Whether or not to use SSL. By default, SSL is used. Note thatnot all services support non-ssl connections.

  • verify (boolean/string) – Whether or not to verify SSL certificates. By defaultSSL certificates are verified. You can provide the following values:

    • False - do not validate SSL certificates. SSL will still be used (unless use_ssl is False),but SSL certificates will not be verified.

    • path/to/cert/bundle.pem - A filename of the CA cert bundle to uses. You can specify thisargument if you want to use a different CA cert bundle than the one used by botocore.

ADBI::DBIConnection object,

Athena table orSQL DML query to be converted. ForSQL, the query need to be wrapped withDBI::SQL() andfollow AWS Athena DML formatlink

Name of destination table

Extra parameters, currently not used

Partition Athena table

location to store output file, must be in s3 uri format for example ("s3://mybucket/data/").

File type forname, currently supportc("NULL","csv", "tsv", "parquet", "json", "orc")."NULL" will let Athena set the file type for you.

Compressname, currently can only compressc("parquet", "orc") (AWS Athena CTAS)

Ifname should be created with data or not.

ADBI::DBIConnection object,as returned bydbConnect().

Athena catalog, default set to NULL to return all tables from all Athena catalogs

Athena schema, default set to NULL to return all tables from all Athena schemas.Note: The use of DATABASE and SCHEMA is interchangeable within Athena.

Other parameters passed on to methods.

AdbConnect object, as returned bydbConnect()

Table name, if left default RAthena will use the default fromdplyr'scompute function.

SQL code to be sent to the data

passesRAthena table creation parameters:file_type,s3_location,partition.

Allows overwriting the destination table. Cannot beTRUE ifappend is alsoTRUE.

if TRUE, will create a temporary table that is local to this connection and will be automatically deleted when the connection expires

a list of character vectors. Each element of the list will create a new unique index over the specified column(s). Duplicate rows will result in failure.

a list of character vectors. Each element of the list will create a new index.

if TRUE (the default), will automatically ANALYZE the new table so that the query optimiser has useful information.

Should the table creation be wrapped in a transaction? This typically makes things faster, but you may want to suppress if the database doesn't support transactions, or you're wrapping in a transaction higher up (and your database doesn't support nested transactions.)

Partition Athena table (needs to be a named list or vector) for example:c(var1 = "2019-20-13")

s3 bucket to store Athena table, must be set as a s3 uri for example ("s3://mybucket/data/")

What file type to store data.frame on s3, RAthena currently supportsc("tsv", "csv", "parquet"). Default delimited file type is "tsv", in previous versionsofRAthena (=< 1.4.0) file type "csv" was used as default. The reason for the change is that columns containingArray/JSON format cannot be written toAthena due to the separating value ",". This would cause issues with AWS Athena.Note: "parquet" format is supported by thearrow package and it will need to be installed to utilise the "parquet" format.

FALSE | TRUE To determine if to compress file.type. If file type isc("csv", "tsv") then "gzip" compression is used, for file type "parquet""snappy" compression is used.

• file_type: What file type to store data.frame on s3, RAthena currently supportsc("NULL","csv", "parquet", "json")."NULL" will let Athena set the file_type for you.

• s3_location: s3 bucket to store Athena table, must be set as a s3 uri for example ("s3://mybucket/data/")

• partition: Partition Athena table, requires to be a partitioned variable from previous table.

Table name, if left default RAthena will use the default fromdplyr'scompute function.

AdbConnect object, as returned bydbConnect()

AdbConnect object, as returned bydbConnect()

A character string specifying a table name. Names will beautomatically quoted so you can use any sequence of characters, notjust any valid bare table name.

A data.frame to write to the database.

other parameters currently not supported in RAthena

Partition Athena table (needs to be a named list or vector) for example:c(var1 = "2019-20-13")

s3 bucket to store Athena table, must be set as a s3 uri for example ("s3://mybucket/data/")

What file type to store data.frame on s3, RAthena currently supportsc("tsv", "csv", "parquet"). Default delimited file type is "tsv", in previous versionsofRAthena (=< 1.4.0) file type "csv" was used as default. The reason for the change is that columns containingArray/JSON format cannot be written toAthena due to the separating value ",". This would cause issues with AWS Athena.Note: "parquet" format is supported by thearrow package and it will need to be installed to utilise the "parquet" format.

FALSE | TRUE To determine if to compress file.type. If file type isc("csv", "tsv") then "gzip" compression is used, for file type "parquet""snappy" compression is used.

Split the data frame by max number of rows i.e. 100,000 so that multiple files can be uploaded into AWS S3. By default when compressionis set toTRUE and file.type is "csv" or "tsv" max.batch will split data.frame into 20 batches. This is to help theperformance of AWS Athena when working with files compressed in "gzip" format.max.batch will not split the data.framewhen loading file in parquet format. For more information please go tolink

Allows overwriting the destination table. Cannot beTRUE ifappend is alsoTRUE.

Allow appending to the destination table. Cannot beTRUE ifoverwrite is alsoTRUE. Existing Athena DDL file type will be retainedand used when uploading data to AWS Athena. If parameterfile.type doesn't match AWS Athena DDL file type a warning message will be creatednotifying user andRAthena will use the file type for the Athena DDL.

Additional field types used to override derived types.

if TRUE, will create a temporary table that is local to this connection and will be automatically deleted when the connection expires

a list of character vectors. Each element of the list will create a new unique index over the specified column(s). Duplicate rows will result in failure.

a list of character vectors. Each element of the list will create a new index.

if TRUE (the default), will automatically ANALYZE the new table so that the query optimiser has useful information.

Should the table creation be wrapped in a transaction? This typically makes things faster, but you may want to suppress if the database doesn't support transactions, or you're wrapping in a transaction higher up (and your database doesn't support nested transactions.)

AdbConnect object, as returned bydbConnect()

AdbConnect object, as returned bydbConnect()

Installation method. By default, "auto" automatically finds amethod that will work in the local environment. Change the default to forcea specific installation method. Note that the "virtualenv" method is notavailable on Windows. Note also that since this command runs without privilegethe "system" method is available only on Windows.

The path to aconda executable. Use"auto" to allowreticulate to automatically find an appropriateconda binary.SeeFinding Conda andconda_binary() for more details.

Name of Python environment to install within, by default environment name RAthena.

the python version installed in the created condaenvironment. Python 3.13 is installed by default.

other arguments passed toreticulate::conda_install() orreticulate::virtualenv_install().

The name of a profile to use. If not given, then the default profile is used.To set profile name, theAWS Command Line Interface (AWS CLI) will need to be configured.To configure AWS CLI please refer to:Configuring the AWS CLI.

Default region when creating new connections. Please refer tolink forAWS region codes (region code example: Region = EU (Ireland) region_name = "eu-west-1")

The identification number of the MFA device that is associated with the IAM user who is making the GetSessionToken call.Specify this value if the IAM user has a policy that requires MFA authentication. The value is either the serial number for a hardware device(such asGAHT12345678) or an Amazon Resource Name (ARN) for a virtual device (such as arn:aws:iam::123456789012:mfa/user).

The value provided by the MFA device, if MFA is required. If any policy requires the IAM user to submit an MFA code,specify this value. If MFA authentication is required, the user must provide a code when requesting a set of temporarysecurity credentials. A user who fails to provide the code receives an "access denied" response when requesting resourcesthat require MFA authentication.

The duration, in seconds, that the credentials should remain valid. Acceptable duration for IAM user sessions rangefrom 900 seconds (15 minutes) to 129,600 seconds (36 hours), with 3,600 seconds (1 hour) as the default.

If set toTRUE environmental variablesAWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY andAWS_SESSION_TOKEN will be set.

A database connection.

The table name, passed on todbQuoteIdentifier(). Options are:

• a character string with the unquoted DBMS table name,e.g."table_name",

• a call toId() with components to the fully qualified table name,e.g.Id(schema = "my_schema", table = "table_name")

• a call toSQL() with the quoted and fully qualified table namegiven verbatim, e.g.SQL('"my_schema"."table_name"')

Either a character vector or a data frame.

A named character vector: Names are column names, values are types.Names are escaped withdbQuoteIdentifier().Field types are unescaped.

A data frame: field types are generated usingdbDataType().

Additional field types used to override derived types.

Partition Athena table (needs to be a named list or vector) for example:c(var1 = "2019-20-13")

s3 bucket to store Athena table, must be set as a s3 uri for example ("s3://mybucket/data/").By default s3.location is set s3 staging directory fromAthenaConnection object.

What file type to store data.frame on s3, RAthena currently supportsc("tsv", "csv", "parquet", "json"). Default delimited file type is "tsv", in previous versionsofRAthena (=< 1.6.0) file type "csv" was used as default. The reason for the change is that columns containingArray/JSON format cannot be written toAthena due to the separating value ",". This would cause issues with AWS Athena.Note: "parquet" format is supported by thearrow package and it will need to be installed to utilise the "parquet" format."json" format is supported byjsonlite package and it will need to be installed to utilise the "json" format.

FALSE | TRUE To determine if to compress file.type. If file type isc("csv", "tsv") then "gzip" compression is used, for file type "parquet""snappy" compression is used. CurrentlyRAthena doesn't support compression for "json" file type.

Other arguments used by individual methods.

A database connection.

A data frame

EitherTRUE,FALSE,NA or a string.

IfTRUE, always translate row names to a column called "row_names".IfFALSE, never translate row names. IfNA, translaterownames only if they're a character vector.

A string is equivalent toTRUE, but allows you to override thedefault name.

For backward compatibility,NULL is equivalent toFALSE.

What file type to store data.frame on s3, RAthena currently supportsc("csv", "tsv", "parquet", "json").Note: This parameter is used for format any special characters that clash with file type separator.

Other arguments used by individual methods.

AnAthenaConnection object, produced byDBI::dbConnect()

AdbConnect object, as returned bydbConnect()

The Athena workgroup name.

If set toTRUE, the settings for the workgroup override client-side settings.If set toFALSE, client-side settings are used. For more information, seeWorkgroup Settings Override Client-Side Settings.

Indicates that the Amazon CloudWatch metrics are enabled for the workgroup.

The upper data usage limit (cutoff) for the amount of bytes a single query in a workgroup is allowed to scan.

If set toTRUE, allows members assigned to a workgroup to reference Amazon S3 Requester Pays buckets in queries.If set toFALSE, workgroup members cannot query data from Requester Pays buckets, and queries that retrieve datafrom Requester Pays buckets cause an error. The default is false. For more information about Requester Pays buckets,seeRequester Pays Buckets in the Amazon Simple Storage Service Developer Guide.

The workgroup description.

A tag that you can add to a resource. A tag is a label that you assign to an AWS Athena resource (a workgroup).Each tag consists of a key and an optional value, both of which you define. Tags enable you to categorize workgroups in Athena, for example,by purpose, owner, or environment. Use a consistent set of tag keys to make it easier to search and filter workgroups in your account.The maximum tag key length is 128 Unicode characters in UTF-8. The maximum tag value length is 256 Unicode characters in UTF-8.You can use letters and numbers representable in UTF-8, and the following characters:"+ - = . _ : / @". Tag keys and values are case-sensitive.Tag keys must be unique per resource. Please use the helper functiontag_options() to create tags for work group, if no tags are required please putNULL for this parameter.

A tag key. The tag key length is from 1 to 128 Unicode characters in UTF-8. You can use letters and numbers representable in UTF-8, and the following characters:"+ - = . _ : / @".Tag keys are case-sensitive and must be unique per resource.

A tag value. The tag value length is from 0 to 256 Unicode characters in UTF-8. You can use letters and numbers representable in UTF-8, and the following characters:"+ - = . _ : / @".Tag values are case-sensitive.

The option to delete the workgroup and its contents even if the workgroup contains any named queries

If set toTRUE, indicates that the previously-specified query results location (also known as a client-side setting) for queries in this workgroup should be ignored and set to null.If set toFALSE the out put location in the workgroup's result configuration will be updated with the new value.For more information, seeWorkgroup Settings Override Client-Side Settings.

The workgroup state that will be updated for the given workgroup.

The engine version requested when a workgroup is updated.

SelectedEngineVersion

The engine version requested by the user.

EffectiveEngineVersion

The engine version on which the query runs.