Micronaut Object Storage

Micronaut Object Storage provides a uniform API to create, read and delete objects in the major cloud providers

Version: 2.9.0

1 Introduction

Micronaut Object Storage provides a uniform API to create, read and delete objects in the major cloud providers:

There is also alocal storage implementation for testing purposes.

Using this API enables the creation of truly multi-cloud, portable applications.

2 Release History

For this project, you can find a list of releases (with release notes) here:

https://github.com/micronaut-projects/micronaut-object-storage/releases

3 Quick Start

To get started, you need to declare a dependency for the actual cloud provider you are using. See the actual cloudprovider documentation for more details:

Then, you can inject in your controllers/services/etc. a bean of typeObjectStorageOperations, theparent interface that allows you to use the API in a generic way for all cloud providers:

@Singletonpublic class ProfileService {    private static final Logger LOG = LoggerFactory.getLogger(ProfileService.class);    private final ObjectStorageOperations<?, ?, ?> objectStorage;    public ProfileService(ObjectStorageOperations<?, ?, ?> objectStorage) {        this.objectStorage = objectStorage;    }}

@Singletonclass ProfileService {    final ObjectStorageOperations<?, ?, ?> objectStorage    ProfileService(ObjectStorageOperations<?, ?, ?> objectStorage) {        this.objectStorage = objectStorage    }}

@Singletonopen class ProfileService(private val objectStorage: ObjectStorageOperations<*, *, *>) {}

If your application is not multi-cloud, and/or you need cloud-specific details, you can use a concrete implementation.For example, for AWS S3:

@Controllerpublic class UploadController {    private final AwsS3Operations objectStorage;    public UploadController(AwsS3Operations objectStorage) {        this.objectStorage = objectStorage;    }}

@Controllerclass UploadController {    final AwsS3Operations objectStorage    UploadController(AwsS3Operations objectStorage) {        this.objectStorage = objectStorage    }}

@Controlleropen class UploadController(private val objectStorage: AwsS3Operations) {}

If you have multiple object storages configured, it is possible to select which one to work with viabean qualifiers.

For example, given the following configuration:

src/main/resources/application-ec2.yml

micronaut.object-storage.aws.pictures.bucket=pictures-bucketmicronaut.object-storage.aws.logos.bucket=logos-bucket

micronaut:  object-storage:    aws:      pictures:        bucket: pictures-bucket      logos:        bucket: logos-bucket

[micronaut]  [micronaut.object-storage]    [micronaut.object-storage.aws]      [micronaut.object-storage.aws.pictures]        bucket="pictures-bucket"      [micronaut.object-storage.aws.logos]        bucket="logos-bucket"

micronaut {  objectStorage {    aws {      pictures {        bucket = "pictures-bucket"      }      logos {        bucket = "logos-bucket"      }    }  }}

{  micronaut {    object-storage {      aws {        pictures {          bucket = "pictures-bucket"        }        logos {          bucket = "logos-bucket"        }      }    }  }}

{  "micronaut": {    "object-storage": {      "aws": {        "pictures": {          "bucket": "pictures-bucket"        },        "logos": {          "bucket": "logos-bucket"        }      }    }  }}

You then need to use@Named("pictures") or@Named("logos") to specify which of the object storages you want to use.

Uploading files

public String saveProfilePicture(String userId, Path path) {    UploadRequest request = UploadRequest.fromPath(path, userId); //(1)    UploadResponse<?> response = objectStorage.upload(request); //(2)    return response.getKey(); //(3)}

String saveProfilePicture(String userId, Path path) {    UploadRequest request = UploadRequest.fromPath(path, userId) //(1)    UploadResponse response = objectStorage.upload(request) //(2)    response.key //(3)}

open fun saveProfilePicture(userId: String, path: Path): String? {    val request = UploadRequest.fromPath(path, userId) //(1)    val response = objectStorage.upload(request) //(2)    return response.key //(3)}

1	You can use any of theUploadRequest static methods to build an upload request.
2	The upload operation returns anUploadResponse, which wraps the cloud-specific SDK responseobject.
3	The response object contains some common properties for all cloud vendor, and a`getNativeResponse()` method that canbe used for accessing the vendor-specific response object.

In case you want to have better control of the upload options used, you can use the methodupload(UploadRequest, Consumer) ofObjectStorageOperations, which will give you access to thecloud vendor-specific request class or builder.

For example, for AWS S3:

UploadResponse<PutObjectResponse> response = objectStorage.upload(objectStorageUpload, builder -> {    builder.acl(ObjectCannedACL.PUBLIC_READ);});

UploadResponse<PutObjectResponse> response = objectStorage.upload(objectStorageUpload, { builder ->    builder.acl(ObjectCannedACL.PUBLIC_READ)})

val response = objectStorage.upload(objectStorageUpload) { builder: PutObjectRequest.Builder ->    builder.acl(ObjectCannedACL.PUBLIC_READ)}

Retrieving files

public Optional<Path> retrieveProfilePicture(String userId, String fileName) {    Path destination = null;    try {        String key = userId + "/" + fileName;        Optional<InputStream> stream = objectStorage.retrieve(key) //(1)            .map(ObjectStorageEntry::getInputStream);        if (stream.isPresent()) {            destination = File.createTempFile(userId, "temp").toPath();            Files.copy(stream.get(), destination, StandardCopyOption.REPLACE_EXISTING);            return Optional.of(destination);        } else {            return Optional.empty();        }    } catch (IOException e) {        LOG.error("Error while trying to save profile picture to the local file [{}]: {}", destination, e.getMessage());        return Optional.empty();    }}

Optional<Path> retrieveProfilePicture(String userId, String fileName) {    String key = "${userId}/${fileName}"    Optional<InputStream> stream = objectStorage.retrieve(key) //(1)            .map(ObjectStorageEntry::getInputStream)    if (stream.isPresent()) {        Path destination = File.createTempFile(userId, "temp").toPath()        Files.copy(stream.get(), destination, StandardCopyOption.REPLACE_EXISTING)        return Optional.of(destination)    } else {        return Optional.empty()    }}

open fun retrieveProfilePicture(userId: String, fileName: String): Path? {    val key = "$userId/$fileName"    val stream = objectStorage.retrieve<ObjectStorageEntry<*>>(key) //(1)        .map { obj: ObjectStorageEntry<*> -> obj.inputStream }    return if (stream.isPresent) {        val destination = File.createTempFile(userId, "temp").toPath()        Files.copy(stream.get(), destination, StandardCopyOption.REPLACE_EXISTING)        destination    } else {        null    }}

1	The retrieve operation returns anObjectStorageEntry, from which you can get an`InputStream`.There is also a`getNativeEntry()` method that gives you access to the cloud vendor-specific response object.

Deleting files

public void deleteProfilePicture(String userId, String fileName) {    String key = userId + "/" + fileName;    objectStorage.delete(key); //(1)}

void deleteProfilePicture(String userId, String fileName) {    String key = "${userId}/${fileName}"    objectStorage.delete(key) //(1)}

open fun deleteProfilePicture(userId: String, fileName: String) {    val key = "$userId/$fileName"    objectStorage.delete(key) //(1)}

1	The delete operation returns the cloud vendor-specific delete response object in case you need it.

4 Amazon S3

To useAmazon S3, you need the following dependency:

implementation("io.micronaut.objectstorage:micronaut-object-storage-aws")

<dependency>    <groupId>io.micronaut.objectstorage</groupId>    <artifactId>micronaut-object-storage-aws</artifactId></dependency>

Refer to theMicronaut AWS documentation for moreinformation about credentials and region configuration.

The object storage specific configuration options available are:

🔗

Table 1. Configuration Properties forAwsS3Configuration
Property	Type	Description
`micronaut.object-storage.aws.*.enabled`	boolean	Whether to enable or disable this object storage.
`micronaut.object-storage.aws.*.bucket`	java.lang.String	The name of the AWS S3 bucket.

For example:

src/main/resources/application-ec2.yml

micronaut.object-storage.aws.default.bucket=profile-pictures-bucket

micronaut:  object-storage:    aws:      default:        bucket: profile-pictures-bucket

[micronaut]  [micronaut.object-storage]    [micronaut.object-storage.aws]      [micronaut.object-storage.aws.default]        bucket="profile-pictures-bucket"

micronaut {  objectStorage {    aws {      'default' {        bucket = "profile-pictures-bucket"      }    }  }}

{  micronaut {    object-storage {      aws {        default {          bucket = "profile-pictures-bucket"        }      }    }  }}

{  "micronaut": {    "object-storage": {      "aws": {        "default": {          "bucket": "profile-pictures-bucket"        }      }    }  }}

The concrete implementation ofObjectStorageOperations isAwsS3Operations.

Advanced configuration

For configuration properties other than the specified above, you can add bean to your application that implementsBeanCreatedEventListener. For example:

@Singletonpublic class S3ClientBuilderCustomizer implements BeanCreatedEventListener<S3ClientBuilder> {    @Override    public S3ClientBuilder onCreated(@NonNull BeanCreatedEvent<S3ClientBuilder> event) {        return event.getBean()            .overrideConfiguration(c -> c.apiCallTimeout(Duration.of(60, ChronoUnit.SECONDS)));    }}

See the guide forUse the Micronaut Object Storage API to Store Files in Amazon S3 to learn more.

5 Azure Blob Storage

To useAzure Blob Storage, you need the following dependency:

implementation("io.micronaut.objectstorage:micronaut-object-storage-azure")

<dependency>    <groupId>io.micronaut.objectstorage</groupId>    <artifactId>micronaut-object-storage-azure</artifactId></dependency>

Refer to theMicronaut Azure documentation for moreinformation about authentication options.

The object storage specific configuration options available are:

🔗

Table 1. Configuration Properties forAzureBlobStorageConfiguration
Property	Type	Description
`micronaut.object-storage.azure.*.enabled`	boolean	Whether to enable or disable this object storage.
`micronaut.object-storage.azure.*.container`	java.lang.String	The blob container name.
`micronaut.object-storage.azure.*.endpoint`	java.lang.String	The blob service endpoint, in the format ofhttps://{accountName}.blob.core.windows.net.

For example:

src/main/resources/application-azure.yml

azure.credential.client-secret.client-id=<client-id>azure.credential.client-secret.tenant-id=<tenant-id>azure.credential.client-secret.secret=<secret>micronaut.object-storage.azure.default.container=profile-pictures-containermicronaut.object-storage.azure.default.endpoint=https://my-account.blob.core.windows.net

azure:  credential:    client-secret:      client-id: <client-id>      tenant-id: <tenant-id>      secret: <secret>micronaut:  object-storage:    azure:      default:        container: profile-pictures-container        endpoint: https://my-account.blob.core.windows.net

[azure]  [azure.credential]    [azure.credential.client-secret]      client-id="<client-id>"      tenant-id="<tenant-id>"      secret="<secret>"[micronaut]  [micronaut.object-storage]    [micronaut.object-storage.azure]      [micronaut.object-storage.azure.default]        container="profile-pictures-container"        endpoint="https://my-account.blob.core.windows.net"

azure {  credential {    clientSecret {      clientId = "<client-id>"      tenantId = "<tenant-id>"      secret = "<secret>"    }  }}micronaut {  objectStorage {    azure {      'default' {        container = "profile-pictures-container"        endpoint = "https://my-account.blob.core.windows.net"      }    }  }}

{  azure {    credential {      client-secret {        client-id = "<client-id>"        tenant-id = "<tenant-id>"        secret = "<secret>"      }    }  }  micronaut {    object-storage {      azure {        default {          container = "profile-pictures-container"          endpoint = "https://my-account.blob.core.windows.net"        }      }    }  }}

{  "azure": {    "credential": {      "client-secret": {        "client-id": "<client-id>",        "tenant-id": "<tenant-id>",        "secret": "<secret>"      }    }  },  "micronaut": {    "object-storage": {      "azure": {        "default": {          "container": "profile-pictures-container",          "endpoint": "https://my-account.blob.core.windows.net"        }      }    }  }}

The concrete implementation ofObjectStorageOperations isAzureBlobStorageOperations.

Advanced configuration

For configuration properties other than the specified above, you can add bean to your application that implementsBeanCreatedEventListener. For example:

@Singletonpublic class BlobServiceClientBuilderCustomizer implements BeanCreatedEventListener<BlobServiceClientBuilder> {    @Override    public BlobServiceClientBuilder onCreated(@NonNull BeanCreatedEvent<BlobServiceClientBuilder> event) {        HttpPipelinePolicy noOp = (context, next) -> next.process();        return event.getBean().addPolicy(noOp);    }}

6 Google Cloud Storage

To useGoogle Cloud Storage, you need the following dependency:

implementation("io.micronaut.objectstorage:micronaut-object-storage-gcp")

<dependency>    <groupId>io.micronaut.objectstorage</groupId>    <artifactId>micronaut-object-storage-gcp</artifactId></dependency>

Refer to theMicronaut GCP documentation for moreinformation about configuring your GCP project.

The object storage specific configuration options available are:

🔗

Table 1. Configuration Properties forGoogleCloudStorageConfiguration
Property	Type	Description
`micronaut.object-storage.gcp.*.enabled`	boolean	Whether to enable or disable this object storage.
`micronaut.object-storage.gcp.*.bucket`	java.lang.String	The name of the Google Cloud Storage bucket.

For example:

src/main/resources/application-gcp.yml

gcp.project-id=my-gcp-projectmicronaut.object-storage.gcp.default.bucket=profile-pictures-bucket

gcp:  project-id: my-gcp-projectmicronaut:  object-storage:    gcp:      default:        bucket: profile-pictures-bucket

[gcp]  project-id="my-gcp-project"[micronaut]  [micronaut.object-storage]    [micronaut.object-storage.gcp]      [micronaut.object-storage.gcp.default]        bucket="profile-pictures-bucket"

gcp {  projectId = "my-gcp-project"}micronaut {  objectStorage {    gcp {      'default' {        bucket = "profile-pictures-bucket"      }    }  }}

{  gcp {    project-id = "my-gcp-project"  }  micronaut {    object-storage {      gcp {        default {          bucket = "profile-pictures-bucket"        }      }    }  }}

{  "gcp": {    "project-id": "my-gcp-project"  },  "micronaut": {    "object-storage": {      "gcp": {        "default": {          "bucket": "profile-pictures-bucket"        }      }    }  }}

The concrete implementation ofObjectStorageOperations isGoogleCloudStorageOperations.

Advanced configuration

For configuration properties other than the specified above, you can add bean to your application that implementsBeanCreatedEventListener. For example:

@Singletonpublic class StorageOptionsBuilderCustomizer implements BeanCreatedEventListener<StorageOptions.Builder> {    @Override    public StorageOptions.Builder onCreated(@NonNull BeanCreatedEvent<StorageOptions.Builder> event) {        return event.getBean()            .setTransportOptions(HttpTransportOptions.newBuilder().setConnectTimeout(60_000).build());    }}

See the guide forUse the Micronaut Object Storage API to Store Files in Google Cloud Storage to learn more.

7 Oracle Cloud Infrastructure (OCI) Object Storage

To useOracle Cloud Infrastructure (OCI) Object Storage, you need the following dependency:

implementation("io.micronaut.objectstorage:micronaut-object-storage-oracle-cloud")

<dependency>    <groupId>io.micronaut.objectstorage</groupId>    <artifactId>micronaut-object-storage-oracle-cloud</artifactId></dependency>

Refer to theMicronaut Oracle Cloud documentationfor more information about authentication options.

The object storage specific configuration options available are:

🔗

Table 1. Configuration Properties forOracleCloudStorageConfiguration
Property	Type	Description
`micronaut.object-storage.oracle-cloud.*.enabled`	boolean	Whether to enable or disable this object storage.
`micronaut.object-storage.oracle-cloud.*.bucket`	java.lang.String	The name of the OCI Object Storage bucket.
`micronaut.object-storage.oracle-cloud.*.namespace`	java.lang.String	The OCI Object Storage namespace used.

For example:

src/main/resources/application-oraclecloud.yml

oci.config.profile=DEFAULTmicronaut.object-storage.oracle-cloud.default.bucket=profile-pictures-bucketmicronaut.object-storage.oracle-cloud.default.namespace=MyNamespace

oci:  config:    profile: DEFAULTmicronaut:  object-storage:    oracle-cloud:      default:        bucket: profile-pictures-bucket        namespace: MyNamespace

[oci]  [oci.config]    profile="DEFAULT"[micronaut]  [micronaut.object-storage]    [micronaut.object-storage.oracle-cloud]      [micronaut.object-storage.oracle-cloud.default]        bucket="profile-pictures-bucket"        namespace="MyNamespace"

oci {  config {    profile = "DEFAULT"  }}micronaut {  objectStorage {    oracleCloud {      'default' {        bucket = "profile-pictures-bucket"        namespace = "MyNamespace"      }    }  }}

{  oci {    config {      profile = "DEFAULT"    }  }  micronaut {    object-storage {      oracle-cloud {        default {          bucket = "profile-pictures-bucket"          namespace = "MyNamespace"        }      }    }  }}

{  "oci": {    "config": {      "profile": "DEFAULT"    }  },  "micronaut": {    "object-storage": {      "oracle-cloud": {        "default": {          "bucket": "profile-pictures-bucket",          "namespace": "MyNamespace"        }      }    }  }}

The concrete implementation ofObjectStorageOperations isOracleCloudStorageOperations

Advanced configuration

For configuration properties other than the specified above, you can add bean to your application that implementsBeanCreatedEventListener. For example:

//See https://github.com/oracle/oci-java-sdk/blob/master/bmc-examples/src/main/java/ClientConfigurationTimeoutExample.java@Singletonpublic class ObjectStorageClientBuilderCustomizer implements BeanCreatedEventListener<ObjectStorageClient.Builder> {    public static final int CONNECTION_TIMEOUT_IN_MILLISECONDS = 25000;    public static final int READ_TIMEOUT_IN_MILLISECONDS = 35000;    @Override    public ObjectStorageClient.Builder onCreated(@NonNull BeanCreatedEvent<ObjectStorageClient.Builder> event) {        ClientConfiguration clientConfiguration =            ClientConfiguration.builder()                .connectionTimeoutMillis(CONNECTION_TIMEOUT_IN_MILLISECONDS)                .readTimeoutMillis(READ_TIMEOUT_IN_MILLISECONDS)                .build();        return event.getBean()            .configuration(clientConfiguration);    }}

See the guide forUse the Micronaut Object Storage API to Store Files in Oracle Cloud Infrastructure (OCI) Object Storage to learn more.

8 Local Storage

To use the local storage implementation (useful for tests), you need the following dependency:

testImplementation("io.micronaut.objectstorage:micronaut-object-storage-local")

<dependency>    <groupId>io.micronaut.objectstorage</groupId>    <artifactId>micronaut-object-storage-local</artifactId>    <scope>test</scope></dependency>

Then, simply define a local storage:

micronaut.object-storage.local.default.enabled=true

micronaut:  object-storage:    local:      default:        enabled: true

[micronaut]  [micronaut.object-storage]    [micronaut.object-storage.local]      [micronaut.object-storage.local.default]        enabled=true

micronaut {  objectStorage {    local {      'default' {        enabled = true      }    }  }}

{  micronaut {    object-storage {      local {        default {          enabled = true        }      }    }  }}

{  "micronaut": {    "object-storage": {      "local": {        "default": {          "enabled": true        }      }    }  }}

When added to the classpath,LocalStorageOperations becomes the primary implementation ofObjectStorageOperations.

By default, it will create a temporary folder to store the files, but you can configure it to use a specific folder:

🔗

Table 1. Configuration Properties forLocalStorageConfiguration
Property	Type	Description
`micronaut.object-storage.local.*.enabled`	boolean	Whether to enable or disable this object storage.
`micronaut.object-storage.local.*.path`	java.nio.file.Path	The path of the local storage.

For example:

src/main/resources/application-test.yml

micronaut.object-storage.local.default.path=/tmp/my-object-storage

micronaut:  object-storage:    local:      default:        path: /tmp/my-object-storage

[micronaut]  [micronaut.object-storage]    [micronaut.object-storage.local]      [micronaut.object-storage.local.default]        path="/tmp/my-object-storage"

micronaut {  objectStorage {    local {      'default' {        path = "/tmp/my-object-storage"      }    }  }}

{  micronaut {    object-storage {      local {        default {          path = "/tmp/my-object-storage"        }      }    }  }}

{  "micronaut": {    "object-storage": {      "local": {        "default": {          "path": "/tmp/my-object-storage"        }      }    }  }}

The concrete implementation ofObjectStorageOperations isLocalStorageOperations.

9 Guides

See the following list of guides to learn more about working with Object Storage in the Micronaut Framework:

https://guides.micronaut.io/latest/tag-object_storage.html

10 Repository

You can find the source code of this project in this repository:

https://github.com/micronaut-projects/micronaut-object-storage

Movatterモバイル変換

Table of Contents

Micronaut Object Storage

1 Introduction

2 Release History

3 Quick Start

Uploading files

Retrieving files

Deleting files

4 Amazon S3

Advanced configuration

5 Azure Blob Storage

Advanced configuration

6 Google Cloud Storage

Advanced configuration

7 Oracle Cloud Infrastructure (OCI) Object Storage

Advanced configuration

8 Local Storage

9 Guides

10 Repository