|
| 1 | +--- |
| 2 | +title:"Cloud Storage pipeline integrations" |
| 3 | +description:"How to use Codefresh with Cloud Storage providers" |
| 4 | +group:integrations |
| 5 | +toc:true |
| 6 | +--- |
| 7 | + |
| 8 | +Codefresh integrations with cloud storage providers provide a convenient solution for storing test reports. |
| 9 | +With Codefresh, you can easily configure your pipelines to store test reports in your preferred Cloud Storage provider, such as Amazon S3, Google Cloud Storage, Azure, and MinIO. |
| 10 | + |
| 11 | +For every cloud storage provider, you need to first create a storage bucket in your storage provider account, connect the account with Codefresh to create an integration, and configure your pipelines to[create and upload test reports]({{site.baseurl}}/docs/testing/test-reports/). |
| 12 | + |
| 13 | +##Connecting your storage account to Codefresh |
| 14 | + |
| 15 | +When you connect your storage provider account to Codefresh, Codefresh creates subfolders in the storage bucket for every build, with the build IDs as folder names. Test reports generated for a build are uploaded to the respective folder. The same bucket can store test reports from multiple pipeline builds. |
| 16 | + |
| 17 | +1. In the Codefresh UI, on the toolbar, click the Settings icon, and then from the sidebar select**Pipeline Integrations**. |
| 18 | +1. Scroll down to**Cloud Storage**, and click**Configure**. |
| 19 | + |
| 20 | + |
| 21 | +{% include |
| 22 | +image.html |
| 23 | +lightbox="true" |
| 24 | +file="/images/pipeline/test-reports/cloud-storage-integrations.png" |
| 25 | +url="/images/pipeline/test-reports/cloud-storage-integrations.png" |
| 26 | +alt="Cloud storage Integrations" |
| 27 | +caption="Cloud storage Integrations" |
| 28 | +max-width="80%" |
| 29 | +%} |
| 30 | + |
| 31 | +{:start="3"} |
| 32 | +1. Click**Add Cloud Storage**, and select your cloud provider for test report storage. |
| 33 | +1. Define settings for your cloud storage provider, as described in the sections that follow. |
| 34 | + |
| 35 | +##Connecting a Google bucket |
| 36 | + |
| 37 | +**In Google** |
| 38 | + |
| 39 | +1. Create a bucket either from the Google cloud console or the`gsutil` command line tool. |
| 40 | + See the[official documentation](https://cloud.google.com/storage/docs/creating-buckets#storage-create-bucket-console){:target="\_blank"} for the exact details. |
| 41 | + |
| 42 | +**In Codefresh** |
| 43 | +1.[Connect your storage account](#connecting-your-storage-account) and select**Google Cloud Storage**. |
| 44 | + |
| 45 | +{% include |
| 46 | +image.html |
| 47 | +lightbox="true" |
| 48 | +file="/images/pipeline/test-reports/cloud-storage-google.png" |
| 49 | +url="/images/pipeline/test-reports/cloud-storage-google.png" |
| 50 | +alt="Google cloud storage" |
| 51 | +caption="Google cloud storage" |
| 52 | +max-width="80%" |
| 53 | +%} |
| 54 | + |
| 55 | +{:start="2"} |
| 56 | +1. Define the settings: |
| 57 | +* Select**OAuth2** as the connection method, which is the easiest way. |
| 58 | +* Enter an arbitrary name for your integration. |
| 59 | +* Select**Allow access to read and write into storage** as Codefresh needs to both write to and read from the bucket. |
| 60 | +1. Click**Save**. |
| 61 | +1. When Codefresh asks for extra permissions from your Google account, accept the permissions. |
| 62 | + |
| 63 | +The integration is ready. You will use the name of the integration as an environment variable in your Codefresh pipeline. |
| 64 | + |
| 65 | +>**NOTE**: |
| 66 | + An alternative authentication method is to use**JSON Config** with a[Google service account key](https://console.cloud.google.com/apis/credentials/serviceaccountkey){:target="\_blank"}. |
| 67 | + In that case, download the JSON file locally and paste its contents in the**JSON config** field. |
| 68 | + For more information, see the[official documentation](https://cloud.google.com/iam/docs/creating-managing-service-account-keys){:target="\_blank"}. |
| 69 | + |
| 70 | +##Connecting an Amazon S3 bucket |
| 71 | + |
| 72 | +**Create an S3 bucket in AWS (Amazon Web Services)** |
| 73 | + |
| 74 | +1. Create an S3 bucket in AWS. |
| 75 | + See the[official documentation](https://docs.aws.amazon.com/quickstarts/latest/s3backup/step-1-create-bucket.html){:target="\_blank"}, or use the[AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/s3api/create-bucket.html){:target="\_blank"}. |
| 76 | +1. Define the necessary IAM (Identity and Access Management) policy settings. |
| 77 | + Here's an example IAM policy that you can use as a reference: |
| 78 | +``` |
| 79 | +{ |
| 80 | + "Version": "2012-10-17", |
| 81 | + "Statement": [ |
| 82 | + { |
| 83 | + "Effect": "Allow", |
| 84 | + "Action": [ |
| 85 | + "s3:ListBucket" |
| 86 | + ], |
| 87 | + "Resource": [ |
| 88 | + "arn:aws:s3:::cf-backup*" |
| 89 | + ] |
| 90 | + }, |
| 91 | + { |
| 92 | + "Effect": "Allow", |
| 93 | + "Action": [ |
| 94 | + "s3:PutObject", |
| 95 | + "s3:GetObject", |
| 96 | + "s3:DeleteObject" |
| 97 | + ], |
| 98 | + "Resource": [ |
| 99 | + "arn:aws:s3:::cf-backup*/*" |
| 100 | + ] |
| 101 | + } |
| 102 | + ] |
| 103 | +} |
| 104 | +``` |
| 105 | + |
| 106 | +1. Note down the**Access** and**Secret** keys generated when you created the S3 bucket. |
| 107 | + |
| 108 | +**Define S3 settings in Codefresh** |
| 109 | +1. Select**Amazon Cloud Storage** as your[Cloud Storage provider](#connecting-your-storage-account). |
| 110 | +1. Define the settings: |
| 111 | +* Enter an arbitrary name for your integration. |
| 112 | +* Paste the**AWS Access Key ID** and**AWS Secret Access Key**. |
| 113 | +1. Click**Save**. |
| 114 | + |
| 115 | +{% include |
| 116 | +image.html |
| 117 | +lightbox="true" |
| 118 | +file="/images/pipeline/test-reports/cloud-storage-s3.png" |
| 119 | +url="/images/pipeline/test-reports/cloud-storage-s3.png" |
| 120 | +alt="S3 cloud storage" |
| 121 | +caption="S3 cloud storage" |
| 122 | +max-width="80%" |
| 123 | +%} |
| 124 | + |
| 125 | +After setting up and verifying the S3 bucket integration, you can use: |
| 126 | +* The name of the integration as an environment variable in your Codefresh pipeline. |
| 127 | +* Any[external secrets that you have defined]({{site.baseurl}}/docs/integrations/secret-storage/) (such as Kubernetes secrets), as values, by clicking on the lock icon that appears next to field: |
| 128 | +* If you have already specified the resource field during secret definition, just enter the name of the secret directly in the text field, for example,`my-secret-key`. |
| 129 | +* If you didn't include a resource name during secret creation, enter the full name in the field, for example,`my-secret-resource@my-secret-key`. |
| 130 | + |
| 131 | +##Connecting Azure Blob/File storage |
| 132 | + |
| 133 | +**Create a storage account in Azure** |
| 134 | + |
| 135 | +1. For Azure, create a storage account. |
| 136 | + See the[official documentation](https://docs.microsoft.com/en-us/azure/storage/common/storage-account-create){:target="\_blank"}. |
| 137 | +1. Find one of the[two access keys](https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage){:target="\_blank"} already created. |
| 138 | +1. Note down the**Account Name** and**Access key for the account**. |
| 139 | + |
| 140 | +**Define Azure settings in Codefresh** |
| 141 | +1. Select**Azure File/Blob Storage** as your[Cloud Storage provider](#connecting-your-storage-account). |
| 142 | +1. Define the settings: |
| 143 | +* Enter an arbitrary name for your integration. |
| 144 | +* Paste the**Azure Account Name** and**Azure Account Key**. |
| 145 | +1. Click**Save**. |
| 146 | + |
| 147 | + |
| 148 | +{% include |
| 149 | +image.html |
| 150 | +lightbox="true" |
| 151 | +file="/images/pipeline/test-reports/cloud-storage-azure.png" |
| 152 | +url="/images/pipeline/test-reports/cloud-storage-azure.png" |
| 153 | +alt="Azure cloud storage" |
| 154 | +caption="Azure cloud storage" |
| 155 | +max-width="60%" |
| 156 | +%} |
| 157 | + |
| 158 | +After setting up and verifying the Azure File/Blob integration, you can use: |
| 159 | +* The name of the integration as an environment variable in your Codefresh pipeline. |
| 160 | +* Any[external secrets that you have defined]({{site.baseurl}}/docs/integrations/secret-storage/) (such as Kubernetes secrets), as values, by clicking on the lock icon that appears next to field: |
| 161 | +* If you have already specified the resource field during secret definition, just enter the name of the secret directly in the text field, for example,`my-secret-key`. |
| 162 | +* If you didn't include a resource name during secret creation, enter the full name in the field, for example,`my-secret-resource@my-secret-key`. |
| 163 | + |
| 164 | + |
| 165 | +##Connecting MinIO storage |
| 166 | + |
| 167 | +**Create a storage account in MinIO** |
| 168 | +1. Configure the MinIO server. |
| 169 | + See the[official documentation](https://docs.min.io/docs/minio-quickstart-guide.html){:target="\_blank"}. |
| 170 | +1. Copy the Access and Secret keys. |
| 171 | + |
| 172 | +**Set up a MinIO integration in Codefresh** |
| 173 | + |
| 174 | +1. Select**MinIO Cloud Storage** as your[Cloud Storage provider](#connecting-your-storage-account). |
| 175 | +1. Define the settings: |
| 176 | +***NAME**: The name of the MinIO storage. Any name that is meaningful to you. |
| 177 | +***ENDPOINT**: The URL to the storage service object. |
| 178 | +***PORT**: Optional. The TCP/IP port number. If not defined, defaults to port`80` for HTTP, and`443` for HTTPS. |
| 179 | +***Minio Access Key**: The ID that uniquely identifies your account, similar to a user ID. |
| 180 | +***Secret Minio Key**: The password of your account. |
| 181 | +***Use SSL**: Select to enable secure HTTPS access. Not selected by default. |
| 182 | +1. Click**Save**. |
| 183 | + |
| 184 | + {% include |
| 185 | +image.html |
| 186 | +lightbox="true" |
| 187 | +file="/images/pipeline/test-reports/cloud-storage-minio.png" |
| 188 | +url="/images/pipeline/test-reports/cloud-storage-minio.png" |
| 189 | +alt="MinIO cloud storage" |
| 190 | +caption="MinIO cloud storage" |
| 191 | +max-width="60%" |
| 192 | +%} |
| 193 | + |
| 194 | + |
| 195 | +##Related articles |
| 196 | +[Amazon Web Services (AWS) pipeline integration]({{site.baseurl}}/docs/integrations/amazon-web-services/) |
| 197 | +[Microsoft Azure pipeline integration]({{site.baseurl}}/docs/integrations/microsoft-azure/) |
| 198 | +[Google Cloud pipeline integration]({{site.baseurl}}/docs/integrations/google-cloud/) |
| 199 | +[Creating test reports]({{site.baseurl}}/docs/testing/test-reports/) |
| 200 | +[Codefresh YAML for pipeline definitions]({{site.baseurl}}/docs/pipelines/what-is-the-codefresh-yaml/) |
| 201 | +[Steps in pipelines]({{site.baseurl}}/docs/pipelines/steps/) |