@npalm could I get your input on this?

Sorry won't be able to review any PR till the end of the month.

@damianrekosz can you check the commen from copilot. Can you also quickly explain the use case for different tags on the bucket.

can you check the commen from copilot.

The comment from Copilot isn't accurate. Removingflatten will cause an error because the arguments form a tuple. Also,runner_binaries_s3_tags will not produce a list of maps, as its type is set tomap(string).

Can you also quickly explain the use case for different tags on the bucket.

Yeah, sure. Some organizations may tag their S3 buckets differently - eg each bucket might be tagged with its name or department to provide better cost management visibility when custom tags in cost allocation are enabled.

Thanks for the suggestions. Unfortunately, without configuration at the runner level, we won't be able to assign different tags to different runner pools, such as arm64 and x64. This PR was meant to introduce that capability. The approach you suggested would only allow managing additional tags for S3 buckets, which wouldn't be sufficient, at least not in my organization.
For example with my changes, I could tag the arm64 bucket witharchitecture: arm64 and the x64 bucket witharchitecture: x64, while also defining global S3 tags at the module level to apply a common set of tags to all buckets - making the setup much more flexible.

Thanks for the suggestions. Unfortunately, without configuration at the runner level, we won't be able to assign different tags to different runner pools, such as arm64 and x64. This PR was meant to introduce that capability. The approach you suggested would only allow managing additional tags for S3 buckets, which wouldn't be sufficient, at least not in my organization. For example with my changes, I could tag the arm64 bucket witharchitecture: arm64 and the x64 bucket witharchitecture: x64, while also defining global S3 tags at the module level to apply a common set of tags to all buckets - making the setup much more flexible.

But the bucket is shared with multiple runners in case the architecture and os is the same. So I think mapping per runner config is confusing. Costs related to those buckets should be smalle. Since it is not about much data.

You're right, the cost isn't a major concern here, but organizations may have an AWS Config rule with auto-remediation enabled to tag all S3 buckets with the bucket's name. Such a configuration will cause drifts in the runner module, because without specifying the tags at the runner level, the drifts won't be resolved, causing the tags to be removed at every apply and re-applied by AWS Config.

You're right, the cost isn't a major concern here, but organizations may have an AWS Config rule with auto-remediation enabled to tag all S3 buckets with the bucket's name. Such a configuration will cause drifts in the runner module, because without specifying the tags at the runner level, the drifts won't be resolved, causing the tags to be removed at every apply and re-applied by AWS Config.

Yup, so what main is to apply the s3_tags directly to all bukcets created by the runner-binary module. And for multi runner not pass the tags in the runner-config. But as top-level variable in the multi-runner module. It was now on two places in that module.

Okay, let's investigate this case. I have a multi-runner configuration with 2 pools:

1. arm64
2. x64

The 2 pools will create separate S3 buckets - one for each architecture - and now I'd like to tag the created buckets with specific tags. Let's assume the following buckets were created:

1. multi-runner-linux-arm64-dist
2. multi-runner-linux-x64-dist

Now each bucket should have its ownBucketName tag to eliminate the drifts caused by my AWS Config auto-remediation rule that automatically tags new buckets.

I can now add the tags at the runner configuration level:

• formulti-runner-linux-arm64-dist:

  runner_binaries_s3_tags:    BucketName: multi-runner-linux-arm64-dist

• formulti-runner-linux-x64-dist

  runner_binaries_s3_tags:    BucketName: multi-runner-linux-x64-dist

which gives me the option to eliminate the drifts.
Defining s3_tags as a top-level variable would only allow assigning common tags to all buckets, not individual ones. That's why I'd like to add the option to define tags at the runner module level, in addition to the global s3_tags included in this PR.