Fluent Bit is deployed as a DaemonSet (a POD that runs on every node of the cluster). When Fluent Bit runs, it will read, parse and filter the logs of every POD and will enrich log with some more information.

This will enable container (open policy agent) logs available in aws cloudwatch. The log group name where the logs will be /aws/containerinsights/${CLUSTER_NAME}/application ; here CLUSTER_NAME will be "tooling" for prod.

fluentbit.yaml will have

Here’s how the ClusterRole, ClusterRoleBinding, and ConfigMap are linked and their roles in this configuration:

ClusterRole

TheClusterRole namedfluent-bit-role defines the permissions that Fluent Bit requires to interact with Kubernetes resources. It specifies:

• Non-resource URL access: Allows access to/metrics with theget verb.
• Resource access: Grants permissions tonamespaces,pods,pods/logs,nodes, andnodes/proxy with theget,list, andwatch verbs.

ClusterRoleBinding

TheClusterRoleBinding namedfluent-bit-role-binding links theClusterRole to a subject, enabling Fluent Bit to use the permissions.

• Subject: TheServiceAccount namedfluent-bit in thelogging namespace.
• RoleRef: Specifies that the binding refers to thefluent-bit-role ClusterRole.

This linkage ensures that thefluent-bit ServiceAccount has the necessary permissions to collect logs and interact with Kubernetes objects.

ConfigMap

TheConfigMap namedfluent-bit-config provides configuration data for Fluent Bit. It contains:

• Fluent Bit configurations: Specifies input sources (e.g., application logs), filtering (e.g., Kubernetes metadata), and output destinations (e.g., CloudWatch Logs).
• Parser definitions: Defines parsers for structured log formats, such asdocker andsyslog.

How They Are Linked

1. Permissions for Log Access:

   • Thefluent-bit DaemonSet runs pods using thefluent-bit ServiceAccount.
   • Thefluent-bit-role-binding binds thefluent-bit-role ClusterRole to thefluent-bit ServiceAccount.
   • This setup allows Fluent Bit to access logs, Kubernetes metadata, and node information.

2. Configuration Data:

   • The DaemonSet mounts thefluent-bit-config ConfigMap to/fluent-bit/etc/ within its pods.
   • Fluent Bit reads configurations from this directory to process logs according to the defined rules.

This structure ensures Fluent Bit operates with the correct permissions and configurations in a Kubernetes environment. Let me know if you need further clarification or adjustments!