- Notifications
You must be signed in to change notification settings - Fork545
[AWS] Add dimensions to firewall, transit gateway and vpn data streams#6339
[AWS] Add dimensions to firewall, transit gateway and vpn data streams#6339constanca-m merged 11 commits intoelastic:mainfrom
Conversation
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
agithomas left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
LGTM!
packages/aws/changelog.yml Outdated
| @@ -1,4 +1,9 @@ | |||
| # newer versions go on top | |||
| - version: "1.36.9" | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
nit: in#6365 we bumped minor version, here only patch one
why the difference? :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
1.37 changes were not related to TSDB, that is why I bumped to 1.38 in that PR. Unlike here, since 1.36.8 was a change to add dimensions.
elasticmachine commentedJun 5, 2023 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
elasticmachine commentedJun 5, 2023 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
🌐 Coverage report
|
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
elasticmachine commentedJun 5, 2023
Package aws - 1.38.3 containing this change is available athttps://epr.elastic.co/search?package=aws |
#6339)* Add dimensions VPN.Signed-off-by: constanca-m <constanca.manteigas@elastic.co>* Add dimensions transit gateway.Signed-off-by: constanca-m <constanca.manteigas@elastic.co>* Add dimensions firewall.Signed-off-by: constanca-m <constanca.manteigas@elastic.co>* Update version.Signed-off-by: constanca-m <constanca.manteigas@elastic.co>* Update changelog.Signed-off-by: constanca-m <constanca.manteigas@elastic.co>* Add 2 ecs dimension fields.Signed-off-by: constanca-m <constanca.manteigas@elastic.co>* Remove cloud.availability_zone as dimension (not present)Signed-off-by: constanca-m <constanca.manteigas@elastic.co>* Update md.Signed-off-by: constanca-m <constanca.manteigas@elastic.co>* Resolve conflict.Signed-off-by: constanca-m <constanca.manteigas@elastic.co>---------Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
Uh oh!
There was an error while loading.Please reload this page.
What does this PR do?
Add dimensions to firewall, transit gateway and vpn data streams.
Transit gateway: A transit gateway can have more than one attachment, so it needs to be set as dimensions as well. A transit gateway is unique per AWS account.
VPN: Each tunnel belongs to a VPN. Combination tunnel id + vpn id is unique within the AWS scope account (account id + region).

Note: for each VPN site to site connection, there are four documents sent to ES. Is this expected?
Firewall: The firewall name is unique per region in an account. A firewall can have more than one engine and one Availability Zone, so both need to be set as dimension to avoid documents being overwritten. Since we can have more than one custom action in the same firewall, we also set that as dimension.
Question:
CustomActionfields holds the name of the action. Since a firewall can have different groups (names are unique, but we don't have access to that field in this data stream), then we can also have different custom actions with the exact same name associated with one firewall. I tried to reproduce this, but even with multiple rules with the same name I kept receiving just one document for each rule. Without considering TSDB, can we receive two documents for the same action at the same time? If so, dimensions are not enough for this data stream.Regarding ECS fields, more info can be found here#5193 (comment).
Checklist
changelog.ymlfile.Author's Checklist
How to test this PR locally
Refer to#6293.
Related issues
Relates to#6293.