kcreddy left a comment•

edited

Loading

…ata streamStorage of the response_elements, request_parameters and additional_eventdatais a potentially significant cost, but different users have differentrequirements for their present, so there is no ideal approach. Giventhat it is likely that this optimisation will be a common desire,provide a UI option to allow users to easily configure this behaviourwithout the requirement of adding processors to remove the fields in an@Custom pipeline. Note also that there is a TODO in the pipelineaddition here to move from a remove after creation model, spendingfruitless work, to a non-creation model, which would not be possible toimplement in an@Custom pipeline.

The fields were identified by running the following shell script in thethe security_detection_engine/kibana/security_rule directory.for f in *; do  jq 'select(.attributes.required_fields != null)|.attributes.required_fields|.[]|select(.name != null)|select(.name|contains("cloudtrail.flattened"))|.name'<$fdone|sort|uniqThe test for this is derived from the test-copy-object-json.log testcase which includes one of the required fields and a number of otherfields under cloudtrail.flattened. So comparing the test added here tothat demonstrates whether is works.

romulets left a comment