- Notifications
You must be signed in to change notification settings - Fork545
Comments
Spring boot package [Memory - data stream]#2979
Conversation
elasticmachine commentedApr 3, 2022 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
💚 Build Succeeded
Expand to view the summaryBuild stats
Test stats 🧪
🤖 GitHub commentsTo re-run your PR in the CI, just comment with:
|
elasticmachine commentedApr 3, 2022
Pinging @elastic/integrations (Team:Integrations) |
| "memory_used": 81920, | ||
| "total_capacity": 81920 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Used and total are the same? Does it mean that you have 100% consumed the memory?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Yes.
Uh oh!
There was an error while loading.Please reload this page.
| @@ -0,0 +1,74 @@ | |||
| { | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Could you dump an event that collects most of defined metrics?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
updated.
| type: long | ||
| - name: object_pending_finalization_count | ||
| type: long | ||
| - name: verbose |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
What does the verbose mean here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Verbose: Tests if verbose output for the memory system is enabled. But I think its a boolean value and does not make any sense to collect that so we removed it.
| fields: | ||
| - name: count | ||
| type: long | ||
| description: Returns count of direct buffer pool memory |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Could you please make all descriptions consistent? Why is there everywhere a verb "Returns"? Let's make it consistent with other data streams.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Yes its updated
| fields: | ||
| - name: count | ||
| type: long | ||
| description: Returns count of mapped buffer pool memory |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
What is this metric about?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Count of buffer pool memory
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Still, what does it mean :) Is it the number of buffer pools, number of memory units, etc.?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
yes it is number of buffer pools
packages/spring_boot/docs/README.md Outdated
| | spring_boot.memory.buffer_pool.mapped.total_capacity | Returns total capacity of mapped buffer pool memory | long | | ||
| | spring_boot.memory.memory.heap_memory_usage.committed | Returns committed heap memory usage of JVM | long | | ||
| | spring_boot.memory.memory.heap_memory_usage.init | Returns init heap memory usage of JVM | long | | ||
| | spring_boot.memory.memory.heap_memory_usage.max | Returns max heap memory usage of JVM | long | | ||
| | spring_boot.memory.memory.heap_memory_usage.used | Returns used heap memory usage of JVM | long | | ||
| | spring_boot.memory.memory.non_heap_memory_usage.committed | Returns committed non-heap memory usage of JVM | long | | ||
| | spring_boot.memory.memory.non_heap_memory_usage.init | Returns init non-heap memory usage of JVM | long | | ||
| | spring_boot.memory.memory.non_heap_memory_usage.max | Returns max non-heap memory usage of JVM | long | | ||
| | spring_boot.memory.memory.non_heap_memory_usage.used | Returns used non-heap memory usage of JVM | long | | ||
| | spring_boot.memory.memory.object_pending_finalization_count | Returns the approximate number of objects for which finalization is pending | long | | ||
| | spring_boot.memory.memory.verbose | Tests if verbose output for the memory system is enabled | boolean | | ||
| | spring_boot.memory.memory_manager.code_cache_manager.name | Name of the cacheManager to qualify the cache | keyword | | ||
| | spring_boot.memory.memory_manager.code_cache_manager.valid | Return the validation | boolean | | ||
| | spring_boot.memory.memory_manager.metaspace_manager.name | Name of the Metaspace Manager to qualify the cache | keyword | | ||
| | spring_boot.memory.memory_manager.metaspace_manager.valid | Return the validation | boolean | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Please work together with@yug-elastic to refactor those metrics.memory is repeated a few times per metric.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Sure, putting the link to the similar discussion for Apache Spark here for ready reference
#2811 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Its updated
| This is the `memory` data stream. | ||
| - This data stream gives Memory information. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
nit: memory
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Done
| name: source.geo.location | ||
| type: geo_point | ||
| - external: ecs | ||
| name: source.geo.region_iso_code | ||
| - external: ecs | ||
| name: source.geo.region_name | ||
| - external: ecs | ||
| name: source.ip | ||
| - external: ecs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This data stream is about memory, so why do we need all these field definitions? Process, source, tls, HTTP. Did you add them on purpose?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
yeah removed those extra ecs fields
| This is the `memory` data stream. | ||
| - This data stream gives memory information. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I don't think that this description explains what kind of data the data stream is handling. Please rephrase it to something meaningful (what kind of metrics, from where).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Ok updated
packages/spring_boot/docs/README.md Outdated
| | os.name | Operating system name, without the version. | keyword | | ||
| | os.name.text | Multi-field of `os.name`. | match_only_text | | ||
| | os.version | Operating system version as a raw string. | keyword | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Sample event doesn't contain metrics for these entries. Does the integration collect these values?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
missed it from Readme, but its updated now
packages/spring_boot/docs/README.md Outdated
| | event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword | | ||
| | event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. `event.kind` gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data coming in at a regular interval or not. | keyword | | ||
| | event.module | Name of the module this data is coming from. If your monitoring agent supports the concept of modules or plugins to process events of a given source (e.g. Apache logs), `event.module` should contain the name of this module. | keyword | | ||
| | event.outcome | This is one of four ECS Categorization Fields, and indicates the lowest level in the ECS category hierarchy. `event.outcome` simply denotes whether the event represents a success or a failure from the perspective of the entity that produced the event. Note that when a single transaction is described in multiple events, each event may populate different values of `event.outcome`, according to their perspective. Also note that in the case of a compound event (a single event that contains multiple logical events), this field should be populated with the value that best captures the overall success or failure from the perspective of the event producer. Further note that not all events will have an associated outcome. For example, this field is generally not populated for metric events, events with `event.type:info`, or any events for which an outcome does not make logical sense. | keyword | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Same
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
yes updated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
What did you update here? I can still seeevent.outcome and I can't see in the sample event. Could you please explain why is this field here?
packages/spring_boot/docs/README.md Outdated
| | event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. `event.kind` gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data coming in at a regular interval or not. | keyword | | ||
| | event.module | Name of the module this data is coming from. If your monitoring agent supports the concept of modules or plugins to process events of a given source (e.g. Apache logs), `event.module` should contain the name of this module. | keyword | | ||
| | event.outcome | This is one of four ECS Categorization Fields, and indicates the lowest level in the ECS category hierarchy. `event.outcome` simply denotes whether the event represents a success or a failure from the perspective of the entity that produced the event. Note that when a single transaction is described in multiple events, each event may populate different values of `event.outcome`, according to their perspective. Also note that in the case of a compound event (a single event that contains multiple logical events), this field should be populated with the value that best captures the overall success or failure from the perspective of the event producer. Further note that not all events will have an associated outcome. For example, this field is generally not populated for metric events, events with `event.type:info`, or any events for which an outcome does not make logical sense. | keyword | | ||
| | message | For log events the message field contains the log message, optimized for viewing in a log viewer. For structured logs without an original message field, other fields can be concatenated to form a human-readable summary of the event. If multiple messages exist, they can be combined into one message. | match_only_text | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Same question. It's a metric based data stream. Why is there a message?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
yeah its cleared
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
kush-elastic left a comment• edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
LGTM! But let's wait for others approval.
mtojek left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Just a formality on my side. Thanks,@kush-elastic, for reviewing.
…ations into feature/labels-cloud-ephemeral-components* 'feature/labels-cloud-ephemeral-components' of github.com:v1v/integrations: (23 commits) use branch standardise labels/tags Update .ci/Jenkinsfile [ci][terraform][aws] tags with metadata Bump github.com/elastic/elastic-package from 0.46.0 to 0.47.0 (elastic#3182) Spring boot package [Memory - data stream] (elastic#2979) [cisco_ise] Add Cisco ISE package (elastic#2855) [apache_spark][executor] Add Apache Spark package with Executor data stream (elastic#2943) [apache_spark][driver] Add Apache Spark package with Driver data stream (elastic#2945) cisco_duo: simplify grok expression for handling ports (elastic#3170) cisco_duo: fix handling of IP addresses with port numbers (elastic#3117) [sophos] Various improvements and log samples from 18.5 (elastic#3127) [cisco_asa] fix visualizations (elastic#3146) [apache_spark][application] Add Apache Spark package with Application data stream (elastic#2941) Fix pagination bug that skipped events when more than one page is present. (elastic#3140) Cis k8s name migration (elastic#3113) remove exported fields; these will be added to Kibana docs instead (elastic#3093) Exclude ecosystem as owner of subdirectories of packages (elastic#3132) [awsfargate] bump package version (elastic#3130) add first csp rule template (elastic#3081) ...
Uh oh!
There was an error while loading.Please reload this page.
What does this PR do?
Checklist
changelog.ymlfile.manifest.ymlfile to point to the latest Elastic stack release (e.g.^8.0.0).How to test this PR locally
Note: We have covered dashboards and the visualisations for all data streams of spring boot into separate PR. Also Kibana version will be updated to 8.1.0 in manifest.yml after testing this integration on 8.1.0.