Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Netdata Functions: brainstorm on future enhancements and new functions#14412

hugovalente-pm started this conversation inIdeas
Discussion options

After the initial release, opening this public discussion to get community feedback on future enhancements and new functions that would be useful to be added.

Sharing some ideas collected so far:

  1. Enhancements:

    1. Processes: add some visualizations on top of the table where users could choose metrics they would like to see plotted on a chart
    2. Processes: add some grouping mechanism to group table rows on specific attributes, e.g. PPID, Category
    3. Processes: the ability to see the history on processes running at some point back, e.g. at 3 am the cpu was going crazy and it would be nice to go back on the process list to see which process was causing issues
    4. Define more freeze columns
    5. Export the data from the table
    6. Auto-fit a column or all columns: like double click
    7. Auto-fit a column or all columns: like double click

  2. New functions

  3. Retrieve slow queries from a DB collector, e.g. PostgreSQL

  4. Results fromnetstat to troubleshoot network-related issues

  5. Live file descriptors, see what process is writing where

Looking forward to getting your feedback and ideas!

You must be logged in to vote

Replies: 12 comments 4 replies

Comment options

I think some docker related stuff could be very handy too in terms of candidates for next functions. eg "Processes - Docker"

andrewmaguire@ml-demo-ml-enabled:~/tutorial-environment$ sudo docker compose ps --allNAME                                IMAGE                      COMMAND                  SERVICE             CREATED             STATUS              PORTStutorial-environment_app_1          tutorial-environment_app   "/bin/sh -c '/app ht…"   app                 4 minutes ago       Up 4 minutes        0.0.0.0:8081->80/tcp, :::8081->80/tcptutorial-environment_db_1           grafana/tns-db:latest      "/db"                    db                  4 minutes ago       Up 4 minutes        0.0.0.0:8082->80/tcp, :::8082->80/tcptutorial-environment_grafana_1      grafana/grafana:8.4.5      "/run.sh"                grafana             4 minutes ago       Up 4 minutes        0.0.0.0:3000->3000/tcp, :::3000->3000/tcptutorial-environment_loki_1         grafana/loki:2.4.2         "/usr/bin/loki -conf…"   loki                4 minutes ago       Up 4 minutes        0.0.0.0:3100->3100/tcp, :::3100->3100/tcptutorial-environment_prometheus_1   prom/prometheus:v2.34.0    "/bin/Prometheus --c…"   prometheus          4 minutes ago       Up 4 minutes        0.0.0.0:9090->9090/tcp, :::9090->9090/tcptutorial-environment_promtail_1     grafana/promtail:2.0.0     "/usr/bin/promtail -…"   promtail            4 minutes ago       Up 4 minutes
andrewmaguire@ml-demo-ml-enabled:~/tutorial-environment$ sudo docker psCONTAINER ID   IMAGE                               COMMAND                  CREATED         STATUS         PORTS                                       NAMES8954a680b822   Grafana/loki:2.4.2                  "/usr/bin/loki -conf…"   5 minutes ago   Up 5 minutes   0.0.0.0:3100->3100/tcp, :::3100->3100/tcp   tutorial-environment_loki_17f45e5790f1b   grafana/tns-db:latest               "/db"                    5 minutes ago   Up 5 minutes   0.0.0.0:8082->80/tcp, :::8082->80/tcp       tutorial-environment_db_190ccce3637f0   grafana/promtail:2.0.0              "/usr/bin/promtail -…"   5 minutes ago   Up 5 minutes                                               tutorial-environment_promtail_1e239ff09eb74   grafana/grafana:8.4.5               "/run.sh"                5 minutes ago   Up 5 minutes   0.0.0.0:3000->3000/tcp, :::3000->3000/tcp   tutorial-environment_grafana_1a5fb2314da87   tutorial-environment_app            "/bin/sh -c '/app ht…"   5 minutes ago   Up 5 minutes   0.0.0.0:8081->80/tcp, :::8081->80/tcp       tutorial-environment_app_16a7e75e721eb   prom/prometheus:v2.34.0             "/bin/prometheus --c…"   5 minutes ago   Up 5 minutes   0.0.0.0:9090->9090/tcp, :::9090->9090/tcp   tutorial-environment_prometheus_13da507f6272f   andrewm4894/netdata-ml-app:latest   "/bin/sh -c 'gunicor…"   7 hours ago     Up 7 hours                                                 netdata-ml-app
andrewmaguire@ml-demo-ml-enabled:~/tutorial-environment$ sudo docker compose toptutorial-environment_app_1UID    PID      PPID     C    STIME   TTY   TIME       CMDroot   269834   269713   0    22:33   ?     00:00:00   /app http://db   tutorial-environment_db_1UID    PID      PPID     C    STIME   TTY   TIME       CMDroot   269814   269711   0    22:33   ?     00:00:00   /db   tutorial-environment_grafana_1UID   PID      PPID     C    STIME   TTY   TIME       CMD472   269908   269872   0    22:33   ?     00:00:01   grafana-server --homepath=/usr/share/grafana --config=/etc/grafana/grafana.ini --packaging=docker cfg:default.log.mode=console cfg:default.paths.data=/var/lib/grafana cfg:default.paths.logs=/var/log/grafana cfg:default.paths.plugins=/var/lib/grafana/plugins cfg:default.paths.provisioning=/etc/grafana/provisioning   tutorial-environment_loki_1UID     PID      PPID     C    STIME   TTY   TIME       CMD10001   269861   269803   0    22:33   ?     00:00:00   /usr/bin/loki -config.file=/etc/loki/local-config.yaml   tutorial-environment_prometheus_1UID      PID      PPID     C    STIME   TTY   TIME       CMDnobody   269652   269596   0    22:33   ?     00:00:00   /bin/prometheus --config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/prometheus   tutorial-environment_promtail_1UID    PID      PPID     C    STIME   TTY   TIME       CMDroot   269806   269650   0    22:33   ?     00:00:01   /usr/bin/promtail -config.file=/etc/promtail/config.yml
andrewmaguire@ml-demo-ml-enabled:~/tutorial-environment$ sudo docker container lsCONTAINER ID   IMAGE                               COMMAND                  CREATED         STATUS         PORTS                                       NAMES8954a680b822   grafana/loki:2.4.2                  "/usr/bin/loki -conf…"   9 minutes ago   Up 9 minutes   0.0.0.0:3100->3100/tcp, :::3100->3100/tcp   tutorial-environment_loki_17f45e5790f1b   grafana/tns-db:latest               "/db"                    9 minutes ago   Up 9 minutes   0.0.0.0:8082->80/tcp, :::8082->80/tcp       tutorial-environment_db_190ccce3637f0   grafana/promtail:2.0.0              "/usr/bin/promtail -…"   9 minutes ago   Up 9 minutes                                               tutorial-environment_promtail_1e239ff09eb74   grafana/grafana:8.4.5               "/run.sh"                9 minutes ago   Up 9 minutes   0.0.0.0:3000->3000/tcp, :::3000->3000/tcp   tutorial-environment_grafana_1a5fb2314da87   tutorial-environment_app            "/bin/sh -c '/app ht…"   9 minutes ago   Up 9 minutes   0.0.0.0:8081->80/tcp, :::8081->80/tcp       tutorial-environment_app_16a7e75e721eb   prom/prometheus:v2.34.0             "/bin/prometheus --c…"   9 minutes ago   Up 9 minutes   0.0.0.0:9090->9090/tcp, :::9090->9090/tcp   tutorial-environment_prometheus_13da507f6272f   andrewm4894/netdata-ml-app:latest   "/bin/sh -c 'gunicor…"   7 hours ago     Up 7 hours                                                 netdata-ml-app
$ sudo docker statsCONTAINER ID   NAME                                CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O     PIDS8954a680b822   tutorial-environment_loki_1         0.08%     15.3MiB / 7.268GiB    0.21%     4.12kB / 1.62kB   0B / 65.5kB   87f45e5790f1b   tutorial-environment_db_1           0.01%     2.637MiB / 7.268GiB   0.04%     1.24kB / 0B       0B / 0B       990ccce3637f0   tutorial-environment_promtail_1     0.26%     10.66MiB / 7.268GiB   0.14%     2.93kB / 2.92kB   0B / 274kB    9e239ff09eb74   tutorial-environment_grafana_1      0.04%     25.99MiB / 7.268GiB   0.35%     72.1kB / 41.2kB   0B / 15.6MB   8a5fb2314da87   tutorial-environment_app_1          0.01%     6.363MiB / 7.268GiB   0.09%     56.5kB / 297kB    0B / 0B       106a7e75e721eb   tutorial-environment_prometheus_1   0.00%     30.4MiB / 7.268GiB    0.41%     298kB / 55.3kB    0B / 0B       73da507f6272f   netdata-ml-app                      0.02%     266MiB / 7.268GiB     3.57%     0B / 0B           0B / 111kB    10
You must be logged in to vote
0 replies
Comment options

@hugovalente-pm ability to add functions to a custom dashboard in some way is something i'm sure some users could find useful too.

eg if was a way to save a particular set of filters, groupings etc as a "Saved Function View" and then add that to custom dashboards could be useful.

You must be logged in to vote
0 replies
Comment options

Hello@hugovalente-pm ,

Next Sunday I will start modifying eBPF plugin memory management to use our new array allocator. Considering that you wroteResults from netstat to troubleshoot network-related issues, should I also starting preparing eBPF plugin to deliver the data? If so, what are the socket information we are planing to show?

Best regards!

You must be logged in to vote
1 reply
@hugovalente-pm
Comment options

@thiagoftsm at the moment we are just collecting feedback understand what community is looking for, thenetstat one just come from some high-level discussion so nothing is yet defined/clear at the moment.
if you have some ideas already you can probably start a specific github issue for that

Comment options

One possibility that comes to mind for a new function would be listing the cgroup hierarchy, together with configured limits and actual resource usage. This ties in to some extent with the Docker info that@andrewm4894 suggests, but is of more generic use as well. We actually havemost of the information already in the dashboard, but the dashboard only properly tracks leaf cgroups, and in a number of cases it’svery important to be able to see intermediate nodes in the tree structure.

You must be logged in to vote
0 replies
Comment options

On the point about network information, we may also want a way to query firewall information. Effectively, the output ofnft list ruleset for systems using nftables, or the output ofiptables -L -v && ip6tables -L -v && ebtables -L -v && arptables -L -v for systems still using xtables.

This would give us a way for users to see specific traffic counters (also, we ideally need a way to collect these counters without getting nfacct involved, it’s entirely doable for at least nftables just using netlink), as well as a way to clearly see the current active firewall rules.

You must be logged in to vote
0 replies
Comment options

One addition for the processes, being able to group by cgroup would be ahuge improvement here, that gets us grouping by systemd service and logind session, as well as fitting logically with a number of other setups, and is for most users likely to be the most logical grouping after a simple PPID tree grouping.

You must be logged in to vote
1 reply
@thiagoftsm
Comment options

eBPF.plugin gives for user the possibility to collectreal parent,parent andprocess PID. We can organize all necessary information in a shared memory that can be used by all.

Comment options

I think maybe some filespace and disk size utility functions might be other good candidates. Perhaps might need to be more push button and less eager to refresh.

Thinking about du and similar utilities

https://www.redhat.com/sysadmin/du-command-options

You must be logged in to vote
0 replies
Comment options

I opened theticket before discovering this discussion -- but how about auto triggering functions based on certain events.

Eg: Trigger the process function when a node experiences elevated anomaly rates (an anomaly event) or when a particular alert is triggered - so the user can go back and check process state.

Also (kinda) solves this user request

the ability to see the history on processes running at some point back, e.g. at 3 am the cpu was going crazy and it would be nice to go back on the process list to see which process was causing issues

You must be logged in to vote
0 replies
Comment options

Some feedback from dear discord users:

Is there any way to know what process (threads) were running at a point in time using Netdata out of the box? Perhaps functions can be useful.
not at the moment, i want that feature too
e.g. a "top/htop" history
same - we have discussed for example maybe some wayt to have it be triggered to record following some alerts or something as "always on" might just be a bit too hard in terms of data storage and processing - not sure though - defo worth exploring
My workloads are usually pinned to specific cores and such, so knowing what was running where is helpful.

You must be logged in to vote
0 replies
Comment options

Potentially related

#15156

You must be logged in to vote
1 reply
@shyamvalsan
Comment options

We are about to release grouping in functions, so we can group all the PIDS from a user in a single row and look at the resources. But disk usage isn't one of the columns in the processes function unfortunately.

Comment options

Is there any scope to create our own scripts that can be run from a central location and deployed? An example we have dozens on VPSs for clients, Netdata provides an excellent tool to monitor them, but not to maintain.

An option to perform certain custom utilities or scripts would save a lot of time. E.g. updates, clearing disk space, SSH key rotation, configuation changes.

The above would also allow us to write custom scripts to check and report on those things e.g. how many packages can be updated, what key PHP.ini settings are used on each server.

You must be logged in to vote
1 reply
@shyamvalsan
Comment options

@iveoles Yes, this is indeed the end game we have in mind. For users to be able to run their own custom scripts centrally through Netdata Cloud using functions. Do you envision only admin users having access to such functions that can execute code on your infrastructure, this was one of the areas we are wary about in terms of getting into write/execute actions from the Netdata UI.

cc:@ktsaou

Comment options

making more features available for selfhosting, for local installs/local infra management, the "security" benefits arent really there and just makes netdata worse to use

You must be logged in to vote
0 replies
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Category
Ideas
Labels
None yet
7 participants
@hugovalente-pm@Ferroin@iveoles@andrewm4894@shyamvalsan@smashyalts@thiagoftsm
[8]ページ先頭
©2009-2025 Movatter.jp