No trial. No credit card required. Just your GitHub account.
2 reactionsAnnouncing dotnet monitor in .NET 6
![Sourabh Shirhatti [MSFT]](/image.pl?url=https%3a%2f%2fdevblogs.microsoft.com%2fdotnet%2fwp-content%2fuploads%2fsites%2f10%2fletter-avatar%2f9228d33c03603017171cd81a0ce66d32.svg&f=jpg&w=240)
We first introduceddotnet monitor as anexperimental tool in June 2020 and have been hard at work to turn it into a production grade toolover the last year. Today I’m excited to announce the first supported release ofdotnet monitor as part of .NET 6.
dotnet monitor already powers theDiagnostic tools for .NET applications on Azure App Service on Linux, and we’re excited to now see it used in even more environments with this release.
What isdotnet monitor?
Running a .NET application in diverse environments can make collecting diagnostics artifacts (e.g., logs, traces, process dumps) challenging.dotnet monitor is a tool that provides an unified way to collect these diagnostic artifacts regardless of whether running you’re running on your desktop machine or in a kubernetes cluster.
There are two different mechanisms for collection of these diagnostic artifacts:
- AnHTTP API for on demand collection of artifacts. You can call these API endpoints when you already know your application is experiencing an issue and you are interested in gathering more information.
- Triggers for rule-based configuration for always-on collection of artifacts. You may configure rules to collect diagnostic artifacts when a desired condition is met, for example, collect a process dump when you have sustained high CPU.
Getting Started
dotnet monitor is available via two different distribution mechanisms:
- As a .NET CLI tool; and
- As a container image available via the Microsoft Container Registry (MCR).
.NET CLI Tool
Thedotnet monitor CLI tool requires a .NET 6 SDK installed as a pre-requisite. If you do not have a new enough SDK, you can install a new one from theDownload .NET webpage.
You can download the latest version ofdotnet monitor using the following command:
dotnet tool install -g dotnet-monitor --version 6.0.0If you already havedotnet monitor installed and want to update:
dotnet tool update -g dotnet-monitor --version 6.0.0Container image
Thedotnet monitor container image is available on MCR. You can pull the latest image using the following command:
docker pull mcr.microsoft.com/dotnet/monitor:6.0.0HTTP API
dotnet monitor exposes an HTTP API to query available processes, collect diagnostics artifacts, and check on the status on the requested artifacts
The HTTP API exposed by dotnet monitor exposesthe following HTTP endpoints:
/processes– Gets detailed information about discoverable processes./dump– Captures managed dumps of processes without using a debugger./gcdump– Captures GC dumps of processes./trace– Captures traces of processes without using a profiler./metrics– Captures a snapshot of metrics of the default process in the Prometheus exposition format./livemetrics– Captures live streaming metrics of a process./logs– Captures logs of processes./info– Gets info about dotnet monitor./operations– Gets egress operation status or cancels operations.
The example below showcases the use ofdotnet monitor to streamDebug level logs fromMicrosoft.AspNetCore.Server.Kestrel.Connections category for a duration of sixty seconds from the target process.
PS> curl.exe -X POST "https://localhost:52323/logs?name=myWebApp&durationSeconds=60" ` -H "Accept: application/x-ndjson" ` -H "Content-Type: application/json" ` --negotiate -u $(whoami)` -d '{"filterSpecs": {"Microsoft.AspNetCore.Server.Kestrel.Connections": "Debug"}}' {"Timestamp":"2021-11-05 08:12:54Z","LogLevel":"Debug","EventId":39,"EventName":"ConnectionAccepted","Category":"Microsoft.AspNetCore.Server.Kestrel.Connections","Message":"Connection id u00220HMD06BUKL2CUu0022 accepted.","State":{"Message":"Connection id u00220HMD06BUKL2CUu0022 accepted.","ConnectionId":"0HMD06BUKL2CU","{OriginalFormat}":"Connection id u0022{ConnectionId}u0022 accepted."}}{"Timestamp":"2021-11-05 08:12:54Z","LogLevel":"Debug","EventId":1,"EventName":"ConnectionStart","Category":"Microsoft.AspNetCore.Server.Kestrel.Connections","Message":"Connection id u00220HMD06BUKL2CUu0022 started.","State":{"Message":"Connection id u00220HMD06BUKL2CUu0022 started.","ConnectionId":"0HMD06BUKL2CU","{OriginalFormat}":"Connection id u0022{ConnectionId}u0022 started."}}{"Timestamp":"2021-11-05 08:12:54Z","LogLevel":"Debug","EventId":9,"EventName":"ConnectionKeepAlive","Category":"Microsoft.AspNetCore.Server.Kestrel.Connections","Message":"Connection id u00220HMD06BUKL2CUu0022 completed keep alive response.","State":{"Message":"Connection id u00220HMD06BUKL2CUu0022 completed keep alive response.","ConnectionId":"0HMD06BUKL2CU","{OriginalFormat}":"Connection id u0022{ConnectionId}u0022 completed keep alive response."},"Scopes":[{"ConnectionId":"0HMD06BUKL2CU"},{"RequestId":"0HMD06BUKL2CU:00000002","RequestPath":"/"}]}As demonstrated in the example above you can usedotnet monitor to on-demand egress diagnostic artifacts from a target process. In addition to logs, you can also collect traces, memory dumps, GC dumps, and metrics from the target process.
Triggers
dotnet monitor can be configured to automatically collect diagnostic artifacts based on conditions within the discovered processes. When a new process is discovered,dotnet monitor will attempt to apply conifgured rules if the process matches the filter on a rule. The applied rule will start monitoring the process for the condition that the trigger describes. If that condition is satisfied, the action list is executed assuming the specified limits have not yet been reached.
The example below showcases an example wheredotnet monitor will collect no more than one triage dump every hour if it detects sustained high CPU usage of greater than 80% for a duration of greater than one minute.
{ "CollectionRules": { "HighCpuRule": { "Filters": [ { "Key": "ProcessName", "Value": "MyApp", "MatchType": "Exact" } ], "Trigger": { "Type": "EventCounter", "Settings": { "ProviderName": "System.Runtime", "CounterName": "cpu-usage", "GreaterThan": 80, "SlidingWindowDuration": "00:01:00" } }, "Limits": { "ActionCount": 1, "ActionCountSlidingWindowDuration": "1:00:00" }, "Actions": [ { "Type": "CollectDump", "Settings": { "Type": "Triage", "Egress": "myBlobStorageAccount" } } ] } }}As demonstrated in the example above, you can usedotnet monitor to customize the always-on collection of diagnostic artifacts based on heuristics you define for what is considered anomalous behavior for your application. To learn more about the various ways to customize collection, take a look at the docs onCollection Rules. We’re excited to learn what problems you can solve in production withdotnet monitor!
Feedback
Let us know what we can do to make it easier to diagnose what’s wrong with your .NET application. You cancreate issues or just follow along with the progress of the project on ourGitHub repository.
Author
I work on .NET, ASP.NET Core, and gRPC.UT Austin Alumnus and Bengaluru native
7 comments
Discussion is closed.Login to edit/delete existing comments.
Pavel Biryukov Thanks Sourabh, interesting!
Please have a look at curl (strange x-ndjson):
-H “Accept: application/x-ndjson”
Fabian Schmied Also, you’re saying, “running on your desktop machine or in a kubernetes cluster.” However, desktop machine is not a production scenario, is it? How do I use
dotnet monitorwhen the application is running on a server machine, do I have to install the .NET SDK on that machine? I wouldn’t usually want to install an SDK on a production server, would I?Fabian Schmied Can I read up somewhere how this works internally? I’m specifically interested in how a
dotnet monitorDocker container can retrieve or generate data such as a memory dump from a process running within another container.Fabian Schmied I’ve found this comment on the original intro blog post:https://devblogs.microsoft.com/dotnet/introducing-dotnet-monitor/#comment-6679 . So, apparently,
dotnet monitorcommunicates with the CLR’s diagnostic server via IPC (Windows named pipes or Unix domain sockets), which, it appears, is also accessible across container boundaries? I wonder what happens if multiple dotnet app containers are running on the same node – willdotnet monitorbe able to select the “right” one? How?
Tat Sean Pang The github document on dotnet-monitor still having a lot of broken links and way too brief explanation. So, please fix those. Thanks.
aleksander-kovac Congratulations to the team on the updates. Needles to say, the monitor is a critical tool for solving production issues.
Madhusudana B.S You showed how to use dotnet-monitor as sidecar in Kubernetes, can it be done for Docker-composed based deployment ?.
Example: If I have .NET 6 application running in docker composed based deployment , can I run dotnet-monitor sidecar to get the diagnostic data from the running application without restarting the application.
