Repository files navigation

Welcome to the unified Microsoft Sentinel and Microsoft 365 Defender repository! This repository contains out of the box detections, exploration queries, hunting queries, workbooks, playbooks and much more to help you get ramped up with Microsoft Sentinel and provide you security content to secure your environment and hunt for threats. The hunting queries also include Microsoft 365 Defender hunting queries for advanced hunting scenarios in both Microsoft 365 Defender and Microsoft Sentinel. You can also submit toissues for any samples or resources you would like to see here as you onboard to Microsoft Sentinel. This repository welcomes contributions and refer to this repository'swiki to get started. For questions and feedback, please contactAzureSentinel@microsoft.com

• Microsoft Sentinel documentation
• Microsoft 365 Defender documentation
• Security Community Webinars
• Getting started with GitHub

We value your feedback. Here are some channels to help surface your questions or feedback:

1. General product specific Q&A for SIEM and SOAR - Join in theMicrosoft Sentinel Tech Community conversations
2. General product specific Q&A for XDR - Join in theMicrosoft 365 Defender Tech Community conversations
3. Product specific feature requests - Upvote or post new onMicrosoft Sentinel feedback forums
4. Report product or contribution bugs - File a GitHub Issue usingBug template
5. General feedback on community and contribution process - File a GitHub Issue usingFeature Request template

This project welcomes contributions and suggestions. Most contributions require you to agree to aContributor License Agreement (CLA) declaring that you have the right to, and actually do, grant usthe rights to use your contribution. For details, visithttps://cla.microsoft.com.

Note: If you are a first time contributor to this repository,General GitHub Fork the repo guidance before cloning orSpecific steps for the Sentinel repo.

Brand new or update to a contribution via these methods:

• Submit for review directly on GitHub website
  • Browse to the folder you want to upload your file to
  • Choose Upload Files and browse to your file.
  • You will be required to create your own branch and then submit the Pull Request for review.
• UseGitHub Desktop orVisual Studio orVSCode
  • Fork the repo
  • Clone the repo
  • Create your own branch
  • Do your additions/updates in GitHub Desktop
  • Be sure to merge master back to your branch before you push.
  • Push your changes to GitHub

• After you push your changes, you will need to submit thePull Request (PR)
• Details about the Proposed Changes are required, be sure to include a minimal level of detail so a review can clearly understand the reason for the change and what he change is related to in the code.
• After submission, check thePull Request for comments
• Make changes as suggested and update your branch or explain why no change is needed. Resolve the comment when done.

As part of the PR checks we run a structure validation to make sure all required parts of the YAML structure are included. For Detections, there is a new section that must be included. See thecontribution guidelines for more information. If this section or any other required section is not included, then a validation error will occur similar to the below.The example is specifically if the YAML is missing the entityMappings section:

A total of 1 test files matched the specified pattern.[xUnit.net 00:00:00.95]     Kqlvalidations.Tests.DetectionTemplateStructureValidationTests.Validate_DetectionTemplates_HaveValidTemplateStructure(detectionsYamlFileName: "ExcessiveBlockedTrafficGeneratedbyUser.yaml") [FAIL]  X Kqlvalidations.Tests.DetectionTemplateStructureValidationTests.Validate_DetectionTemplates_HaveValidTemplateStructure(detectionsYamlFileName: "ExcessiveBlockedTrafficGeneratedbyUser.yaml") [104ms]  Error Message:   Expected object to be <null>, but found System.ComponentModel.DataAnnotations.ValidationException with message "An old mapping for entity 'AccountCustomEntity' does not have a matching new mapping entry."

As part of the PR checks we run a syntax validation of the KQL queries defined in the template. If this check fails go to Azure Pipeline (by pressing on the errors link on the checks tab in your PR)In the pipeline you can see which test failed and what is the cause:

Example error message:

A total of 1 test files matched the specified pattern.[xUnit.net 00:00:01.81]     Kqlvalidations.Tests.KqlValidationTests.Validate_DetectionQueries_HaveValidKql(detectionsYamlFileName: "ExcessiveBlockedTrafficGeneratedbyUser.yaml") [FAIL]  X Kqlvalidations.Tests.KqlValidationTests.Validate_DetectionQueries_HaveValidKql(detectionsYamlFileName: "ExcessiveBlockedTrafficGeneratedbyUser.yaml") [21ms]  Error Message:   Template Id:fa0ab69c-7124-4f62-acdd-61017cf6ce89 is not valid Errors:The name 'SymantecEndpointProtection' does not refer to any known table, tabular variable or function., Code: 'KS204', Severity: 'Error', Location: '67..93',The name 'SymantecEndpointProtection' does not refer to any known table, tabular variable or function., Code: 'KS204', Severity: 'Error', Location: '289..315'

If you are using custom logs table (a table which is not defined on all workspaces by default) you should verifyyour table schema is defined in json file in the folderAzure-Sentinel\.script\tests\KqlvalidationsTests\CustomTables

Example for table tablexyz.json

{"Name":"tablexyz","Properties": [    {"Name":"SomeDateTimeColumn","Type":"DateTime"    },    {"Name":"SomeStringColumn","Type":"String"    },    {"Name":"SomeDynamicColumn","Type":"Dynamic"    }  ]}

In order to run the KQL validation before submitting Pull Request in you local machine:

• You need to have.Net Core 3.1 SDK installedHow to download .Net (Supports all platforms)
• Open Shell and navigate toAzure-Sentinel\\.script\tests\KqlvalidationsTests\
• Executedotnet test

Example of output (in Ubuntu):

Welcome to .NET Core 3.1!---------------------SDK Version: 3.1.403Telemetry---------The .NET Core tools collect usage data in order to help us improve your experience. The data is anonymous. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.Read more about .NET Core CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry----------------Explore documentation: https://aka.ms/dotnet-docsReport issues and find source on GitHub: https://github.com/dotnet/coreFind out what's new: https://aka.ms/dotnet-whats-newLearn about the installed HTTPS developer cert: https://aka.ms/aspnet-core-httpsUse 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli-docsWrite your first app: https://aka.ms/first-net-core-app--------------------------------------------------------------------------------------Test run for /mnt/c/git/Azure-Sentinel/.script/tests/KqlvalidationsTests/bin/Debug/netcoreapp3.1/Kqlvalidations.Tests.dll(.NETCoreApp,Version=v3.1)Microsoft (R) Test Execution Command Line Tool Version 16.7.0Copyright (c) Microsoft Corporation.  All rights reserved.Starting test execution, please wait...A total of 1 test files matched the specified pattern.Test Run Successful.Total tests: 171     Passed: 171 Total time: 25.7973 Seconds

Similarly to KQL Validation, there is an automatic validation of the schema of a detection.The schema validation includes the detection's frequency and period, the detection's trigger type and threshold, validity of connectors Ids (valid connectors Ids list), etc.A wrong format or missing attributes will result with an informative check failure, which should guide you through the resolution of the issue, but make sure to look into the format of already approved detection.

In order to run the KQL validation before submitting Pull Request in you local machine:

• You need to have.Net Core 3.1 SDK installedHow to download .Net (Supports all platforms)
• Open Shell and navigate toAzure-Sentinel\\.script\tests\DetectionTemplateSchemaValidation\
• Executedotnet test

When you submit a pull request, a CLA-bot will automatically determine whether you need to providea CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructionsprovided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted theMicrosoft Open Source Code of Conduct.For more information see theCode of Conduct FAQ orcontactopencode@microsoft.com with any additional questions or comments.

For information on what you can contribute and further details, refer to the"get started" section on the project'swiki.