Repository files navigation

Create a Synapse Analytics environment based on best practices to achieve a successful proof of concept. While settings can be adjusted,the major deployment differences are based on whether or not you used Private Endpoints for connectivity. If you do not already usePrivate Endpoints for other Azure deployments, it's discouraged to use them for a proof of concept as they have many other networkingdependancies than what can be configured here.

The following commands should be executed from the Azure Cloud Shell athttps://shell.azure.com using bash:

git clone https://github.com/MSUSSolutionAccelerators/Azure-Synapse-Analytics-PoCcd Azure-Synapse-Analytics-PoCbash deploySynapse.sh

You can manually configure the Bicep parameters and update default settings such as the Azure region, database name, credentials, and private endpoint integration. The following commands should be executed from the Azure Cloud Shell athttps://shell.azure.com using bash:

git clone https://github.com/MSUSSolutionAccelerators/Azure-Synapse-Analytics-PoCcd Azure-Synapse-Analytics-PoCcode Bicep/main.parameters.jsonaz deployment sub create --template-file Bicep/main.bicep --parameters Bicep/main.parameters.json --name Azure-Synapse-Analytics-PoC --location eastusbash deploySynapse.sh

You can manually configure the Terraform parameters and update default settings such as the Azure region, database name, credentials, and private endpoint integration. The following commands should be executed from the Azure Cloud Shell athttps://shell.azure.com using bash:

git clone https://github.com/MSUSSolutionAccelerators/Azure-Synapse-Analytics-PoCcd Azure-Synapse-Analytics-PoCcode Terraform/terraform.tfvarsterraform -chdir=Terraform initterraform -chdir=Terraform planterraform -chdir=Terraform applybash deploySynapse.sh

• DW1000 Dedicated SQL Pool
• Example scripts for configuring and using:
  • Row Level Security
  • Column Level Security
  • Dynamic Data Masking
  • Materialized Views
  • JSON data parsing
• Example notebooks for testing:
  • Spark and Serverless Metastore integration
  • Spark Delta Lake integration

• config container for Azure Synapse Analytics Workspace
• data container for queried/ingested data

• Logging and telemetry for Azure Synapse Analytics
• Logging and telemetry for Azure Data Lake Storage Gen2

• Enable Result Set Caching
• Create a pipeline to auto pause/resume the Dedicated SQL Pool
• Feature flag to enable/disable Private Endpoints
• Serverless SQL Demo Data Database
• Proper service and user permissions for Azure Synapse Analytics Workspace and Azure Data Lake Storage Gen2
• Parquet Auto Ingestion pipeline to optimize data ingestion using best practices
• Lake Database Auto DDL creation (views) for all files used by Ingestion pipeline

• You can find a Synapse_Dedicated_SQL_Pool_Test_Plan.jmx JMeter file under the artifacts folder that is configured to work with your recently deployed Synapse Environment.

• Synapse Data Explorer Pool deployment
• Purview Deployment and Configuration
• Azure ML Services Deployment and Configuration
• Cognitive Services Deployment and Configuration

Copyright (c) Microsoft Corporation

All rights reserved.

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ""Software""), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE

This project welcomes contributions and suggestions. Most contributions require you to agree to aContributor License Agreement (CLA) declaring that you have the right to, and actually do, grant usthe rights to use your contribution. For details, visithttps://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to providea CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructionsprovided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted theMicrosoft Open Source Code of Conduct.For more information see theCode of Conduct FAQ orcontactopencode@microsoft.com with any additional questions or comments.

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must followMicrosoft's Trademark & Brand Guidelines.Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.Any use of third-party trademarks or logos are subject to those third-party's policies.