Repository files navigation

This repository is dedicated to unsolved LeetCode SQL questions implemented in PySpark.

This notebook contains a collection of SQL-related challenges sourced fromLeetCode. The primary goal is to enhance SQL skills while leveragingPySpark, a powerful framework for large-scale data processing.

For each question, you will find:

• A brief problem description
• The relevant dataset or schema context
• A step-by-step Unsolved implemented using PySpark DataFrames and SQL functions

All Unsolveds in this notebook are implemented usingPySpark to ensure consistency with big data processing practices. The approach utilizesPySpark’s Python API, DataFrame API, and built-in functions to solve problems efficiently.

Disclaimer:
All questions and problem statements belong toLeetCode. This notebook is solely for educational purposes, and all credit and ownership of the questions remain withLeetCode. If you have any questions, please reach out toinfo.bilearner@gmail.com.

• Databricks Account:
  • You need access to a Databricks account. Databricks provides a collaborative environment for big data analytics, including support for PySpark, which is crucial for running your Unsolveds.
• Basic Knowledge of PySpark:
  • Familiarity with PySpark is essential. Users should understand how to write PySpark code to manipulate dataframes, perform transformations, and execute actions.
• SQL Knowledge:
  • Basic knowledge of SQL is required to understand and write SQL queries within the PySpark framework. This includes understanding SQL syntax, querying databases, and manipulating data.
• GitHub Repository Access:
  • Access to your GitHub repository containing unsolved LeetCode SQL questions. Users should be able to clone the repository, review the Unsolveds, and potentially contribute if allowed.

Databricks Premium is a paid plan that offers advanced features such as higher compute power, security options, and integrations.

1. Go toDatabricks website.
2. Click"Start your free trial" (for a trial) or go to"Sign In" if you have an account.
3. Choose"AWS", "Azure", or "GCP" as your cloud provider.
4. Follow the registration process, providing details like your email, company, and cloud provider credentials.

1. In the cloud provider console (AWS, Azure, or GCP), create a Databricks workspace.
2. Select thePremium plan during setup.
3. Configure networking and security settings as required.
4. Once created, launch the workspace from the cloud console.

1. Inside the Databricks workspace, go toCompute.
2. ClickCreate Cluster.
3. Choose a cluster name and select a runtime version (latest recommended).
4. Select the number of workers (scale as needed).
5. ClickCreate Cluster.

1. Navigate toWorkspace > Users > Your Name.
2. ClickCreate > Notebook.
3. Name the notebook and selectPython as the language.
4. Attach it to your running cluster.

Databricks Community Edition is a free, limited version ideal for learning PySpark.

1. Go toDatabricks Community Edition Signup.
2. Enter your email and complete the registration.
3. Check your email for the verification link and activate your account.
4. Log in to your Databricks Community workspace.

1. Click onCompute in the left panel.
2. ClickCreate Cluster.
3. Name your cluster.
4. Choose the latest runtime version.
5. ClickCreate Cluster (Community Edition supports only small clusters).

1. Go toWorkspace > Users > Your Name.
2. ClickCreate > Notebook.
3. Name the notebook and selectPython.
4. Attach it to the running cluster.

1. Clone Your GitHub Repository:

   • First, ensure you have Git installed on your local machine. If not, download and install it fromGit's official website.
   • Open your terminal (command prompt) and navigate to the directory where you want to clone your repository.
   • Clone your GitHub repository using the command:

     git clone <repository_url>

   • Replace<repository_url> with the URL of your GitHub repository. This will download your repository to your local machine.

2. Install Required Dependencies:

   • Make sure you have Python installed on your machine. It's recommended to use Anaconda or Miniconda to manage your Python environments.
   • Install Jupyter Notebook and PySpark dependencies if you haven't already:

     pip install jupyter pyspark

3. Launch Jupyter Notebook:

   • Navigate to the directory where your Jupyter Notebook files are located. Typically, this would be the root directory of your cloned repository.
   • Start Jupyter Notebook by running the command:

     jupyter notebook

   • This command will open a new tab in your web browser with the Jupyter Notebook interface.

4. Open and Run Your Notebook:

   • In the Jupyter Notebook interface, navigate to the directory where your notebook file (*.ipynb) is located.
   • Click on the notebook file to open it.
   • Once the notebook is open, you can run each cell by pressingShift + Enter or using the "Run" button in the toolbar.
   • Ensure that Spark is correctly initialized and configured in your notebook. You may need to import necessary libraries and set up the Spark session if it's not done automatically.

5. Verify Spark Installation and Configuration:

   • Check if Spark is installed and configured correctly by running a basic Spark operation in one of the notebook cells. For example:

     frompyspark.sqlimportSparkSession# Initialize Spark sessionspark=SparkSession.builder \                    .appName("MyApp") \                    .getOrCreate()# Verify Spark sessionspark

   • If Spark is configured correctly, you should see the Spark session information printed without any errors.

6. Execute and Test Your Notebook:

   • Execute each cell in your notebook to ensure that all code runs as expected.
   • Validate the results of the LeetCode SQL questions solutions to ensure correctness and functionality with PySpark.

7. Save Your Work:

   • Once you have verified that everything is working correctly, save your notebook with any changes you have made.

• Environment Management: Consider using virtual environments or conda environments to manage dependencies and avoid conflicts between different projects.
• Documentation: It's helpful to include documentation within your notebook, such as explanations of the SQL solutions and any specific configurations required for Spark.
• Version Control: Regularly commit your changes to Git and push them to your GitHub repository to keep a versioned history of your work.

By following these steps, you should be able to successfully import and run your LeetCode SQL questions notebook using PySpark in Jupyter Notebook on your local machine.

Note:
The Databricks Community Edition will be more than adequate for this activity.

🚀 Support This Project! ⭐

Hey everyone! 👋

We've created this GitHub repository to help the community solveLeetCode SQL questions using PySpark. If you find this project helpful, please considergiving it a star ⭐ and sharing it with other learners who might benefit from it!

Your support will help grow this resource and make it even better for everyone. Let’s learn and improve together! 🚀

🔗LeetCode SQL Questions-(PySpark Unsolved)

Happy coding! 💻🔥