Repository files navigation

Practice the Julialang ML workflow using data fromAcea Smart Water Analytics competition.

This instruction is forUbuntu andWindows 10/11 user.

Please note that we use docker to run containers, andcontainers running on Windows 10 cannot access native GPU resources (more specifically NVIDIA CUDA) under normal circumstances.

You need to install or setup the following tools

1. Git
2. Docker and Docker Compose
3. VS Code

If you are aWindows user, please install additional toolsfirst.

1. Windows Terminal
2. Windows Subsystem for Linux (WSL)

See the following subsectionsin order to install or set up the necessary tools.

Download Windows Terminal fromhere to get a better terminal experience.

You must be running Windows 10 version 2004 and higher (Build 19041 and higher) or Windows 11.If you're running an older build, or just prefer not to use the install command and would like step-by-step directions, seeWSL manual installation steps for older versions.

1. Open theWindows Terminal. Then, installWSL2 and the Ubuntu-20.04 distribution as default with the following command.

   wsl--install-d Ubuntu-20.04wsl--set-default Ubuntu-20.04

2. Check if your Ubuntu-20.04 distribution usesWSL 2.

   wsl-l-v

   If your Ubuntu-20.04 distribution uses WSL 1, please use the following command to set your Ubuntu-20.04 distribution to use WSL 2.

   wsl--set-version Ubuntu-20.042

We will use Git installed in the container for version control. VS Code will detect the git configuration file in the host and bring the basic configuration to the container (thanks to VS Code 🙌), so please follow the instructions below to set it up

1. Open theWindows Terminal and start the Ubuntu-20.04 shell.

   wsl

2. Set global configuration file (~/.gitconfig).

   ℹ️ You should change the YOUR_USERNAME and YOUR_EMAIL to your own.

   git config --global user.name"YOUR_USERNAME"git config --global user.email"YOUR_EMAIL"

I think you can set up basic ~/.gitconfig file smoothly. 😃

ℹ️ You should know the terminology of container technology.

• Docker: A program that implements container technology.
• Container: Similar to virtual environments. Your application will run inside the container.
• Image: Think of it as a snapshot of the container. Can quickly generate containers from images.

1. DownloadDocker desktop for Windows fromofficial website and install it.
2. Start Docker Desktop, and selectSettings (gear icon in the upper right corner) > General to check whether theUse WSL 2 based engine check box is selected. If it is not checked, please select it.

If you want to use NVIDIA GPU in container you also need to installNVIDIA Container Toolkit.I think you can install docker, docker compose and NVIDIA Container Toolkit smoothly by yourself. 😃

1. Download VS Code from theofficial website and install it.
2. InstallRemote Development fromhere.

⚠️

• This is an instruction to establish a development environment, not a deployment environment.
• Make sure docker is running in the background.

1. StartVS Code.
2. PressF1 orctrl+shift+p to open Command Palette (interactive prompt will appear in the top center)
3. TypeRemote-WSL: New WSL Window to search for command (search function is not case sensitive).
4. Use the keyboard up and down keys to select theRemote-WSL: New WSL Window command and press "Enter" (the selected command will be highlighted) or use the mouse to click theRemote-WSL: New WSL Window command.
5. Switch to the new window connected to WSL. Check for theWSL: Ubuntu-20.04 prompt on the left side of the status bar (at the bottom of the window).
6. ClickSource Control tab in the left side bar.
7. Click theClone Repository to clone this repo.
8. Paste thehttps://github.com/yezhengkai/acea-smart-water-analytics.git into the interactive prompt at the top center of the window and press "Enter".
9. In the interactive prompt at the top center of the window, select thepreferred path to store the cloned repository and press "Enter".
10. After the clone is successful, a notification will appear in the lower right corner to remind you to open the cloned repository. Please click"Open".
11. After successfully opening the project in WSL, a notification will be displayed in the lower right corner to remind you that you can use the container. Click"Reopen in Container" to building the image and run the container.
12. When image and container are successfully built, pressctrl+shift+` in VS Code to open integrated terminal . Install development dependencies and setup pre-commit hooks with the following command.

    inv env.init-dev

13. Your development environment has been well established. You can leave VS Code or go to next section.

1. StartVS Code.
2. ClickSource Control tab in the left side bar.
3. Click theClone Repository to clone this repo.
4. Paste thehttps://github.com/yezhengkai/acea-smart-water-analytics.git into the interactive prompt at the top center of the window and press "Enter".
5. In the interactive prompt at the top center of the window, select thepreferred path to store the cloned repository and press "Enter".
6. After the clone is successful, a notification will appear in the lower right corner to remind you to open the cloned repository. Please click"Open".
7. After successfully opening the project, a notification will be displayed in the lower right corner to remind you that you can use the container. Click"Reopen in Container" to building the image and run the container.
8. When image and container are successfully built, pressctrl+shift+` in VS Code to open integrated terminal .Install development dependencies and setup pre-commit hooks with the following command.

   inv env.init-dev

9. Your development environment has been well established. You can leave VS Code or go to next section.

⚠️

• Make sure docker is running in the background.
• Ensure that the development environment is set up correctly.
• If your container is gone, open the folder in VS Code(note for Windows user: VS Code should open in WSL) and pressF1 orctrl+shift+p to useRemote-Containers: Reopen in Container command.
• If devcontainer has changed, open the folder in VS Code(note for Windows user: VS Code should open in WSL) and pressF1 orctrl+shift+p to useRemote-Containers: Rebuild and Reopen in Container.

Follow the instruction below to run container.

1. StartVS Code.
2. ClickRemote Explorer tab in the left side bar.
3. SelectContainers in the drop-down menu to display the available Dev Containers.
4. Click theOpen Folder in Container icon on the right side of the acea-smart-water-analytics label.
5. The VS Code will reopen the window that connect the container (check if there is a prompt forDev Container: acea-smart-water-analytics in the lower left corner of the window).

⚠️ The predefined tasks defined in thetasks directory use relative paths, so make sure that the current working directory isacea-smart-water-analytics

When VS Code is connected to the container, you can use the integrated terminal (pressctrl+shift+` to open terminal) to run following tasks.

• Install development dependencies and setup pre-commit hooks.

  inv env.init-dev

• Download all dependencies of the current julia project and precompile all dependencies.

  inv env.julia-instantiate

• Runjupyter lab at notebooks directory.

  inv nb.lab

• RunPluto.jl at notebooks directory.

  inv nb.pluto

• Format your code throughJuliaFormatter.jl.

  inv style.reformat

• Run test coverage check.

  inv test.cov

• Run test cases.

  invtest

• Install WSL
• Install Docker Desktop on Windows
• Using Docker as a Dev Environment with VS Code: Part 2
• DrWatson.jl Docs
• GitHub: terasakisatoshi/jldev_poetry
• GitHub: cjolowicz/hypermodern-python
• GitHub: Lee-W/cookiecutter-python-template
• GitHub: dgoings/docker-dev-example