Repository files navigation

This project isARCHIVED and it will not receive maintenance anymore because the MLflowfinally supports basic authentication natively! \o/

version:"3.9"services:mlflow:image:ghcr.io/mlflow/mlflow:v2.5.0ports:      -5000:5000command:mlflow server --host 0.0.0.0 --app-name basic-auth

Find further details inMLflow Authentication — MLflow 2.5.0 documentation.

A dockerized MLflow Tracking Server with basic auth (username and password).

You will have three options to deploy the server:AWS,Heroku, andlocal.

We provide aTerraform stack that can be easily used to deploy the MLflow Tracking Server.

NOTE: This project is not intended to be used for production deployments. It is intended to be used for testing and development.

The environment variables below are required to deploy this project.

• AWS Account
• AWS CLI
• Terraform CLI

To deploy MLflow, you'll need to:

1. Create an AWS account if you don't already have one.

2. Configure AWS CLI to use your AWS account.

3. Clone this repository.

git clone https://github.com/DougTrajano/mlflow-server.git

4. Openmlflow-server/terraform folder.

cd mlflow-server/terraform

5. Run the following command to create all the required resources:

terraform initterraform apply -var mlflow_username="YOUR-USERNAME" -var mlflow_password="YOUR-PASSWORD"

Multiple usernames and passwords can also be specified in a comma-delimited string:

terraform apply -var mlflow_username="USERNAME1,USERNAME2,USERNAME3" -var mlflow_password="PASSWORD1,PASSWORD2,PASSWORD3"

See a full list of variables that can be used interraform/variables.tf.

6. Type "yes" when prompted to continue.

Plan: 21 to add, 0 to change, 0 to destroy.Changes to Outputs:  + artifact_bucket_id = (known after apply)  + mlflow_password    = (sensitive value)  + mlflow_username    = "doug"  + service_url        = (known after apply)  + status             = (known after apply)Do you want to perform these actions?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.  Enter a value: yes

This will create the following resources:

• AnS3 bucket is used to store MLflow artifacts.
• An [IAM role andpolicy that allows MLflow to access the S3 bucket.
• AnAurora RDS Serverless database (PostgreSQL) is used to store MLflow data.
• AnApp Runner that will run the MLflow Tracking Server.
• (Optional) A set of network resources such asVPC,Subnet, andSecurity group.

• Heroku Account
• AWS Account
  • TheHeroku deployment will use an Amazon S3 bucket for storing the MLflow tracking data.
• AWS CLI
• Terraform CLI

1. Create an AWS account if you don't already have one.

2. Configure AWS CLI to use your AWS account.

3. Clone this repository.

git clone https://github.com/DougTrajano/mlflow-server.git

4. Openmlflow-server/terraform folder.

cd mlflow-server/terraform

5. Run the following command to create only the S3 bucket

terraform initterraform apply -var environment="heroku" -target="module.s3"

6. Type "yes" when prompted to continue.

Plan: 5 to add, 0 to change, 0 to destroy.Changes to Outputs:  + artifact_bucket_id = (known after apply)Do you want to perform these actions?  Terraform will perform the actions described above.  Only 'yes' will be accepted to approve.  Enter a value: yes

2. Create an IAM Policy for the S3 bucket as follows:

{"Version":"2012-10-17","Statement": [        {"Effect":"Allow","Action": ["s3:ListBucket"            ],"Resource":"arn:aws:s3:::mlflow-heroku-20220723133820303500000001"          },        {"Effect":"Allow","Action": ["s3:*","s3-object-lambda:*"            ],"Resource":"arn:aws:s3:::mlflow-heroku-20220723133820303500000001/*"        }    ]}

3. Create an IAM User and attach the IAM Policy previously created.

Take note of the IAM User access key and secret key, you'll need them in the step 5.

4. Click on the "Deploy to Heroku" button below.

5. Follow the instructions on the new page to create an MLflow Tracking Server.

• Docker andDocker Compose.

1. Clone this repository.

git clone https://github.com/DougTrajano/mlflow-server.git

2. Open themlflow-server folder.

cd mlflow-server

3. Run the following command to create all the required resources:

docker-compose up -d --build

The link that you will use to access the MLflow Tracking Server will depend on the deployment method you choose.

• ForAWS, the link will be something likehttps://XXXXXXXXX.aws-region.awsapprunner.com/.
  • You can find it in theAWS App Runner console.
• ForHeroku, the link will be something likehttps://XXXXXXXXX.herokuapp.com/.
  • You can find it in theHeroku dashboard.
• ForLocal, the link will be something likehttp://localhost:80/.

Also, you can track your experiments using MLflow API.

importosimportmlflowos.environ["MLFLOW_TRACKING_URI"]="<<YOUR-MLFLOW-TRACKING-URI>>"os.environ["MLFLOW_EXPERIMENT_NAME"]="<<YOUR-EXPERIMENT-NAME>>"os.environ["MLFLOW_TRACKING_USERNAME"]="<<YOUR-MLFLOW-USERNAME>>"os.environ["MLFLOW_TRACKING_PASSWORD"]="<<YOUR-MLFLOW-PASSWORD>>"# AWS AK/SK are required to upload artifacts to S3 Bucketos.environ["AWS_ACCESS_KEY_ID"]="<<AWS-ACCESS-KEY-ID>>"os.environ["AWS_SECRET_ACCESS_KEY"]="<<AWS-SECRET-ACCESS-KEY>>"SEED=1993mlflow.start_run()mlflow.log_param("seed",SEED)mlflow.end_run()

• Managing your machine learning lifecycle with MLflow and Amazon SageMaker | AWS Machine Learning Blog
• Introducing AWS App Runner
• MLflow Documentation (current version: 1.27.0)
• soundsensing/mlflow-easyauth: Deploy MLflow with HTTP basic authentication using Docker