Repository files navigation

In this Code Pattern, we will use German Credit data to train, create, and deploy a machine learning model usingWatson Machine Learning. We will create a data mart for this model withWatson OpenScale and configure OpenScale to monitor that deployment, and inject seven days' worth of historical records and measurements for viewing in the OpenScale Insights dashboard.

When the reader has completed this Code Pattern, they will understand how to:

• Create and deploy a machine learning model using the Watson Machine Learning service
• Setup Watson OpenScale Data Mart
• Bind Watson Machine Learning to the Watson OpenScale Data Mart
• Add subscriptions to the Data Mart
• Enable payload logging and performance monitor for subscribed assets
• Enable Quality (Accuracy) monitor
• Enable Fairness monitor
• Enable Drift montitor
• Score the German credit model using the Watson Machine Learning
• Insert historic payloads, fairness metrics, and quality metrics into the Data Mart
• Use Data Mart to access tables data via subscription

1. The developer creates a Jupyter Notebook on Watson Studio.
2. The Jupyter Notebook is connected to a PostgreSQL database, which is used to store Watson OpenScale data.
3. The notebook is connected to Watson Machine Learning and a model is trained and deployed.
4. Watson OpenScale is used by the notebook to log payload and monitor performance, quality, and fairness.

• AnIBM Cloud Account
• IBM Cloud CLI
• IBM Cloud Object Storage (COS)
• An account onIBM Watson Studio.

1. Clone the repository
2. Use free internal DB or Create a Databases for PostgreSQL DB
3. Create a Watson OpenScale service
4. Create a Watson Machine Learning instance
5. Create a notebook in IBM Watson Studio on Cloud Pak for Data
6. Run the notebook in IBM Watson Studio
7. Setup OpenScale to utilize the dashboard

git clone https://github.com/IBM/monitor-wml-model-with-watson-openscalecd monitor-wml-model-with-watson-openscale

Note: Services created must be in the same region, and space, as your Watson Studio service.

• Using theIBM Cloud Dashboard catalog, search for PostgreSQL and choose theDatabases for Postgresservice.
• Wait for the database to be provisioned.
• Click on theService Credentials tab on the left and then clickNew credential + to create the service credentials. Copy them or leave the tab open to use later in the notebook.
• Make sure that the cell in the notebook that has:

KEEP_MY_INTERNAL_POSTGRES=True

is changed to:

KEEP_MY_INTERNAL_POSTGRES=False

Create Watson OpenScale, either on the IBM Cloud or using your On-Premise Cloud Pak for Data.

• Under theSettings tab, scroll down toAssociated services, click+ Add service and chooseWatson:

  

• Search forMachine Learning, Verify this service is being created in the same space as the app in Step 1, and clickCreate.

  

• Alternately, you can choose an existing Machine Learning instance and click onSelect.

• The Watson Machine Learning service is now listed as one of yourAssociated Services.

• In a different browser tab go tohttps://cloud.ibm.com/ and log in to the Dashboard.

• Click on your Watson Machine Learning instance underServices, click onService credentials and then onView credentials to see the credentials.

  

• Save the credentials in a file. You will use them inside the notebook.

• InWatson Studio or your on-premise Cloud Pak for Data, clickNew Project + under Projects or, at the top of the page click+ New and choose the tile forData Science and thenCreate Project.

• Using the project you've created, click on+ Add to project and then choose theNotebook tile, OR in theAssets tab underNotebooks choose+ New notebook to create a notebook.

• Select theFrom URL tab. [1]

• Enter a name for the notebook. [2]

• Optionally, enter a description for the notebook. [3]

• ForRuntime select theDefault Spark Python 3.7 option. [4]

• UnderNotebook URL provide the following url:https://raw.githubusercontent.com/IBM/monitor-wml-model-with-watson-openscale/master/notebooks/WatsonOpenScaleMachineLearning.ipynb

Note: The current default (as of 8/11/2021) is Python 3.8. This will cause an error when installing thepyspark.sql SparkSession library, so make sure that you are using Python 3.7

• Click theCreate notebook button. [6]

Follow the instructions forProvision services and configure credentials:

Your Cloud API key can be generated by going to theUsers section of the Cloud console.

• From that page, click your name, scroll down to theAPI Keys section, and clickCreate an IBM Cloud API key.

• Give your key a name and clickCreate, then copy the created key and paste it below.

Alternately, from theIBM Cloud CLI :

ibmcloud login --ssoibmcloud iam api-key-create'my_key'

• Enter theCLOUD_API_KEY in the cell1.1 Cloud API key.

• In yourIBM Cloud Object Storage instance, create a bucket with a globally unique name. The UI will let you know if there is a naming conflict. This will be used in cell1.3.1 asBUCKET_NAME.

• In yourIBM Cloud Object Storage instance, get the Service Credentials for use asCOS_API_KEY_ID,COS_RESOURCE_CRN, andCOS_ENDPOINT:

  

• Add the COS credentials in cell1.2 Cloud object storage details.

• Insert your BUCKET_NAME in the cell1.2.1 Bucket name.

• Either use the internal Database, which requiresNo Changes or Add yourDB_CREDENTIALS after reading the instructions preceeding that cell and change the cellKEEP_MY_INTERNAL_POSTGRES = True to becomeKEEP_MY_INTERNAL_POSTGRES = False.

• Move your cursor to each code cell and run the code in it. Read the comments for each cell to understand what the code is doing.Important when the code in a cell is still running, the label to the left changes toIn [*]:.Donot continue to the next cell until the code is finished running.

Now that you have created a machine learning model, you can utilize theOpenScale dashboard to gather insights.

You can find a sample notebook with output forWatsonOpenScaleMachineLearning-with-outputs.ipynb.

• Go to the instance ofWatson OpenScale for an IBM Cloud deployment, or to your deployed instance on Cloud Pak for Data on-premise version. Choose theInsights tab to get an overview of your monitored deployments, Accuracy alerts, and Fairness alerts.

• Click on the tile for theSpark German Risk Deployment and you can see tiles for theFairness,Accuracy, andPerformance monitors.

• Click on one of the tiles, such asDrift to view details. Click on a point on the graph for more information on that particular time slice.

• You will see which types of drift were detected. Click on the number to bring up a list of transactions that led to drift.

• Click on theExplain icon on the left-hand menu and you'll see a list of transactions that have been run using an algorithm to provide explainability. Choose one and clickExplain.

• You will see a graph showing all the most influential features with the relative weights of contribution to thePredicted outcome.

• Click theInspect tab and you can experiment with changing the values of various features to see how that would affect the outcome. Click theRun analysis button to see what changes would be required to change the outcome.

Apache 2.0