Serving an Iris Classifier with Seldon¶

Goal of Notebook¶

The goal of this notebook is to wrap a scikit-learn Python model for use as a prediction microservice with Seldon Core.

Setup Python¶

Write requirements.txt and configure our Python virtual environment.

In [ ]:

%%writefile requirements.txt
scikit-learn
dill
pandas
sklearn
seldon-core
requests
matplotlib

%%writefile requirements.txt scikit-learn dill pandas sklearn seldon-core requests matplotlib

In [ ]:

!if [ ! -d "./venv" ]; then python -m venv venv; fi

!if [ ! -d "./venv" ]; then python -m venv venv; fi

In [ ]:

!source ./venv/bin/activate

!source ./venv/bin/activate

In [ ]:

!if [ ! -d "/home/jovyan/.config/pip" ]; then mkdir /home/jovyan/.config/pip; fi

!if [ ! -d "/home/jovyan/.config/pip" ]; then mkdir /home/jovyan/.config/pip; fi

Seldon only runs in the AAW dev environment; modify pip.conf to retrieve packages from the internet.

In [ ]:

%%writefile /home/jovyan/.config/pip/pip.conf
[global]
index-url = https://jfrog.aaw.cloud.statcan.ca/artifactory/api/pypi/pypi-remote/simple

%%writefile /home/jovyan/.config/pip/pip.conf [global] index-url = https://jfrog.aaw.cloud.statcan.ca/artifactory/api/pypi/pypi-remote/simple

In [ ]:

%%capture
%pip install -r requirements.txt

%%capture %pip install -r requirements.txt

We can add the kernel to the JupyterLab launcher for easy reusing.

In [ ]:

!python -m ipykernel install --user --name=sklearn_iris_jsondata

!python -m ipykernel install --user --name=sklearn_iris_jsondata

Deploy Model¶

Write a SeldonDeployment YAML file for the containerized model.

In [ ]:

%%writefile sklearn_iris_jsondata_deployment.yaml
apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: seldon-deployment-example
  namespace: bryanpaget
spec:
  name: sklearn-iris-deployment
  annotations:
    sidecar.istio.io/inject: "false"
  predictors:
  - componentSpecs:
    - spec:
        containers:
        - image: seldonio/sklearn-iris:0.3
          imagePullPolicy: IfNotPresent
          name: sklearn-iris-classifier
    graph:
      children: []
      endpoint:
        type: REST
      name: sklearn-iris-classifier
      type: MODEL
    name: sklearn-iris-predictor
    replicas: 1

%%writefile sklearn_iris_jsondata_deployment.yaml apiVersion: machinelearning.seldon.io/v1alpha2 kind: SeldonDeployment metadata: name: seldon-deployment-example namespace: bryanpaget spec: name: sklearn-iris-deployment annotations: sidecar.istio.io/inject: "false" predictors: - componentSpecs: - spec: containers: - image: seldonio/sklearn-iris:0.3 imagePullPolicy: IfNotPresent name: sklearn-iris-classifier graph: children: [] endpoint: type: REST name: sklearn-iris-classifier type: MODEL name: sklearn-iris-predictor replicas: 1

Deploy the SeldonDeployment.

In [ ]:

!kubectl create -f sklearn_iris_jsondata_deployment.yaml

!kubectl create -f sklearn_iris_jsondata_deployment.yaml

In [ ]:

!kubectl rollout status deploy \
    $(kubectl get deploy -l seldon-deployment-id=seldon-deployment-example \
        -o jsonpath='{.items[0].metadata.name}')

!kubectl rollout status deploy \ $(kubectl get deploy -l seldon-deployment-id=seldon-deployment-example \ -o jsonpath='{.items[0].metadata.name}')

Is the service available?

In [ ]:

for i in range(60):
    state = !kubectl get sdep seldon-deployment-example -o jsonpath='{.status.state}'
    state = state[0]
    print(state)
    if state == "Available":
        break
    time.sleep(1)
assert state == "Available"

for i in range(60): state = !kubectl get sdep seldon-deployment-example -o jsonpath='{.status.state}' state = state[0] print(state) if state == "Available": break time.sleep(1) assert state == "Available"

Test Model¶

With the virtual environment setup and activated, import the necessary Python libraries.

In [ ]:

import json
import pandas as pd

import json import pandas as pd

The Iris Classifier takes a set of four measurements, in this case [5.964, 1.006, 2.081, 1.031], and it returns a set of probabilities, one for each of three types of iris flowers. You can read more about iris data here. Let's send some data to the model.

In [ ]:

res = !curl -v seldon-deployment-example-sklearn-iris-predictor.bryanpaget.svc.cluster.local:8000/api/v0.1/predictions \
   -H 'Content-Type: application/json' \
   -d '{"data": {"ndarray": [[5.964, 1.006, 2.081, 1.031]]}}'

res = !curl -v seldon-deployment-example-sklearn-iris-predictor.bryanpaget.svc.cluster.local:8000/api/v0.1/predictions \ -H 'Content-Type: application/json' \ -d '{"data": {"ndarray": [[5.964, 1.006, 2.081, 1.031]]}}'

Results¶

These are the results in raw JSON.

In [ ]:

parsed = json.loads(res[-1])
print(json.dumps(parsed, indent=4, sort_keys=True))

parsed = json.loads(res[-1]) print(json.dumps(parsed, indent=4, sort_keys=True))

This plotting function reveals a bar chart with the classifier's inference.

In [ ]:

def display_iris_classifier_results(res):
    """Takes the classification results from the iris classifier and plots them in a bar chart for easy viewing of class probabilities.

    Args:
        res (string): results from iris model served by Seldon.
    """
    results = res[-1]
    results = json.loads(results)
    names = results["data"]["names"]
    predictions = results["data"]["ndarray"][0]
    model = results["meta"]["requestPath"]
    model = model.popitem()
    model, version = model[0], model[1]

    df = pd.DataFrame({"Class": names, "Probability": predictions})

    df.plot(
        kind="bar",
        title=model.replace("-", " ").title(),
        xlabel="Classification",
        ylabel="Probability",
        legend=False
    )

display_iris_classifier_results(res)

def display_iris_classifier_results(res): """Takes the classification results from the iris classifier and plots them in a bar chart for easy viewing of class probabilities. Args: res (string): results from iris model served by Seldon. """ results = res[-1] results = json.loads(results) names = results["data"]["names"] predictions = results["data"]["ndarray"][0] model = results["meta"]["requestPath"] model = model.popitem() model, version = model[0], model[1] df = pd.DataFrame({"Class": names, "Probability": predictions}) df.plot( kind="bar", title=model.replace("-", " ").title(), xlabel="Classification", ylabel="Probability", legend=False ) display_iris_classifier_results(res)

Clean Up¶

Clean up by removing the deployment.

In [ ]:

!kubectl delete -f sklearn_iris_jsondata_deployment.yaml

!kubectl delete -f sklearn_iris_jsondata_deployment.yaml