Neural Network Workshop – Lab 9 Neural Network as a Service

Setup Your Flask Service

Imports

  1. Create a new file named application.py
  2. Import the needed packages
    1. numpy as np
    2. flash
    3. sys
    4. keras.models
    5. os
    6. StandardScalar from sklearn.preprocessing

Reference Material

https://docs.python.org/3/reference/import.html

 

Full Solution

Add the following to the file application.py:

# import the necessary packages
import numpy as np
import flask
import sys
from keras.models import model_from_json
import os
from sklearn.preprocessing import StandardScaler

[collapse]

[collapse]
Paths

  1. Create a variable named basepath, it should either be the current directory or the content of the environment variable ‘WEBROOT_PATH’. (For Azure)
  2. Append the folder out to the basepath

Reference Material

https://docs.python.org/3.6/library/os.path.html

https://docs.python.org/2/library/os.html#process-parameters

Hint 1

Read the environment with this command:

 os.environ.get(???)

[collapse]
Hint 2

Specify both the variable to examine and the default of “.”:

basepath = os.environ.get('WEBROOT_PATH', '.')

[collapse]
Hint 3

Use join to create the desired path:

basepath = os.path.join (basepath, "out")

[collapse]
Full Solution 
# find the right path for batch ai vs local
basepath = os.environ.get('WEBROOT_PATH', '.') 
basepath = os.path.join (basepath, "out")

[collapse]

[collapse]
The Main Loop

NOTE: This main loop should go to the very bottom of your file and stay there, everything else need to be defined before it.  Initializing Flash should go at the beginning.

  1. Initialize flask with a parameter of __name__
  2. Set the variable model equal to none
  3. Create a main loop for flask (don’t spend time on this, just cut and paste the code, it is not obvious how to make this both local and Azure friendly)

Reference Material

http://flask.pocoo.org/docs/1.0/api/

http://flask.pocoo.org/snippets/20/

Hint 1

Initialize Flask:

app = flask.Flask(__name__)

[collapse]
Hint 2

Define Model:

model = None

[collapse]
Hint 3

Create the Main loop waiting for input:

# if this is the main thread of execution first load the model and
# then start the server

if __name__ == "__main__":
    print(("* Loading Keras model and Flask starting server..."
        "please wait until server has fully started"))
    load_model()
    app.run()

if __name__ == "main":    
    load_model()

[collapse]
Full Solution 
# initialize our Flask application and the Keras model
app = flask.Flask(__name__)
model = None

# if this is the main thread of execution first load the model and
# then start the server
if __name__ == "__main__":
    print(("* Loading Keras model and Flask starting server..."
    "please wait until server has fully started"))
    load_model()
    app.run()

if __name__ == "main": 
    load_model()

[collapse]

[collapse]

 

Load the Model during Startup

Create load_model function

  1. Create a function named load_model
  2. Initialize two global variables named model and scalar

Reference Material

https://www.w3schools.com/python/python_functions.asp

https://www.python-course.eu/python3_global_vs_local_variables.php

Hint 1

Add the following function, this should be defined near the top of the module and must before any calls to load_model:

def load_model():

[collapse]
Hint 2

Create the globals:

    global model

    global scalar
[collapse]
Full Solution 
def load_model():
    global model
    global scalar

[collapse]

[collapse]
Deserialize the Model, Scalar, and apply the Training to the Model

  1. For the sake of speed, cut and paste the solution, and examine it, also check out the reference on exception handling

Reference Material

https://realpython.com/python-exceptions/#the-try-and-except-block-handling-exceptions

Full Solution

Update the load_model function to look like this:

def load_model():
    global model
    global scalar

    try:
        # load json and create model
        json_file = open(os.path.join (basepath, 'model.json'), 'r')
        loaded_model_json = json_file.read()
        json_file.close()
        model = model_from_json(loaded_model_json)

        #deserialize the scalar to pickle
        import pickle
        scalar = pickle.load(open (os.path.join (basepath, "standardscalar.pickle"), "rb"))

        # load weights into new model
        model.load_weights(os.path.join (basepath, 'model.h5'))
        print("Loaded model from disk")

    except OSError as err:
        output = "OSError: " + err.strerror + " Filename: " +  err.filename
        app.logger.error(output)
        print(output)       

    except (Exception):
        output = "Error: " + sys.exc_info()
        app.logger.error(output)
        print(output)

[collapse]

[collapse]

 

Create REST Handlers

ErrorHandler (500)s

  1. Create a Flask ErrorHandler that responds with a 500 after an exception and log the error to the console.

Reference Material

http://flask.pocoo.org/docs/1.0/api/

Hint 1

Define an internal_error function:

def internal_error(exception):
    return 0

[collapse]
Hint 2

Decorate it with an errorhandler adding this line above the internal_error function:

@app.errorhandler(500)

[collapse]
Hint 3

Log the error adding this line to the beginning of the function:

app.logger.error(output)

[collapse]
Full Solution 
@app.errorhandler(500)
def internal_error(exception):
    app.logger.error(exception)
    return 0

[collapse]

[collapse]
POST

  1. Define a predict function that handles POST for the uri “/predict”
  2. Create the stub for the code

Reference Material

https://keras.io/models/sequential/

http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html

Hint 1

Create your predict function like this:

@app.route("/predict", methods=["POST"])
def predict():

[collapse]
Full Solution

Add the skeleton to get this as the complete answer:

@app.route("/predict", methods=["POST"]) 
def predict():

    # initialize the data dictionary that will be returned from the
    # view
    data = {"success": False}

    # ensure an image was properly uploaded to our endpoint
    if flask.request.method == "POST":

        # indicate that the request was a success
        data["success"] = True

    # return the data dictionary as a JSON response
    resp = flask.jsonify(data)
    resp.headers.add('Access-Control-Allow-Origin', '*')

    return resp

[collapse]

[collapse]
Make Prediction

  1. Read the JSON body of the document that was posted.
  2. Preprocess the input placing it in an array and converting it to float32.
  3. Apply the same Scalar Transform that was applied during training.
  4. Perform a predict on the model with the input, print it to the console for easy debugging.
  5. Return a JSON document containing the raw prediction and whether or not the job was won or not (> .5 = true).

Reference Material

http://flask.pocoo.org/docs/1.0/api/

https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.array.html

http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html

https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.ndarray.astype.html

https://keras.io/models/model/

https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.append.html

http://flask.pocoo.org/docs/1.0/api/

Hint 1

Read the JSON body:

jsonIn = flask.request.get_json()

[collapse]
Hint 2

Preprocess the input placing it in an array and converting it to float32:

input = np.array([[jsonIn['RouteToMarket_Other'], jsonIn['RouteToMarket_Reseller'], jsonIn['RouteToMarket_Telecoverage'], jsonIn['RouteToMarket_Telesales'], jsonIn['SalesStageChangeCount'], jsonIn['TotalDaysIdentifiedThroughClosing'], jsonIn['TotalDaysIdentifiedThroughQualified'], jsonIn['RevenueFromClientPastTwoYears'], jsonIn['RatioDaysIdentifiedToTotalDays'], jsonIn['RatioDaysValidatedToTotalDays'], jsonIn['RatioDaysQualifiedToTotalDays'], jsonIn['DealSizeCategory']]])
input = input.astype('float32')

[collapse]
Hint 3

Apply the same Scalar Transform that was applied during training:

input = scalar.transform

[collapse]
Hint 4

Perform a predict on the model with the input, print it to the console for easy debugging:

preds = model.predict(input)
print(preds)

[collapse]
Hint 5

Return a JSON document containing the raw prediction and whether or not the job was won or not (> .5 = true):

if (preds[0][0] > 0.5):
    won = "True"
else:
    won = "False"

data["predictions"] = []
data["predictions"].append({"probability": float(preds[0][0]), "won": won})

[collapse]
Full Solution

Place this code after the If statement checking to see that it is a POST:

       
        # read the json doc
        jsonIn = flask.request.get_json()
        
        # preprocess the data with the standard scalar
        input = scalar.transform(np.array([[jsonIn['RouteToMarket_Other'], jsonIn['RouteToMarket_Reseller'], jsonIn['RouteToMarket_Telecoverage'], jsonIn['RouteToMarket_Telesales'], jsonIn['SalesStageChangeCount'], jsonIn['TotalDaysIdentifiedThroughClosing'], jsonIn['TotalDaysIdentifiedThroughQualified'], jsonIn['RevenueFromClientPastTwoYears'], jsonIn['RatioDaysIdentifiedToTotalDays'], jsonIn['RatioDaysValidatedToTotalDays'], jsonIn['RatioDaysQualifiedToTotalDays'], jsonIn['DealSizeCategory']]]))
        input = input.astype('float32')
        # classify the input image and then initialize the list
        # of predictions to return to the client

        preds = model.predict(input)
        print(preds)
        
        if (preds[0][0] > 0.5):
            won = "True"
        else:
            won = "False"
        
        data["predictions"] = []
        data["predictions"].append({"probability": float(preds[0][0]), "won": won})
    
        # indicate that the request was a success
        data["success"] = True

[collapse]

[collapse]

 

Try your Model

Create JSON document to post

  1. Add the following to a file named request.json:
{
    "RouteToMarket_Other": 0,
    "RouteToMarket_Reseller": 1,
    "RouteToMarket_Telecoverage": 0,
    "RouteToMarket_Telesales": 0,
    "SalesStageChangeCount": 9,
    "TotalDaysIdentifiedThroughClosing": 119,
    "TotalDaysIdentifiedThroughQualified": 97,
    "RevenueFromClientPastTwoYears": 0,
    "RatioDaysIdentifiedToTotalDays": 0.81,
    "RatioDaysValidatedToTotalDays": 0.644,
    "RatioDaysQualifiedToTotalDays": 0.245,
    "DealSizeCategory": 6
}

[collapse]
Use curl to test the service

  1. Run your web service.
  2. Open an Anaconda Prompt.
  3. Enter the command:
curl -X POST -d @request.json "http://localhost:5000/predict" -H "Content-Type: application/json"

[collapse]

 

Lab Complete!

 

Extra Credit – Output the model

Explore Keras

Keras has a lot to offer, explore the documentation:
https://keras.io

[collapse]