Neural Network Workshop – Lab 7 Improve Your Model

 

 

Create the KerasClassifier

Imports

  1. Use the following imports:
# Evaluating the ANN
from keras.wrappers.scikit_learn import KerasClassifier
from sklearn.model_selection import GridSearchCV
from keras.models import Sequential
from keras.layers import Dense

[collapse]
Create the build_classifier Function

  1. Using the Sequential model from Lab 5 create a function named build_classifier that returns a sequential model.

Reference Material

https://www.w3schools.com/python/python_functions.asp

Hint 1

The contents of Lab 7 that you will use:

model = Sequential()
model.add(Dense(units = 24, activation = 'relu', input_dim=X.shape[1], name= 'Input_Layer'))
model.add(Dense(units = 24, activation = 'relu', name= 'Hidden_Layer_1'))
model.add(Dense(1, activation = 'sigmoid', name= 'Output_Layer'))
model.compile(optimizer= optimizer, loss = 'binary_crossentropy', metrics=['accuracy'])

[collapse]
Hint 2

The function definition looks like this:

def build_classifier(optimizer):
    ???
    return model

[collapse]
Full Solution 
def build_classifier(optimizer):
   model = Sequential()
   model.add(Dense(units = 24, activation = 'relu', input_dim=X.shape[1], name= 'Input_Layer'))
   model.add(Dense(units = 24, activation = 'relu', name= 'Hidden_Layer_1'))
   model.add(Dense(1, activation = 'sigmoid', name= 'Output_Layer'))
   model.compile(optimizer= optimizer, loss = 'binary_crossentropy', metrics=['accuracy'])
   return model

[collapse]

[collapse]
Create the Classifier

  1. Instantiate a KerasClassifier in the variable classifier, pass it the build_classifier function.

Reference Material

https://keras.io/scikit-learn-api/

Full Solution

Set classifier to an instance of KerasClassifier:

classifier = KerasClassifier(build_fn = build_classifier)

[collapse]

[collapse]

 

Perform the GridSearch

Create the parameters to pass to the GridSearch

  1. Create a dictionary named parameters with the following properties:
    • batch_size = an array containing 64
    • epochs = an array containing 25
    • optimizer = an array containing adam, sgd, adamax, nadam

Reference Material

https://www.w3schools.com/python/python_dictionaries.asp

Full Solution 
parameters = {'batch_size': [64],
              'epochs': [25],
              'optimizer': ['adam','sgd','adamax','nadam']}

[collapse]

[collapse]
Instantiate GridSearchCV and fit

  1. Instantiate GridSearchCV in a variable names grid_searech with the following parameters:
    • estimator = classifier
    • param_grid = parameters
    • scoring = ‘accuracy’
    • verbose = 5
  2. Call fit to execute the grid search passing X and y as parameters.

Reference Material

http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html

Hint 1

Instantiate GridSearchCV like this:

grid_search = GridSearchCV(estimator = classifier,
   param_grid = parameters,
   scoring = 'accuracy',
   verbose = 5)

[collapse]
Hint 2

Execute the Grid Search:

grid_search = grid_search.fit(X, y)

[collapse]
Full Solution 
grid_search = GridSearchCV(estimator = classifier,
   param_grid = parameters,
   scoring = 'accuracy',
   verbose = 5)
grid_search = grid_search.fit(X, y)

[collapse]

[collapse]

 

 

What were the GridSearch Results?

Retrieve the Result

  1. Create a new variable called best_parameters, set it to the best_params_ from the grid search.
  2. Create a new variable called best_accuracy, set it to the best_score_ from the grid search.

Reference Material

http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html

Hint 1

Find the best parameters:

best_parameters = grid_search.best_params_

[collapse]
Hint 2

Find the best accuracy:

best_accuracy = grid_search.best_score_

[collapse]
Full Solution 
best_parameters = grid_search.best_params_
best_accuracy = grid_search.best_score_

[collapse]

[collapse]

 

Execute the Grid Search

Run it

  1. Highlight all of the lab 7 code and execute it.
  2. After the search completes examine the values of best_parameters and best_accuracy to see the results!

[collapse]
Clean Up

  1. Highlight all of the lab 7 code and comment it out.

[collapse]

 

 

Lab Complete!

 

Extra Credit – Learn More

Learn More about the Different Optimizers

https://keras.io/optimizers/

[collapse]
Learn more about Activation Functions

https://keras.io/activations/

[collapse]
How do I know which inputs are the most important factors

ftp://ftp.sas.com/pub/neural/importance.html

[collapse]