1. Import or
generate datasets
All of the
machine-learning algorithms will depend on datasets. Datasets can
either be generated, or chosen to use an outside source. Sometimes it
is better to rely on generated data if you main aim is just to know
the expected outcome. Most of the time, several public datasets can
be found on the internet.
2. Transform and
normalize data
Normally, the input
datasets do not come in the shape TensorFlow would expect, so we need
to transform them to the TensorFlow accepted shape. The data is
usually not in the correct dimension or type that the algorithms are
expecting. We will have to transform our data before we can use it.
Most algorithms also expect normalized data too.
TensorFlow has
built-in functions that can normalize the data as follows:
data =
tf.nn.batch_norm_with_global_normalization(…)
3. Partition
datasets into train, test, and validation sets
We generally want to
test our algorithms on different sets that we have trained on. Also,
many algorithms require hyperparameter tuning, so we set aside a
validation set for determining the best set of hyperparameters.
4. Set algorithm
parameters (hyperparameters)
Our algorithms
usually have a set of parameters that we hold constant throughout the
procedure. For example, it can be the number of iterations, the
learning rate, or other fixed parameters of our choosing. It is
considered good form to initialize these together to keep the
consistancy through examples, like:
learning_rate = 0.01
batch_size = 100
iterations = 1000
5. Initialize
variables and placeholders
TensorFlow depends
strongly on knowing what it can and cannot modify. It will
modify/adjust the variables and weight/bias during optimization to
minimize a loss function. To accomplish this, we feed in data through
placeholders. We need to initialize both of these variables and
placeholders
with size and type,
so that TensorFlow knows what to expect. TensorFlow also needs to
know the type of data to expect eg. float32, float64 and float16 .
Note that more bytes in precision result in slower algorithms, but
the less results in less precision. See the following code as a small
example:
a_var =
tf.constant(42)
x_input =
tf.placeholder(tf.float32, [None, input_size])
y_input =
tf.placeholder(tf.float32, [None, num_classes])
6. Define the model
structure
After we have the
data, and have initialized our variables and placeholders, we have to
define the model. This is done by building a computational graph.
TensorFlow chooses what operations and values must be the variables
and placeholders to arrive at our model outcomes. For example a
linear
model looks like:
y_pred =
tf.add(tf.mul(x_input, weight_matrix), b_matrix)
7. Declare the loss
functions
After defining the
model, we must be able to evaluate the output. This is where we
declare the loss function. The loss function is very important as it
tells us how far off our predictions are from the actual values. The
different types of loss functions are explored in greater detail
later such as implementing the Back Propagation recipe in a
TensorFlow way like below:
loss =
tf.reduce_mean(tf.square(y_actual – y_pred))
8. Initialize and
train the model
Now that we have
everything in place, we need to create an instance of our graph, feed
in the data through the placeholders, and let TensorFlow change the
variables to better predict our training data. One way to initialize
the computational graph is:
tf.Session(graph=graph)
as session:
...
session.run(...)
...
which is same as:
session =
tf.Session(graph=graph)
session.run(...)
9. Evaluate the
model
Once we have built
and trained the model, we should evaluate the model by looking at how
well it does with new data through some specified criteria. We
evaluate on the train and test set and these evaluations will allow
us to see if the model is underfit or overfit.
10. Tune
hyperparameters
Most of the time, we
will want to go back and change some of the hyperparamters, based on
the model performance. We then repeat the previous steps with
different hyperparameters and evaluate the model on the validation
set.
11. Deploy/predict
new outcomes
It is also important
to know how to make predictions on new, unseen, data. We can do this
with all of our models, once we have them trained.
How TensorFlow works:
In TensorFlow, we have to set up the data, variables, placeholders, and model before we tell the program to train and change the variables to improve the predictions. TensorFlow accomplishes this through the computational graphs. These computational graphs are a directed graphs with no recursion, which allows for computational parallelism. We create a loss function for TensorFlow to minimize. TensorFlow accomplishes this by modifying the variables in the computational graph. Tensorflow knows how to modify the variables because it keeps track of the computations in the model and automatically computes the gradients for every variable. Because of this, we can see how easy it can be to make changes and try different data sources.
From << TensorFlow Machine Learning Cookbook >>
