General flow of TensorFlow algorithms

1. Import or generate datasets

All of the machine-learning algorithms will depend on datasets. Datasets can either be generated, or chosen to use an outside source. Sometimes it is better to rely on generated data if you main aim is just to know the expected outcome. Most of the time, several public datasets can be found on the internet.

2. Transform and normalize data

Normally, the input datasets do not come in the shape TensorFlow would expect, so we need to transform them to the TensorFlow accepted shape. The data is usually not in the correct dimension or type that the algorithms are expecting. We will have to transform our data before we can use it. Most algorithms also expect normalized data too.

TensorFlow has built-in functions that can normalize the data as follows:

data = tf.nn.batch_norm_with_global_normalization(…)

3. Partition datasets into train, test, and validation sets

We generally want to test our algorithms on different sets that we have trained on. Also, many algorithms require hyperparameter tuning, so we set aside a validation set for determining the best set of hyperparameters.

4. Set algorithm parameters (hyperparameters)

Our algorithms usually have a set of parameters that we hold constant throughout the procedure. For example, it can be the number of iterations, the learning rate, or other fixed parameters of our choosing. It is considered good form to initialize these together to keep the consistancy through examples, like:

learning_rate = 0.01

batch_size = 100

iterations = 1000

5. Initialize variables and placeholders

TensorFlow depends strongly on knowing what it can and cannot modify. It will modify/adjust the variables and weight/bias during optimization to minimize a loss function. To accomplish this, we feed in data through placeholders. We need to initialize both of these variables and placeholders

with size and type, so that TensorFlow knows what to expect. TensorFlow also needs to know the type of data to expect eg. float32, float64 and float16 . Note that more bytes in precision result in slower algorithms, but the less results in less precision. See the following code as a small example:

a_var = tf.constant(42)

x_input = tf.placeholder(tf.float32, [None, input_size])

y_input = tf.placeholder(tf.float32, [None, num_classes])

6. Define the model structure

After we have the data, and have initialized our variables and placeholders, we have to define the model. This is done by building a computational graph. TensorFlow chooses what operations and values must be the variables and placeholders to arrive at our model outcomes. For example a linear

model looks like:

y_pred = tf.add(tf.mul(x_input, weight_matrix), b_matrix)

7. Declare the loss functions

After defining the model, we must be able to evaluate the output. This is where we declare the loss function. The loss function is very important as it tells us how far off our predictions are from the actual values. The different types of loss functions are explored in greater detail later such as implementing the Back Propagation recipe in a TensorFlow way like below:

loss = tf.reduce_mean(tf.square(y_actual – y_pred))

8. Initialize and train the model

Now that we have everything in place, we need to create an instance of our graph, feed in the data through the placeholders, and let TensorFlow change the variables to better predict our training data. One way to initialize the computational graph is:

tf.Session(graph=graph) as session:

...

session.run(...)

...

which is same as:

session = tf.Session(graph=graph)

session.run(...)

9. Evaluate the model

Once we have built and trained the model, we should evaluate the model by looking at how well it does with new data through some specified criteria. We evaluate on the train and test set and these evaluations will allow us to see if the model is underfit or overfit.

10. Tune hyperparameters

Most of the time, we will want to go back and change some of the hyperparamters, based on the model performance. We then repeat the previous steps with different hyperparameters and evaluate the model on the validation set.

11. Deploy/predict new outcomes

It is also important to know how to make predictions on new, unseen, data. We can do this with all of our models, once we have them trained.

How TensorFlow works:

In TensorFlow, we have to set up the data, variables, placeholders, and model before we tell the program to train and change the variables to improve the predictions. TensorFlow accomplishes this through the computational graphs. These computational graphs are a directed graphs with no recursion, which allows for computational parallelism. We create a loss function for TensorFlow to minimize. TensorFlow accomplishes this by modifying the variables in the computational graph. Tensorflow knows how to modify the variables because it keeps track of the computations in the model and automatically computes the gradients for every variable. Because of this, we can see how easy it can be to make changes and try different data sources.

From << TensorFlow Machine Learning Cookbook >>