Tensorflow and Keras

Tensorflow and Keras specific learning

First, let's get some vocab down.

Sequential Model?: A neural network architecture, in Keras language. A sequential model is made up of layers, and the configuration of layers in a neural network is a craft, not a science.

Training: The use of the .fit() function on a Sequential model, which passes the dataset through it a number of times to adjust the weights.

Validation: After a training epoch, if 'validation_spit' or 'validation_data' is defined, the model will be run over data put aside that it hasn't seen yet, for logging and evaluation purposes.

Sample: One element of a dataset. ie. one review.

Batch: a set of N samples. The samples in a batch are processed independently, in parallel. If training, a batch results in only one update to the model. A batch generally approximates the distribution of the input data better than a single input. The larger the batch, the better the approximation; however, it is also true that the batch will take longer to process and will still result in only one update. For inference (evaluate/predict), it is recommended to pick a batch size that is as large as you can afford without going out of memory (since larger batches will usually result in faster evaluation/prediction).

Epoch: an arbitrary cutoff, generally defined as "one pass over the entire dataset", used to separate training into distinct phases, which is useful for logging and periodic evaluation.

Handy things to note:

• Layers can be frozen (weights will not be updated) by passing trainable= False to them as a parameter.
• shuffle=True can be passed to the model.fit() function to shuffle the training data.

Layers

• Dense are standard densely connected NN layers.
• Activation are activation functions applied to an output.
• Dropout will set a fraction of input units to 0 during training time.