HomeDeep LearningDive Into TensorFlow, Part II: Basic Concepts

This is the second article in the series “Dive Into TensorFlow“, here is an index of all the articles in the series that have been published to date:

Part I: Getting Started with TensorFlow

What’s Tensor?

Tensor is the core concept for TensorFlow:

TensorFlow programs use a tensor data structure to represent all data — only tensors are passed between operations in the computation graph. You can think of a TensorFlow tensor as an n-dimensional array or list.

For example, a scaler is a tensor, a vector is a tensor and a matrix is a tensor. A tensor has a rank, a shape and a static type, so a tensor can be represented as a multidimensional array of numbers.

1) Rank

Tensor rank is the number of dimensions of the tensor, for example, a rank one tensor is a vector, a rank two tensor is a matrix.

```1 2 3 4 5 6 7 8 9 10 11 12 13 In : import tensorflow as tf   # scalar rank is 0 In : scalar1 = tf.constant(10000)   # vector rank is 1 In : vector1 = tf.constant([1, 2, 3, 4, 5])   # matrix rank is 2 In : matrix1 = tf.constant([[1, 2, 3], [4, 5, 6]])   # 3-tensor rank is 3 In : cube1 = tf.constant([[, , ], [, , ], [, , ]])```

2) Shape

The TensorFlow documentation uses three notational conventions to describe tensor dimensionality: rank, shape, and dimension number. The following example shows how these relate to one another:

```1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # scalar: rank 0 shape [] In : scalar1.get_shape() Out: TensorShape([])   # vector: rank 1 shape [D0] In : vector1.get_shape() Out: TensorShape([Dimension(5)])   # matrix: rank 2 shape [D0, D1] In : matrix1.get_shape() Out: TensorShape([Dimension(2), Dimension(3)])   # 3-tensor: rank 3 shape [D0, D1, D2] In : cube1.get_shape() Out: TensorShape([Dimension(3), Dimension(3), Dimension(1)])```

Data type

In addition to rank and shape, Tensors have a data type. Following is the list of the data type:

Data type Python type Description
`DT_FLOAT` `tf.float32` 32 bits floating point.
`DT_DOUBLE` `tf.float64` 64 bits floating point.
`DT_INT8` `tf.int8` 8 bits signed integer.
`DT_INT16` `tf.int16` 16 bits signed integer.
`DT_INT32` `tf.int32` 32 bits signed integer.
`DT_INT64` `tf.int64` 64 bits signed integer.
`DT_UINT8` `tf.uint8` 8 bits unsigned integer.
`DT_STRING` `tf.string` Variable length byte arrays. Each element of a Tensor is a byte array.
`DT_BOOL` `tf.bool` Boolean.
`DT_COMPLEX64` `tf.complex64` Complex number made of two 32 bits floating points: real and imaginary parts.
`DT_COMPLEX128` `tf.complex128` Complex number made of two 64 bits floating points: real and imaginary parts.
`DT_QINT8` `tf.qint8` 8 bits signed integer used in quantized Ops.
`DT_QINT32` `tf.qint32` 32 bits signed integer used in quantized Ops.
`DT_QUINT8` `tf.quint8` 8 bits unsigned integer used in quantized Ops.

Variables

In TensorFlow, variables maintain state across executions of the graph. For example:

```1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 In : import tensorflow as tf   # Create a Variable, that will be initialized to the scalar value 100 In : var = tf.Variable(100, name="variable_counter")   # Create an Op to multiply ten to `var`. In : ten = tf.constant(10)   In : new_var = tf.mul(var, ten)   In : update = tf.assign(var, new_var)     # Variables must be initialized by running an `init` Op after having # launched the graph. We first have to add the `init` Op to the graph. In : init_op = tf.initialize_all_variables()   # Launch the graph and run the ops. In : with tf.Session() as sess: sess.run(init_op) print(sess.run(var)) for _ in range(5): sess.run(update) print(sess.run(var)) ...:   # output 100 1000 10000 100000 1000000 10000000```

Variables are in-memory buffers containing tensors, when you train a model with TensorFlow, variables can be used to hold and update parameters. Variables must be explicitly initialized and cab be saved to disk during and after training.

```1 2 3 4 5 6 7 8 9   # Create a variable with a random value. In : weights = tf.Variable(tf.random_normal([4096, 500], stddev=0.25), name="weights")   # Create another variable with the same value as 'weights'. In : weights2 = tf.Variable(weights.initialized_value(), name="weights2")   # Create another variable with twice the value of 'weights' In : weights_twice = tf.Variable(weights.initialized_value() * 2.0, name="weights_twice")```

Fetches

To fetch the outputs of operations, execute the graph with a run() call on the Session object and pass in the tensors to retrieve. Except fetching the single tensor node, you can also fetch multiple tensors:

```1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 In : constant1 = tf.constant([1.0])   In : constant2 = tf.constant([2.0])   In : constant3 = tf.constant([3.0])   In : intermed = tf.add(constant2, constant3)   In : mul = tf.mul(constant1, intermed)   In : with tf.Session() as sess: ....: result = sess.run([mul, intermed]) ....: print(result) ....: [array([ 5.], dtype=float32), array([ 5.], dtype=float32)]```

Feeds

TensorFlow provides a feed mechanism for patching a tensor directly into any operation in the graph.

A feed temporarily replaces the output of an operation with a tensor value. You supply feed data as an argument to a run() call. The feed is only used for the run call to which it is passed. The most common use case involves designating specific operations to be “feed” operations by using tf.placeholder() to create them:

```1 2 3 4 5 6 7 8 9 10 In : pla1 = tf.placeholder(tf.float32)   In : pla2 = tf.placeholder(tf.float32)   In : result = tf.mul(pla1, pla2)   In : with tf.Session() as sess: print(sess.run([result], feed_dict={pla1:[100.], pla2:[200.]})) ....: [array([ 20000.], dtype=float32)]```

Posted by TextMiner