↓
 
Home→Deep Learning→Dive Into TensorFlow, Part II: Basic Concepts

Dive Into TensorFlow, Part II: Basic Concepts

Posted on by June 19, 2016
Deep Learning Specialization on Coursera

This is the second article in the series “Dive Into TensorFlow“, here is an index of all the articles in the series that have been published to date:

Part I: Getting Started with TensorFlow
Part II: Basic Concepts (this article)

What’s Tensor?

Tensor is the core concept for TensorFlow:

TensorFlow programs use a tensor data structure to represent all data — only tensors are passed between operations in the computation graph. You can think of a TensorFlow tensor as an n-dimensional array or list.

For example, a scaler is a tensor, a vector is a tensor and a matrix is a tensor. A tensor has a rank, a shape and a static type, so a tensor can be represented as a multidimensional array of numbers.

1) Rank

Tensor rank is the number of dimensions of the tensor, for example, a rank one tensor is a vector, a rank two tensor is a matrix.

1
2
3
4
5
6
7
8
9
10
11
12
13

In [1]: import tensorflow as tf
 
# scalar rank is 0
In [2]: scalar1 = tf.constant(10000)
 
# vector rank is 1
In [3]: vector1 = tf.constant([1, 2, 3, 4, 5])
 
# matrix rank is 2
In [4]: matrix1 = tf.constant([[1, 2, 3], [4, 5, 6]])
 
# 3-tensor rank is 3
In [5]: cube1 = tf.constant([[[1], [2], [3]], [[4], [5], [6]], [[7], [8], [9]]])

2) Shape

The TensorFlow documentation uses three notational conventions to describe tensor dimensionality: rank, shape, and dimension number. The following example shows how these relate to one another:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# scalar: rank 0 shape []
In [6]: scalar1.get_shape()
Out[6]: TensorShape([])
 
# vector: rank 1 shape [D0]
In [7]: vector1.get_shape()
Out[7]: TensorShape([Dimension(5)])
 
# matrix: rank 2 shape [D0, D1]
In [8]: matrix1.get_shape()
Out[8]: TensorShape([Dimension(2), Dimension(3)])
 
# 3-tensor: rank 3 shape [D0, D1, D2]
In [9]: cube1.get_shape()
Out[9]: TensorShape([Dimension(3), Dimension(3), Dimension(1)])


Data type

In addition to rank and shape, Tensors have a data type. Following is the list of the data type:

Data type Python type Description
DT_FLOAT tf.float32 32 bits floating point.
DT_DOUBLE tf.float64 64 bits floating point.
DT_INT8 tf.int8 8 bits signed integer.
DT_INT16 tf.int16 16 bits signed integer.
DT_INT32 tf.int32 32 bits signed integer.
DT_INT64 tf.int64 64 bits signed integer.
DT_UINT8 tf.uint8 8 bits unsigned integer.
DT_STRING tf.string Variable length byte arrays. Each element of a Tensor is a byte array.
DT_BOOL tf.bool Boolean.
DT_COMPLEX64 tf.complex64 Complex number made of two 32 bits floating points: real and imaginary parts.
DT_COMPLEX128 tf.complex128 Complex number made of two 64 bits floating points: real and imaginary parts.
DT_QINT8 tf.qint8 8 bits signed integer used in quantized Ops.
DT_QINT32 tf.qint32 32 bits signed integer used in quantized Ops.
DT_QUINT8 tf.quint8 8 bits unsigned integer used in quantized Ops.

 

Variables

In TensorFlow, variables maintain state across executions of the graph. For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

In [1]: import tensorflow as tf
 
# Create a Variable, that will be initialized to the scalar value 100
In [2]: var = tf.Variable(100, name="variable_counter")
 
# Create an Op to multiply ten to `var`.
In [3]: ten = tf.constant(10)
 
In [4]: new_var = tf.mul(var, ten)
 
In [5]: update = tf.assign(var, new_var)
 
 
# Variables must be initialized by running an `init` Op after having
# launched the graph.  We first have to add the `init` Op to the graph.
In [6]: init_op = tf.initialize_all_variables()
 
# Launch the graph and run the ops.
In [7]: with tf.Session() as sess:
    sess.run(init_op)
    print(sess.run(var))
    for _ in range(5):
        sess.run(update)
        print(sess.run(var))
   ...:     
 
# output    
100
1000
10000
100000
1000000
10000000

Variables are in-memory buffers containing tensors, when you train a model with TensorFlow, variables can be used to hold and update parameters. Variables must be explicitly initialized and cab be saved to disk during and after training.

1
2
3
4
5
6
7
8
9

 
# Create a variable with a random value.
In [10]: weights = tf.Variable(tf.random_normal([4096, 500], stddev=0.25), name="weights")
 
# Create another variable with the same value as 'weights'.
In [11]: weights2 = tf.Variable(weights.initialized_value(), name="weights2")
 
# Create another variable with twice the value of 'weights'
In [12]: weights_twice = tf.Variable(weights.initialized_value() * 2.0, name="weights_twice")

Fetches

To fetch the outputs of operations, execute the graph with a run() call on the Session object and pass in the tensors to retrieve. Except fetching the single tensor node, you can also fetch multiple tensors:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

In [16]: constant1 = tf.constant([1.0])
 
In [17]: constant2 = tf.constant([2.0])
 
In [18]: constant3 = tf.constant([3.0])
 
In [19]: intermed = tf.add(constant2, constant3)  
 
In [20]: mul = tf.mul(constant1, intermed)
 
In [21]: with tf.Session() as sess:
   ....:     result = sess.run([mul, intermed])
   ....:     print(result)
   ....:     
[array([ 5.], dtype=float32), array([ 5.], dtype=float32)]

Feeds

TensorFlow provides a feed mechanism for patching a tensor directly into any operation in the graph.

A feed temporarily replaces the output of an operation with a tensor value. You supply feed data as an argument to a run() call. The feed is only used for the run call to which it is passed. The most common use case involves designating specific operations to be “feed” operations by using tf.placeholder() to create them:

1
2
3
4
5
6
7
8
9
10

In [23]: pla1 = tf.placeholder(tf.float32)
 
In [24]: pla2 = tf.placeholder(tf.float32)
 
In [25]: result = tf.mul(pla1, pla2)
 
In [26]: with tf.Session() as sess:
    print(sess.run([result], feed_dict={pla1:[100.], pla2:[200.]}))
   ....:     
[array([ 20000.], dtype=float32)]

Posted by TextMiner

Related posts:

  1. Dive Into TensorFlow, Part I: Getting Started with TensorFlow
  2. Deep Learning for Text Mining from Scratch
  3. Deep Learning for NLP resources from Github
Deep Learning Specialization on Coursera

Comments

Dive Into TensorFlow, Part II: Basic Concepts — No Comments

Leave a Reply

Your email address will not be published.

↑