Difference between revisions of "Tensorflow Guide"

From Storrs HPC Wiki
Jump to: navigation, search
(Example: Logging Device placement (GPU Version Guide))
(Example: Logging Device placement (GPU Version Guide))
Line 72: Line 72:
  
 
==Example: Logging Device placement (GPU Version Guide)==
 
==Example: Logging Device placement (GPU Version Guide)==
  hpc-xin@cn02:~$ ssh gpu01
+
  '''hpc-xin@cn02:~$ ssh gpu01'''
  hpc-xin@gpu01:~$ module purge
+
  '''hpc-xin@gpu01:~$ module purge'''
  hpc-xin@gpu01:~$ module load cuda/8.0.61 cudnn/6.0 python/2.7.6-gcc-unicode
+
  '''hpc-xin@gpu01:~$ module load cuda/8.0.61 cudnn/6.0 python/2.7.6-gcc-unicode'''
  hpc-xin@gpu01:~$ python
+
  '''hpc-xin@gpu01:~$ python'''
 
  Python 2.7.6 (default, Apr 20 2016, 16:39:52)  
 
  Python 2.7.6 (default, Apr 20 2016, 16:39:52)  
 
  [GCC 4.8.2] on linux2
 
  [GCC 4.8.2] on linux2
 
  Type "help", "copyright", "credits" or "license" for more information.
 
  Type "help", "copyright", "credits" or "license" for more information.
  >>> import tensorflow as tf
+
  '''>>> import tensorflow as tf'''
 
  I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally
 
  I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally
 
  I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
 
  I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
Line 85: Line 85:
 
  I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so.6 locally
 
  I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so.6 locally
 
  I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally
 
  I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally
  >>> a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
+
  '''>>> a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')'''
  >>> b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
+
  '''>>> b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')'''
  >>> c = tf.matmul(a, b)
+
  '''>>> c = tf.matmul(a, b)'''
  >>> sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
+
  '''>>> sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))'''
 
  I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K40m, pci bus id: 0000:03:00.0)
 
  I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K40m, pci bus id: 0000:03:00.0)
 
  I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla K40m, pci bus id: 0000:82:00.0)
 
  I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla K40m, pci bus id: 0000:82:00.0)
Line 97: Line 97:
 
  /job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: Tesla K40m, pci bus id: 0000:03:00.0
 
  /job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: Tesla K40m, pci bus id: 0000:03:00.0
 
  /job:localhost/replica:0/task:0/gpu:1 -> device: 1, name: Tesla K40m, pci bus id: 0000:82:00.0
 
  /job:localhost/replica:0/task:0/gpu:1 -> device: 1, name: Tesla K40m, pci bus id: 0000:82:00.0
  >>> print(sess.run(c))
+
  '''>>> print(sess.run(c))'''
 
  MatMul: (MatMul): /job:localhost/replica:0/task:0/gpu:0
 
  MatMul: (MatMul): /job:localhost/replica:0/task:0/gpu:0
 
  I tensorflow/core/common_runtime/simple_placer.cc:827] MatMul: (MatMul)/job:localhost/replica:0/task:0/gpu:0
 
  I tensorflow/core/common_runtime/simple_placer.cc:827] MatMul: (MatMul)/job:localhost/replica:0/task:0/gpu:0

Revision as of 21:19, 28 April 2017

Tensorflow
Author Google Brain Team
Website https://www.tensorflow.org/
Source HistoryVersions
Category Machine Learning, Deep Neural Networks
Help documentation


Current Supported Versions in HPC

CPU Support Version Corresponding Python Module Dependence Released Date
v0.10.0rc0 python/2.7.6 - Jul 29, 2016
- - - -
GPU & CPU Support Version Corresponding Python Module Dependence Released Date
v0.12.1 python/2.7.6-gcc-unicode cuda, cudnn Dec 25, 2016
- - - -

Import Modules

TensorFlow v0.10.0rc0 (CPU Support Only)

module load python/2.7.6

TensorFlow v0.12.1 (Both GPU and CPU Support)

 module load cuda/8.0.61 cudnn/6.0 python/2.7.6-gcc-unicode

Application Usage

Check Version Number

 $ python
 Python 2.7.6 (default, Apr 20 2016, 16:39:52) 
 [GCC 4.8.2] on linux2
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import tensorflow as tf
 >>> print tf.__version__
 0.12.1
 >>>

Example: How to Build Computational Graph

$ python
Python 2.7.6 (default, Apr 20 2016, 16:39:52) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> node1 = tf.constant(3.0, tf.float32)
>>> node2 = tf.constant(4.0)
>>> node3 = tf.add(node1, node2)
>>> sess = tf.Session()
>>> sess.run(node3)
7.0
>>> a = tf.placeholder(tf.float32)
>>> b = tf.placeholder(tf.float32)
>>> adder_node = a + b
>>> print(sess.run(adder_node, {a: 3, b:4.5}))
7.5
>>> print(sess.run(adder_node, {a: [1,3], b: [2, 4]}))
[ 3.  7.]
>>> add_and_triple = adder_node * 3
>>> print(sess.run(add_and_triple, {a: 3, b:4.5}))
22.5
>>>

Example: Logging Device placement (GPU Version Guide)

hpc-xin@cn02:~$ ssh gpu01
hpc-xin@gpu01:~$ module purge
hpc-xin@gpu01:~$ module load cuda/8.0.61 cudnn/6.0 python/2.7.6-gcc-unicode
hpc-xin@gpu01:~$ python
Python 2.7.6 (default, Apr 20 2016, 16:39:52) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so.6 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally
>>> a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
>>> b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
>>> c = tf.matmul(a, b)
>>> sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K40m, pci bus id: 0000:03:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:1) -> (device: 1, name: Tesla K40m, pci bus id: 0000:82:00.0)
Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: Tesla K40m, pci bus id: 0000:03:00.0
/job:localhost/replica:0/task:0/gpu:1 -> device: 1, name: Tesla K40m, pci bus id: 0000:82:00.0
I tensorflow/core/common_runtime/direct_session.cc:255] Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: Tesla K40m, pci bus id: 0000:03:00.0
/job:localhost/replica:0/task:0/gpu:1 -> device: 1, name: Tesla K40m, pci bus id: 0000:82:00.0
>>> print(sess.run(c))
MatMul: (MatMul): /job:localhost/replica:0/task:0/gpu:0
I tensorflow/core/common_runtime/simple_placer.cc:827] MatMul: (MatMul)/job:localhost/replica:0/task:0/gpu:0
b: (Const): /job:localhost/replica:0/task:0/gpu:0
I tensorflow/core/common_runtime/simple_placer.cc:827] b: (Const)/job:localhost/replica:0/task:0/gpu:0
a: (Const): /job:localhost/replica:0/task:0/gpu:0
I tensorflow/core/common_runtime/simple_placer.cc:827] a: (Const)/job:localhost/replica:0/task:0/gpu:0
Const: (Const): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:827] Const: (Const)/job:localhost/replica:0/task:0/cpu:0
[[ 22.  28.]
 [ 49.  64.]]

More Examples

https://www.tensorflow.org/tutorials/mandelbrot