机器学习进阶笔记之一 | TensorFlow安装与入门
引言
TensorFlow是Google基于DistBelief进行研发的第二代人工智能学习系统，被广泛用于语音识别或图像识别等多项机器深度学习领域。其命名来源于本身的运行原理。Tensor(张量)意味着N维数组，Flow(流)意味着基于数据流图的计算，TensorFlow代表着张量从图象的一端流动到另一端计算过程，是将复杂的数据结构传输至人工智能神经网中进行分析和处理的过程。
TensorFlow完全开源，任何人都可以使用。可在小到一部智能手机、大到数千台数据中心服务器的各种设备上运行。
『机器学习进阶笔记』系列是将深入解析TensorFlow系统的技术实践，从零开始，由浅入深，与大家一起走上机器学习的进阶之路。
CUDA与TensorFlow安装
按以往经验，TensorFlow安装一条pip命令就可以解决，前提是有fq工具，没有的话去找找墙内别人分享的地址。而坑多在安装支持gpu，需预先安装英伟达的cuda，这里坑比较多，推荐使用ubuntu deb的安装方式来安装cuda，run.sh的方式总感觉有很多问题，cuda的安装具体可以参考。 注意链接里面的tensorflow版本是以前的，tensorflow 现在官方上的要求是cuda7.5+cudnnV4，请在安装的时候注意下。
Hello World
<ol class="dp-sql"> <li class="alt"><span><span>import tensorflow </span><span class="keyword">as</span><span> tf </span></span></li> <li><span> hello = tf.constant(<span class="string">'Hello, TensorFlow!'</span><span>) </span></span></li> <li class="alt"><span> sess = tf.Session() </span></li> <li><span> print sess.run(hello) </span></li> </ol>
首先，通过tf.constant创建一个常量，然后启动Tensorflow的Session，调用sess的run方法来启动整个graph。
接下来我们做下简单的数学的方法：
<ol class="dp-sql"> <li class="alt"><span><span>import tensorflow </span><span class="keyword">as</span><span> tf </span></span></li> <li><span> a = tf.constant(2) </span></li> <li class="alt"><span> b = tf.constant(3) </span></li> <li><span> <span class="keyword">with</span><span> tf.Session() </span><span class="keyword">as</span><span> sess: </span></span></li> <li class="alt"><span> print <span class="string">"a=2, b=3"</span><span> </span></span></li> <li><span> print <span class="string">"Addition with constants: %i"</span><span> % sess.run(a+b) </span></span></li> <li class="alt"><span> print <span class="string">"Multiplication with constants: %i"</span><span> % sess.run(a*b) </span></span></li> <li><span> # <span class="keyword">output</span><span> </span></span></li> <li class="alt"><span> a=2, b=3 </span></li> <li><span> Addition <span class="keyword">with</span><span> constants: 5 </span></span></li> <li class="alt"><span> Multiplication <span class="keyword">with</span><span> constants: 6 </span></span></li> </ol>
接下来用tensorflow的placeholder来定义变量做类似计算：
placeholder的使用见https://www.tensorflow.org/versions/r0.8/api_docs/python/io_ops.html#placeholder
<ol class="dp-sql">
<li class="alt"><span><span>import tensorflow </span><span class="keyword">as</span><span> tf </span></span></li>
<li><span>a = tf.placeholder(tf.int16) </span></li>
<li class="alt"><span>b = tf.placeholder(tf.int16) </span></li>
<li><span><span class="keyword">add</span><span> = tf.</span><span class="keyword">add</span><span>(a, b) </span></span></li>
<li class="alt"><span>mul = tf.mul(a, b) </span></li>
<li><span><span class="keyword">with</span><span> tf.Session() </span><span class="keyword">as</span><span> sess: </span></span></li>
<li class="alt"><span> # Run every operation <span class="keyword">with</span><span> variable input </span></span></li>
<li><span> print <span class="string">"Addition with variables: %i"</span><span> % sess.run(</span><span class="keyword">add</span><span>, feed_dict={a: 2, b: 3}) </span></span></li>
<li class="alt"><span> print <span class="string">"Multiplication with variables: %i"</span><span> % sess.run(mul, feed_dict={a: 2, b: 3}) </span></span></li>
<li><span># <span class="keyword">output</span><span>: </span></span></li>
<li class="alt"><span>Addition <span class="keyword">with</span><span> variables: 5 </span></span></li>
<li><span>Multiplication <span class="keyword">with</span><span> variables: 6 </span></span></li>
<li class="alt"><span>matrix1 = tf.constant([[3., 3.]]) </span></li>
<li><span>matrix2 = tf.constant([[2.],[2.]]) </span></li>
<li class="alt"><span><span class="keyword">with</span><span> tf.Session() </span><span class="keyword">as</span><span> sess: </span></span></li>
<li><span> result = sess.run(product) </span></li>
<li class="alt"><span> print result </span></li>
</ol>线性回归
以下代码来自GitHub - aymericdamien/TensorFlow-Examples: TensorFlow Tutorial and Examples for beginners,仅作学习用
<ol class="dp-sql">
<li class="alt"><span><span>activation = tf.</span><span class="keyword">add</span><span>(tf.mul(X, W), b) </span></span></li>
<li><span> </span></li>
<li class="alt"><span># Minimize the squared errors </span></li>
<li><span>cost = tf.reduce_sum(tf.pow(activation-Y, 2))/(2*n_samples) #L2 loss </span></li>
<li class="alt"><span>optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) #Gradient descent </span></li>
<li><span> </span></li>
<li class="alt"><span># Initializing the variables </span></li>
<li><span>init = tf.initialize_all_variables() </span></li>
<li class="alt"><span> </span></li>
<li><span># Launch the graph </span></li>
<li class="alt"><span><span class="keyword">with</span><span> tf.Session() </span><span class="keyword">as</span><span> sess: </span></span></li>
<li><span> sess.run(init) </span></li>
<li class="alt"><span> </span></li>
<li><span> # Fit <span class="op">all</span><span> training data </span></span></li>
<li class="alt"><span> <span class="keyword">for</span><span> epoch </span><span class="op">in</span><span> range(training_epochs): </span></span></li>
<li><span> <span class="keyword">for</span><span> (x, y) </span><span class="op">in</span><span> zip(train_X, train_Y): </span></span></li>
<li class="alt"><span> sess.run(optimizer, feed_dict={X: x, Y: y}) </span></li>
<li><span> </span></li>
<li class="alt"><span> #Display logs per epoch step </span></li>
<li><span> if epoch % display_step == 0: </span></li>
<li class="alt"><span> print <span class="string">"Epoch:"</span><span>, </span><span class="string">'%04d'</span><span> % (epoch+1), </span><span class="string">"cost="</span><span>, \ </span></span></li>
<li><span> <span class="string">"{:.9f}"</span><span>.format(sess.run(cost, feed_dict={X: train_X, Y:train_Y})), \ </span></span></li>
<li class="alt"><span> <span class="string">"W="</span><span>, sess.run(W), </span><span class="string">"b="</span><span>, sess.run(b) </span></span></li>
<li><span> </span></li>
<li class="alt"><span> print <span class="string">"Optimization Finished!"</span><span> </span></span></li>
<li><span> print <span class="string">"cost="</span><span>, sess.run(cost, feed_dict={X: train_X, Y: train_Y}), \ </span></span></li>
<li class="alt"><span> <span class="string">"W="</span><span>, sess.run(W), </span><span class="string">"b="</span><span>, sess.run(b) </span></span></li>
<li><span> </span></li>
<li class="alt"><span> #Graphic display </span></li>
<li><span> plt.plot(train_X, train_Y, <span class="string">'ro'</span><span>, label=</span><span class="string">'Original data'</span><span>) </span></span></li>
<li class="alt"><span> plt.plot(train_X, sess.run(W) * train_X + sess.run(b), label=<span class="string">'Fitted line'</span><span>) </span></span></li>
<li><span> plt.legend() </span></li>
<li class="alt"><span> plt.show() </span></li>
</ol>逻辑回归
<ol class="dp-sql">
<li class="alt"><span><span>import tensorflow </span><span class="keyword">as</span><span> tf </span></span></li>
<li><span> # Import MINST data </span></li>
<li class="alt"><span> <span class="keyword">from</span><span> tensorflow.examples.tutorials.mnist import input_data </span></span></li>
<li><span> mnist = input_data.read_data_sets(<span class="string">"/tmp/data/"</span><span>, one_hot=</span><span class="keyword">True</span><span>) </span></span></li>
<li class="alt"><span> </span></li>
<li><span> # Parameters </span></li>
<li class="alt"><span> learning_rate = 0.01 </span></li>
<li><span> training_epochs = 25 </span></li>
<li class="alt"><span> batch_size = 100 </span></li>
<li><span> display_step = 1 </span></li>
<li class="alt"><span> </span></li>
<li><span> # tf Graph Input </span></li>
<li class="alt"><span> x = tf.placeholder(tf.float32, [None, 784]) # mnist data image <span class="keyword">of</span><span> shape 28*28=784 </span></span></li>
<li><span> y = tf.placeholder(tf.float32, [None, 10]) # 0-9 digits recognition => 10 classes </span></li>
<li class="alt"><span> </span></li>
<li><span> # <span class="keyword">Set</span><span> model weights </span></span></li>
<li class="alt"><span> W = tf.Variable(tf.zeros([784, 10])) </span></li>
<li><span> b = tf.Variable(tf.zeros([10])) </span></li>
<li class="alt"><span> </span></li>
<li><span> # Construct model </span></li>
<li class="alt"><span> pred = tf.nn.softmax(tf.matmul(x, W) + b) # Softmax </span></li>
<li><span> </span></li>
<li class="alt"><span> # Minimize error using <span class="op">cross</span><span> entropy </span></span></li>
<li><span> cost = tf.reduce_mean(-tf.reduce_sum(y*tf.log(pred), reduction_indices=1)) </span></li>
<li class="alt"><span> # Gradient Descent </span></li>
<li><span> optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) </span></li>
<li class="alt"><span> </span></li>
<li><span> # Initializing the variables </span></li>
<li class="alt"><span> init = tf.initialize_all_variables() </span></li>
<li><span> </span></li>
<li class="alt"><span> # Launch the graph </span></li>
<li><span> <span class="keyword">with</span><span> tf.Session() </span><span class="keyword">as</span><span> sess: </span></span></li>
<li class="alt"><span> sess.run(init) </span></li>
<li><span> </span></li>
<li class="alt"><span> # Training cycle </span></li>
<li><span> <span class="keyword">for</span><span> epoch </span><span class="op">in</span><span> range(training_epochs): </span></span></li>
<li class="alt"><span> avg_cost = 0. </span></li>
<li><span> total_batch = <span class="keyword">int</span><span>(mnist.train.num_examples/batch_size) </span></span></li>
<li class="alt"><span> # Loop over <span class="op">all</span><span> batches </span></span></li>
<li><span> <span class="keyword">for</span><span> i </span><span class="op">in</span><span> range(total_batch): </span></span></li>
<li class="alt"><span> batch_xs, batch_ys = mnist.train.next_batch(batch_size) </span></li>
<li><span> # Run optimization op (backprop) <span class="op">and</span><span> cost op (</span><span class="keyword">to</span><span> get loss value) </span></span></li>
<li class="alt"><span> _, c = sess.run([optimizer, cost], feed_dict={x: batch_xs, </span></li>
<li><span> y: batch_ys}) </span></li>
<li class="alt"><span> # Compute average loss </span></li>
<li><span> avg_cost += c / total_batch </span></li>
<li class="alt"><span> # Display logs per epoch step </span></li>
<li><span> if (epoch+1) % display_step == 0: </span></li>
<li class="alt"><span> print <span class="string">"Epoch:"</span><span>, </span><span class="string">'%04d'</span><span> % (epoch+1), </span><span class="string">"cost="</span><span>, </span><span class="string">"{:.9f}"</span><span>.format(avg_cost) </span></span></li>
<li><span> </span></li>
<li class="alt"><span> print <span class="string">"Optimization Finished!"</span><span> </span></span></li>
<li><span> </span></li>
<li class="alt"><span> # Test model </span></li>
<li><span> correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1)) </span></li>
<li class="alt"><span> # Calculate accuracy </span></li>
<li><span> accuracy = tf.reduce_mean(tf.<span class="func">cast</span><span>(correct_prediction, tf.float32)) </span></span></li>
<li class="alt"><span> print <span class="string">"Accuracy:"</span><span>, accuracy.eval({x: mnist.test.images, y: mnist.test.labels}) </span></span></li>
<li><span> </span></li>
<li class="alt"><span> # result : </span></li>
<li><span> Epoch: 0001 cost= 29.860467369 </span></li>
<li class="alt"><span> Epoch: 0002 cost= 22.001451784 </span></li>
<li><span> Epoch: 0003 cost= 21.019925554 </span></li>
<li class="alt"><span> Epoch: 0004 cost= 20.561320320 </span></li>
<li><span> Epoch: 0005 cost= 20.109135756 </span></li>
<li class="alt"><span> Epoch: 0006 cost= 19.927862290 </span></li>
<li><span> Epoch: 0007 cost= 19.548687116 </span></li>
<li class="alt"><span> Epoch: 0008 cost= 19.429119071 </span></li>
<li><span> Epoch: 0009 cost= 19.397068211 </span></li>
<li class="alt"><span> Epoch: 0010 cost= 19.180813479 </span></li>
<li><span> Epoch: 0011 cost= 19.026808132 </span></li>
<li class="alt"><span> Epoch: 0012 cost= 19.057875510 </span></li>
<li><span> Epoch: 0013 cost= 19.009575057 </span></li>
<li class="alt"><span> Epoch: 0014 cost= 18.873240641 </span></li>
<li><span> Epoch: 0015 cost= 18.718575359 </span></li>
<li class="alt"><span> Epoch: 0016 cost= 18.718761925 </span></li>
<li><span> Epoch: 0017 cost= 18.673640560 </span></li>
<li class="alt"><span> Epoch: 0018 cost= 18.562128253 </span></li>
<li><span> Epoch: 0019 cost= 18.458205289 </span></li>
<li class="alt"><span> Epoch: 0020 cost= 18.538211225 </span></li>
<li><span> Epoch: 0021 cost= 18.443384213 </span></li>
<li class="alt"><span> Epoch: 0022 cost= 18.428727668 </span></li>
<li><span> Epoch: 0023 cost= 18.304270616 </span></li>
<li class="alt"><span> Epoch: 0024 cost= 18.323529782 </span></li>
<li><span> Epoch: 0025 cost= 18.247192113 </span></li>
<li class="alt"><span> Optimization Finished! </span></li>
<li><span> (10000, 784) </span></li>
<li class="alt"><span> Accuracy 0.9206 </span></li>
</ol>这里有个小插曲，ipython notebook在一个notebook打开时，一直在占用GPU资源，可能是之前有一个notebook一直打开着，然后占用着GPU资源，然后在计算Accuracy的”InternalError: Dst tensor is not initialized.” 然后找了github上面也有这个问题InternalError: Dst tensor is not initialized.,可以肯定是GPU的memory相关的问题，所以就尝试加上tf.device(‘/cpu:0’)，将Accuracy这步拉到cpu上计算，但是又出现OOM的问题，最后nvidia-smi时，发现有一个python脚本一直占用3g多的显存，把它kill之后恢复了，之前还比较吐槽怎么可能10000*784个float就把显存撑爆呢，原来是自己的问题。
相关推荐
Micusd 2020-11-19
人工智能 2020-11-19
81510295 2020-11-17
jaybeat 2020-11-17
flyfor0 2020-11-16
lgblove 2020-11-16
Pokemogo 2020-11-16
Pokemogo 2020-11-16
clong 2020-11-13
lizhengjava 2020-11-13
ohbxiaoxin 2020-11-13
Icevivian 2020-11-13
EchoYY 2020-11-12
CSDN人工智能头条 2020-11-11
mogigo00 2020-11-11
jaybeat 2020-11-10
白飞飞Alan 2020-11-11
lemonade 2020-11-10
机器学习之家 2020-11-10