基于Nvidia GPU和Docker容器的深度学习环境搭建
基于Nvidia GPU和Docker容器的深度学习环境搭建
GPU云主机：
操作系统：Ubuntu 16.04 64位
GPU： 1 x Nvidia Tesla P40
1. 安装CUDA Driver
1.1 Pre-installation Actions
安装gcc、g++、make：
# sudo apt-get install gcc g++ make # gcc --version gcc (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609 Copyright (C) 2015 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
如果没有，需安装linux-headers：
# sudo apt-get install linux-headers-$(uname -r)
1.2 安装NVIDIA driver
CUDA安装有两种方式：
1.Package安装
2.Runfile安装
本文选择runfile安装方式。
首先禁用Nouveau：
# lsmod | grep nouveau nouveau 1495040 0 mxm_wmi16384 1 nouveau wmi20480 2 mxm_wmi,nouveau video 40960 1 nouveau i2c_algo_bit 16384 1 nouveau ttm94208 1 nouveau drm_kms_helper155648 1 nouveau drm 364544 3 ttm,drm_kms_helper,nouveau # vi /etc/modprobe.d/blacklist-nouveau.conf blacklist nouveau options nouveau modeset=0 # sudo update-initramfs -u update-initramfs: Generating /boot/initrd.img-4.4.0-62-generic W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.
Reboot云主机：
# reboot
重启后check下Nouveau drivers没有被load：
# lsmod | grep nouveau #
登录：http://developer.nvidia.com/c... 下载相应的runfile：
# wget https://developer.nvidia.com/compute/cuda/10.0/Prod/local_installers/cuda_10.0.130_410.48_linux
开始安装CUDA Driver:
# chmod +x cuda_10.0.130_410.48_linux # sudo sh ./cuda_10.0.130_410.48_linux Logging to /tmp/cuda_install_1699.log Using more to view the EULA. Do you accept the previously read EULA? accept/decline/quit: accept Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 410.48? (y)es/(n)o/(q)uit: y Do you want to install the OpenGL libraries? (y)es/(n)o/(q)uit [ default is yes ]: y Do you want to run nvidia-xconfig? This will update the system X configuration file so that the NVIDIA X driver is used. The pre-existing X configuration file will be backed up. This option should not be used on systems that require a custom X configuration, such as systems with multiple GPU vendors. (y)es/(n)o/(q)uit [ default is no ]: Install the CUDA 10.0 Toolkit? (y)es/(n)o/(q)uit: y Enter Toolkit Location [ default is /usr/local/cuda-10.0 ]: Do you want to install a symbolic link at /usr/local/cuda? (y)es/(n)o/(q)uit: y Install the CUDA 10.0 Samples? (y)es/(n)o/(q)uit: y Enter CUDA Samples Location [ default is /root ]: Installing the NVIDIA display driver... Installing the CUDA Toolkit in /usr/local/cuda-10.0 ... Missing recommended library: libGLU.so Missing recommended library: libX11.so Missing recommended library: libXi.so Missing recommended library: libXmu.so Installing the CUDA Samples in /root ... Copying samples to /root/NVIDIA_CUDA-10.0_Samples now... Finished copying samples. =========== = Summary = =========== Driver: Installed Toolkit: Installed in /usr/local/cuda-10.0 Samples: Installed in /root, but missing recommended libraries Please make sure that - PATH includes /usr/local/cuda-10.0/bin - LD_LIBRARY_PATH includes /usr/local/cuda-10.0/lib64, or, add /usr/local/cuda-10.0/lib64 to /etc/ld.so.conf and run ldconfig as root To uninstall the CUDA Toolkit, run the uninstall script in /usr/local/cuda-10.0/bin To uninstall the NVIDIA Driver, run nvidia-uninstall Please see CUDA_Installation_Guide_Linux.pdf in /usr/local/cuda-10.0/doc/pdf for detailed information on setting up CUDA. Logfile is /tmp/cuda_install_1699.log
安装成功！
Reboot云主机：
# reboot
设备验证：
# ls /dev/nvidia*
ls: cannot access '/dev/nvidia*': No such file or directory
# vi nvidia-probe.sh
#!/bin/bash
### BEGIN INIT INFO
# Provides: jd.com
# Required-Start: $local_fs $network
# Required-Stop: $local_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: nvidia service
# Description: nvidia service daemon
### END INIT INFO
/sbin/modprobe nvidia
if [ "$?" -eq 0 ]; then
# Count the number of NVIDIA controllers found.
NVDEVS=`lspci | grep -i NVIDIA`
N3D=`echo "$NVDEVS" | grep "3D controller" | wc -l`
NVGA=`echo "$NVDEVS" | grep "VGA compatible controller" | wc -l`
N=`expr $N3D + $NVGA - 1`
for i in `seq 0 $N`; do
mknod -m 666 /dev/nvidia$i c 195 $i
done
mknod -m 666 /dev/nvidiactl c 195 255
else
exit 1
fi
/sbin/modprobe nvidia-uvm
if [ "$?" -eq 0 ]; then
# Find out the major device number used by the nvidia-uvm driver
D=`grep nvidia-uvm /proc/devices | awk '{print $1}'`
mknod -m 666 /dev/nvidia-uvm c $D 0
else
exit 1
fi
# chmod +x nvidia-probe.sh
# ./nvidia-probe.sh
# ls /dev/nvidia*
/dev/nvidia0 /dev/nvidiactl /dev/nvidia-uvm/dev下成功发现设备!
配置开机自启动：
# cp nvidia-probe.sh /etc/init.d/ # sudo update-rc.d nvidia-probe.sh defaults 95
1.3 Post-installation Actions
配置环境变量：
# vi /etc/profile
......
export PATH=/usr/local/cuda-10.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}开机启动Persistence Daemon：
# vi /etc/rc.local ...... /usr/bin/nvidia-persistenced --verbose exit 0
1.4 CUDA driver验证
查看Driver Version：
# cat /proc/driver/nvidia/version NVRM version: NVIDIA UNIX x86_64 Kernel Module 410.48 Thu Sep 6 06:36:33 CDT 2018 GCC version: gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)
使用deviceQuery示例验证：
# cd ~/NVIDIA_CUDA-10.0_Samples/1_Utilities/deviceQuery/ # make "/usr/local/cuda-10.0"/bin/nvcc -ccbin g++ -I../../common/inc -m64-gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_37,code=sm_37 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_75,code=compute_75 -o deviceQuery.o -c deviceQuery.cpp "/usr/local/cuda-10.0"/bin/nvcc -ccbin g++ -m64 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_37,code=sm_37 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_75,code=compute_75 -o deviceQuery deviceQuery.o mkdir -p ../../bin/x86_64/linux/release cp deviceQuery ../../bin/x86_64/linux/release # cd ../../bin/x86_64/linux/release/ # ls deviceQuery # ./deviceQuery ./deviceQuery Starting... CUDA Device Query (Runtime API) version (CUDART static linking) Detected 1 CUDA Capable device(s) Device 0: "Tesla P40" CUDA Driver Version / Runtime Version 10.0 / 10.0 CUDA Capability Major/Minor version number:6.1 Total amount of global memory: 22919 MBytes (24032378880 bytes) (30) Multiprocessors, (128) CUDA Cores/MP: 3840 CUDA Cores GPU Max Clock rate:1531 MHz (1.53 GHz) Memory Clock rate: 3615 Mhz Memory Bus Width: 384-bit L2 Cache Size: 3145728 bytes Maximum Texture Dimension Size (x,y,z) 1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384) Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers Total amount of constant memory: 65536 bytes Total amount of shared memory per block: 49152 bytes Total number of registers available per block: 65536 Warp size: 32 Maximum number of threads per multiprocessor: 2048 Maximum number of threads per block: 1024 Max dimension size of a thread block (x,y,z): (1024, 1024, 64) Max dimension size of a grid size(x,y,z): (2147483647, 65535, 65535) Maximum memory pitch: 2147483647 bytes Texture alignment: 512 bytes Concurrent copy and kernel execution: Yes with 2 copy engine(s) Run time limit on kernels: No Integrated GPU sharing Host Memory:No Support host page-locked memory mapping: Yes Alignment requirement for Surfaces:Yes Device has ECC support:Enabled Device supports Unified Addressing (UVA): Yes Device supports Compute Preemption:Yes Supports Cooperative Kernel Launch:Yes Supports MultiDevice Co-op Kernel Launch: Yes Device PCI Domain ID / Bus ID / location ID: 0 / 0 / 7 Compute Mode: < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) > deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 10.0, CUDA Runtime Version = 10.0, NumDevs = 1 Result = PASS
参考：
https://github.com/NVIDIA/nvi...https://docs.nvidia.com/cuda/...
2. 安装Nvidia-docker
2.1 安装Docker
安装docker-ce：
#sudo apt-get remove docker docker-engine docker.io # sudo apt-get install \ apt-transport-https \ ca-certificates \ curl \ software-properties-common # curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - # sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) \ stable" # sudo apt-get update # sudo apt-get install docker-ce # docker version Client: Version: 18.06.1-ce API version: 1.38 Go version:go1.10.3 Git commit:e68fc7a Built: Tue Aug 21 17:24:56 2018 OS/Arch: linux/amd64 Experimental: false Server: Engine: Version: 18.06.1-ce API version: 1.38 (minimum version 1.12) Go version: go1.10.3 Git commit: e68fc7a Built:Tue Aug 21 17:23:21 2018 OS/Arch: linux/amd64 Experimental: false
2.2 安装nvidia-docker
安装nvidia-docker：
# Add the package repositories curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \ sudo apt-key add - distribution=$(. /etc/os-release;echo $ID$VERSION_ID) curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \ sudo tee /etc/apt/sources.list.d/nvidia-docker.list sudo apt-get update # Install nvidia-docker2 and reload the Docker daemon configuration sudo apt-get install -y nvidia-docker2 sudo pkill -SIGHUP dockerd
验证nvidia-docker：
# docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi Thu Oct 25 09:03:27 2018 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 410.48 Driver Version: 410.48| |-------------------------------+----------------------+----------------------+ | GPU NamePersistence-M| Bus-IdDisp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 Tesla P40 On | 00000000:00:07.0 Off |0 | | N/A 20CP8 9W / 250W | 0MiB / 22919MiB | 1% Default | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+
2.3 配置Docker默认runtime
cat /etc/docker/daemon.json
{
"default-runtime": "nvidia",
"runtimes": {
"nvidia": {
"path": "nvidia-container-runtime",
"runtimeArgs": []
}
}
}重启服务：
# systemctl restart docker # systemctl status docker
2.4 运行TensorFlow卷积神经Model
Docker运行：
# docker run --rm --name tensorflow -ti tensorflow/tensorflow:r0.9-devel-gpu root@bd0fb3758da2:~# python --version Python 2.7.6 root@bd0fb3758da2:~# python -m tensorflow.models.image.mnist.convolutional
参考：
https://docs.docker.com/insta...相关推荐
lxyd000 2020-10-30
AI启蒙研究院 2020-10-16
Maryhuan 2020-09-20
lanzhichen 2020-06-21
87354452 2020-06-10
lybbb 2020-06-08
veppay 2020-04-08
yangliuhbhd 2020-01-28
蜡笔小鑫爱看雪 2020-01-19
DEPHI 2013-04-14
liujainq 2019-12-30
杨友山 2019-12-10
人亦有言进退维谷 2019-12-04
糊一笑 2019-11-25
macair 2019-11-13
kevinli 2019-10-30
csdnyasin 2019-10-30
一一空 2019-10-24