GPU深度学习开发

1、GPU深度学习开发GPU SDK FOR DEEP LEARNING2AgendaWhy Deep Learning Boost Today?Nvidia SDK for Deep Learning?CUDA 8.0cuDNN TensorRT (GIE) NCCLWHY DEEP LEARNING BOOST TODAY?“ Googles AI engine also reflects how the world of computer hardware is changing. (It) depends on machines equipped with GPUs And it depe

2、nds on these chips more than the larger tech universe realizes.”GPUDNNBIG DATA13NVIDIA SDK for Deep Learning14CUDA 8.0CUDA 8 WHATS NEWStacked Memory NVLINKFP16 mathP100 SupportUnified MemoryLarger DatasetsDemand Paging New Tuning APIsStandard C/C+ Allocators CPU/GPU Data Coherence &AtomicsLibrariesN

3、ew nvGRAPH librarycuBLAS improvements for Deep LearningDeveloper ToolsCritical Path Analysis2x Faster Compile Time OpenACC ProfilingDebug CUDA Apps on Display GPU16CUDA 8.0: Unified MemoryUnified MemoryDramatically lower developer effortCUDA 8: Pascal Unified MemoryLarger datasets, simpler programmi

4、ng, higher performanceCUDA 8 Unified Memory-ExampleAllocating 4xmore than P100 physical memoryCUDA 8 Unified Memory-ExampleAccessing data simultaneously by CPU and GPU codes21CUDA 8.0: Compiler2X Faster Compiler Times on CUDANVCC Speedups on CUDA 823cuDNNcuDNN: Design GoalsNVIDIA cuDNNAccelerating D

5、eep Learning/cudnnHigh performance building blocks for deep learning frameworksDrop-in acceleration for widely used deep learning frameworks such as Caffe, MxNet, CNTK, Tensorflow, Theano, Torch and othersAccelerates industry vetted deep learning algorithms, such as convolutions, LSTM, fully connect

6、ed, and pooling layersFast deep learning training performance tuned for NVIDIA GPUsDeep Learning Training Performance Caffe AlexNetSpeed-up of Images/Sec vs K40 in 2013K40K80 +cuDNN1M40 +cuDNN4P100 +cuDNN580 x70 x60 x 50 x 40 x 30 x 20 x 10 x 0 x“ NVIDIA has improved the speed of cuDNN with each rel

7、ease while extending the interface to more operations and devices at the same time.” Evan Shelhamer, Lead Caffe Developer, UC BerkeleyAlexNet training throughput on CPU: 1x E5-2680v3 12 Core 2.5GHz. 128GB System Memory, Ubuntu 14.04M40 bar: 8x M40 GPUs in a node, P100: 8x P100 NVLink-enabledNEW IN c

8、uDNNAccelerating Deep Learning/cudnncuDNN v5.12.7x faster training of networks with 3x3 convolutions such as VGGcuDNN v56x speedup on LSTM recurrent neural network in TorchUp to 44% faster training on a single NVIDIA Pascal GPUImproved performance and reduced memory usage with FP16 routines on Pasca

9、l GPUsSpeed-up of training vs. cuDNN v40 x1x2x3x4xUp to 2.7x Faster TrainingcuDNN v4 + K40cuDNN v5.1 + M40Alexnet OWTGooglenetVGGcuDNN 4 + K40 vs. cuDNN 5.1 + M40 on Torch and Intel Xeon Haswell single-socket 16-core E5-2698 v32.3Ghz 3.6GHz TurboTypical layers in CNNConvolution34TensorRT (GIE)A COMP

10、LETE COMPUTE PLATFORMMANAGETRAINDEPLOYDIGITSDATA CENTERAUTOMOTIVETRAINTESTMANAGE / AUGMENTEMBEDDEDTENSORRT: GPUINFERENCE ENGINEHigh-performance framework makes it easy to develop GPU-accelerated inferenceProduction deployment solution for deep learning inferenceOptimized inference for a given traine

11、d neural network and target GPUSolutions for Hyperscale, ADAS, Embedded Supports deployment of 32-bit or 16-bit inferenceMaximum Performance for Deep Learning Inference/gpu-inference-engineGPU Inference Engine for HyperscaleImageClassificationObjectDetectionImageSegmentation-TENSORRT: GPU INFERENCE

12、ENGINETENSORRT: GPU INFERENCE ENGINEHigh-performance framework makes it easy to develop GPU-accelerated inferenceProduction deployment solution for deep learning inferenceOptimized inference for a given trained neural network and target GPUSolutions for Hyperscale, ADAS, EmbeddedSupports deployment

13、of 32-bit or 16-bit or int8 inferenceMaximum Performance for Deep Learning Inference/gpu-inference-engineGPU Inference Engine for AutomotivePedestrian DetectionLane TrackingTraffic Sign Recognition-NVIDIA DRIVE PX 2TENSORRT: GPU INFERENCE ENGINEOptimizationsFuse network layersEliminate concatenation

14、 layersKernel specializationAuto-tuning for target platformSelect optimal tensor layoutBatch size tuningTRAINEDNEURAL NETWORKOPTIMIZED INFERENCE RUNTIME/gpu-inference-engineGPU INFERENCE ENGINEGoogleNet PerformanceBATCH=1M4TX1TX1 FP16GIE3.7 ms13.9 ms16.5ms (N=2)Caffe15 ms33 msn/a/gpu-inference-engin

15、eBATCH=16M4TX1TX1 FP16GIE39 ms164 ms99 msCaffe67 ms255 msn/aJetson TX1 HALF2 column uses fp1644NCCL: A multi-GPU collective communicationlibrary4-GPU PCIE TREE+ exists today+ all GPUs have P2P accessPCIe is bottleneckno NVLinkcan be thought of as unidirectional ringPCIe 12 GB/sCPUGPU0GPU1GPU2GPU3PCI

16、eswitch46810120.00010.0010.01101001000Bandwidth GB/s0.11Problem size MiBTheoryDataALL-REDUCEperformance at different problem sizes (4 GPUs)latency: 55 us20max achieved: 9.6 GB/smax P2P memcpy:10.4 GB/s46810120.00010.0010.01101001000Bandwidth GB/s0.11Problem size MiBTheoryDataBROADCASTperformance at different problem sizes (4 GPUs)latency: 38 us20max achieved: 10.4 GB/smax P2P memcpy:10.4 GB/s46810120.00010.0011001000Bandwidth GB/s0.010.1110Problem size per GPU (4x in total) MiBTheoryDataREDUCE-SCATTERperformance at different problem sizes (4 GPUs)latency: 48