2024微软人工智能系统概述

Self-driving Surveillancedetection Medicaldiagnostics GamePersonalassistant

DeepLearning深度学习正在改变世界

Art Imagerecognition Speechrecognition Naturallanguage Generativemodel Reinforcementlearning catdoghoneybadgercatdoghoneybadger

CatDogRaccoonlossloss𝑑𝑤1

𝑑𝑤2

𝑑𝑤3

𝑑𝑤4

𝑑𝑤5

ErrorsDogRDMA14Mimages海量的（标识）数据RDMA14Mimages

深度学习算法的进步 语言、框架

计算能力 深度学习+系统的进步:编程语言、优化、计算机体系结构、并行计算以及分布式系统E.g.,imageclassificationproblemMNISTImageNetWebImages60Ksamples16MsamplesBillionsofImages10categories1000categoriesOpenedcategoriesTESTERRORRATE(%)TESTERRORRATE(%)123AlexNet,16.4%ReLU,Dropout,2012Inception,6.7%Batchnormalization,2015ResNet,3.57%Residualway,2015 AlexNet,16.4%ReLU,Dropout,2012Inception,6.7%Batchnormalization,2015ResNet,3.57%Residualway,2015EfficientNet,3.1%NASLeNet,convolution,max-pooling,softmax,1998EfficientNet,3.1%NASLeNet,convolution,max-pooling,softmax,1998 ImagerecognitionSpeechrecognition

NaturallanguageReinforcementlearning TPUv3360TPUv3360TopsV100TPUv1125Tops90TopsPerformance(Op/Sec)?TPUDedicatedPerformance(Op/Sec)?TPUDedicatedHardwareGPUCPUMoore’slaw5KopsENIAC~500GopsXeonE5108x105x

1970 1980 1990 2000

2019CompilerBackendTVMTensorFlowXLACompilerBackendTVMTensorFlowXLALanguageFrontendSwiftforTensorFlowMxNetCNTKLanguageFrontendSwiftforTensorFlowMxNetCNTKPyTorchCustompurposemachinelearningalgorithmsTheanoDisBeliefCaffeAlgebra&linearAlgebra&linearlibsCPUGPUDensematmulengineGPUFPGASpecialAIacceleratorsTPUGraphCoreOtherASICs CustompurposemachinelearningalgorithmsTheanoCustompurposemachinelearningalgorithmsTheanoDisBeliefCaffeDeeplearningframeworksprovideeasierwaystoleveragevariouslibrariesMachineLearningLanguageandCompilerPowerfulCompilerInfrastructure:Codeoptimization,sparsityoptimization,hardwaretargetingAFull-FeaturedProgrammingLanguageforML:ExpressiveandflexibleControlflow,recursion,sparsityAlgebra&Algebra&linearlibsCPUGPUAIframeworkDensematmulengineSIMD→MIMDSparsitySupportControlFlowandDynamicityAssociatedMemory End-to-EndAIUserExperiencesModel,Algorithm,Pipeline,Experiment,End-to-EndAIUserExperiencesModel,Algorithm,Pipeline,Experiment,LifeCycleManagementProgrammingInterfacesComputationgraph,(auto)GradientcalculationIR,CompilerinfrastructureProgrammingInterfacesComputationgraph,(auto)GradientcalculationIR,CompilerinfrastructureHardwareAPIs(GPU,CPU,FPGA,ASIC)ResourceManagement/SchedulerHardwareAPIs(GPU,CPU,FPGA,ASIC)ResourceManagement/Scheduler ScalableNetworkStack(RDMA,IB,NVLink)DeepLearningRuntime:Optimizer,Planner,ExecutorArchitecture(singlenodeandCloud)

class3class4class5class6class7class8更广泛的AI系统生态class

机器学习新模式(RL)

深度学习算法和框架classclassclass

自动机器学习(AutoML)安全与隐私模型推导、压缩与优化

通用AI算法支持与进化深度神经网络编译架构及优化

深度学习任务运行和优 通用资源管理和调度化环境 统

新型硬件及相关高性能网络和计算栈 （2）开始训练

定义网络结构 Fullyconnected最后几层

Convolutionalneuralnetwork等Locality强的数据

Recurrentneuralnetwork化的数据，比如文本信息、知识图

Transformerneuralnetwork比如文本信息 #ArecursiveTreeBankmodelinadozenlinesofJPLcode#Walkthetree,accumulatingembeddingvecs#Wordembeddingmodelisusedattheleafnodetomapword#indexintohigh-dimensionalsemanticwordrepresentation.#Getsemanticrepresentationsforleftandrightchildren.#Acompositionfunctionisusedtolearnsemantic#representationforphraseattheinternalnode.#Maptreeembeddingtosentiment

更多样化的结构更复杂的依赖关系更细粒度的计算模式ExecutionRuntimeCPU,GPU,RDMAdevicesGraphdefinition(IR)xw*b+yFront-endLanguageBinding:Python,Lua,R,C++OptimizationBatching,Cache,Overlap ExecutionRuntimeCPU,GPU,RDMAdevicesGraphdefinition(IR)xw*b+yFront-endLanguageBinding:Python,Lua,R,C++OptimizationBatching,Cache,OverlapData-FlowGraph(DFG)asIntermediateRepresentation

x y z*a+bΣc

TensorFlow AddgradientbackpropagationAddgradientbackpropagationData-FlowGraph(DFG)xyz𝛻x𝛻y*a*𝐠𝛻z+bΣc+𝐠𝛻a𝛻bΣ𝐠x y z

𝛻x 𝛻yCPUcodeGPUcode

* a+ +𝐠b 𝛻bΣ Σ𝐠c

𝛻a

𝛻zxyz𝛻x𝛻y*a*𝐠𝛻z+bΣc+𝐠𝛻a𝛻bΣ𝐠xyz𝛻x𝛻y*a*𝐠𝛻z+bΣc+𝐠𝛻a𝛻bΣ𝐠......1......1Operators IDEProgrammingwith:VSCode,JupiterNotebookIDEProgrammingwith:VSCode,JupiterNotebookLanguageIntegratedwithmainstreamPL:PyTorchandTensorFlowinsidePythonCompilerIntermediaterepresentationCompilationOptimizationBasicdatastructure:TensorLexicalanalysis:TokenUsercontrolled:mini-batchBasiccomputation:DAGParsing:ASTDataparallelismandmodelparallelismAdvancefeatures:controlflowSemanticanalysis:SymbolicADLoopnetsanalysis:pipelineparallelism,controlflowGeneralIRs:MLIRCodeoptimizationDataflowanalysis:Arithmetic,FusionCodegenerationHardwaredependentoptimizations:matrixcomputation,layoutResourceallocationandscheduler:memory,recomputation,RuntimesSinglenode:CuDNNMultimode:Parameterservers,AllreducerComputationclusterresourcemanagementandjobschedulerHardwareHardwareaccelerators:CPU/GPU/ASIC/FPGANetworkaccelerators:RDMA/IB/NVLinkFrameworksArchitectureCompilerBackendTVMTensorFlowXLALanguageFrontendSwiftforTensorFlowMxNetTensorFlowCNTKPyTorch CompilerBackendTVMTensorFlowXLALanguageFrontendSwiftforTensorFlowMxNetTensorFlowCNTKPyTorchDeeplearningframeworksSpecialAIacceleratorsTPUGraphCoreOtherASICsAIFrameworkDensematmulengineGPUFPGAimport"tensorflow/core/framework/to";SpecialAIacceleratorsTPUGraphCoreOtherASICsAIFrameworkDensematmulengineGPUFPGAMachineLearningLanguageandCompilerPowerfulCompilerInfrastructure:Codeoptimization,sparsityoptimization,hardwaretargetingAFull-FeaturedProgrammingLanguageforML:ExpressiveandflexibleControlflow,recursion,sparsityMachineLearningLanguageandCompilerPowerfulCompilerInfrastructure:Codeoptimization,sparsityoptimization,hardwaretargetingAFull-FeaturedProgrammingLanguageforML:ExpressiveandflexibleControlflow,recursion,sparsitySIMD→SIMD→MIMDSparsitySupportControlFlowandDynamicityAssociatedMemory//SyntacticallysimilartoLLVM:func@testFunction(%arg0:i32){%x=call@thingToCall(%arg0):(i32)->i32br^bb1^bb1:%y=addi%x,%x:i32return%y:i32}深度学习高度依赖数据规模和模型规模

8layers1.416%Error2012AlexNet

Image152layersGFLOP%Error2015ResNetSpeech提高训练速度可以加快深度学习模型的开发速度大规模部署深度学习模型需要更快和更高效的推演速度Inferenceperformance→Servinglatency

80GFLOP7,000hrsofData8%Error2014DeepSpeech1

465GFLOP12,000hrsofData5%Error2015DeepSpeech2 Differentarchitectures:CNN,RNN,Transformer,…

Highcomputationresourcerequirements:modelsize,…Differentgoals:throughput,accuracy,…BeBetransparenttovarioususerrequirementsapplyoverheterogeneoushardwareenvironmentScale-out LocalEfficiency MemoryEffectivenessHardware SSD CPU/GPU/FGPA InfiniBand/NVLinkHyper-params OptimizerMini-batchLearningrateOptimizations Hardware SSD CPU/GPU/FGPA InfiniBand/NVLinkHyper-params OptimizerMini-batchLearning