2024微软人工智能系统计算图的编译与优化

1 y+*bxy+*bxw算子表达式IR2.内存优化3.内核优化4.调度优化 AddLogWhileSubMatMulMergeMulConvBroadCastDivBatchNormReduceAddLogWhileSubMatMulMergeMulConvBroadCastDivBatchNormReduceReluLossMapTanhTransposeReshapeExpConcatenateSelectFloorSigmoid….. xwxw*b+yxwxw*b+y p1/(p0+(p3-p4)) PAGE12PAGE12Batchsame-typeoperatorsleverageGPUmassiveparallelism++×𝝈+M𝝈×𝒕𝒂𝒏𝒉+ +MMMMMRf Rzht-1xtData-flowgraphofaGRUcellWzWoWfht+×𝝈+M+×𝝈+M𝝈×𝒕𝒂𝒏𝒉+ +MMMMMRf Rzht-1xtWzWoWfht+×𝝈×𝝈+M𝒕𝒂𝒏𝒉Mht-1RWxthtData-flowgraphofaGRUcellPAGEPAGE14 1.计算图优化2内存优化3.内核优化4.调度优化PAGEPAGE19 PAGE21PAGE21AutoTM:AutomaticTensorMovementinHeterogeneousMemorySystemsusingIntegerLinearProgramming AutoTM:AutomaticTensorMovementinHeterogeneousMemorySystemsusingIntegerLinearProgrammingPAGEPAGE23 1.计算图优化2.内存优化3内核优化4.调度优化问题问题:每个后端平台针对每个算子都需要单独实现至少一个内核考虑：编程模型、数据排布、线程模型、缓存大小等等y+*bxw xw *xw*b+y张量运算表达式张量运算表达式C[i]=A[i]+B[i]361.计算图优化2.内存优化3.内核优化4调度优化y+*bxwy+*bxw算子表达式IRD[i]=max(C[i],0)C[i]=A[i]+B[i]

计算图优化XLA,TVM,nGraph计算图优化XLA,TVM,nGraphD[i]=max(A[i]+B[i],0)D[i]=max(A[i]+B[i],0)MatMulMatMul:C[i]=reduce_sum(A[m,i]*B[i,n])globalglobaladd_relu_kernel(args…){//..Code}voidlaunch_kernel(add_relu_kernel);算子生成AutoTVM,TC,Halide,Tiramisu,etcSMSMSMSMSMSMSMSMSMSMSM如何最终将每个算子的计算映射到硬件上？NNFusion:全局计算调度优化xw*b+yxw*b+y算子表达式IRTask-parallelGraph算子表达式IRBlockFusionOptimizerBlockFusionOptimizer 计算调度优化计算调度优化通过引出新的调度原语来支持任务级调度挑战：调度算子的任务到挑战：调度算子的任务到GPU上的挑战 错误依赖死锁

block7Conv

block8Conv

block9

block10

CannotschedulebarrierCo-scheduleto

Matmulblock0ConvMatmul

Matmulblock1ConvMatmul

Matmulblock2ConvMatmul

Matmulblock3ConvMatmul

Matmulblock4ConvMatmul

block5ConvMatmul

WaitforbarrierUnabletoexecuteGPUGPUSMSMSMSMSMSM依赖关系的映射依赖关系的映射 MatMulConvBE0BE1BE2BE3BE4BE5vblock6MatMulConvBE0BE1BE2BE3BE4BE5vblock6vblock5vblock5vblock4vblock10vblock4vblock7vblock0vblock8vblock1vblock6vblock0vblock7vblock1vblock2vblock3vblock8vblock2vblock9vblock3GPUGPUSMSMSMSMSMSM并行度的映射并行度的映射 [64,512][512,128][64,512][512,128]LowerLatencyUnderutilizationofGPUresourceBE0BE1BE2LowerLatencyUnderutilizationofGPUresourceBE0BE1BE2BE3BE4BE5vblock5vblock4 MatMul 𝜎Convvblock6vblock7vblo