NVIDIA 面向大语言模型的全栈 AI 解决方案

NVIDIA面向大语言模型的全栈AI解决方案大语言模型生态系统快速演进导致优化异常挑战•新的算子和定制技术使得优化成为持续变化的目标需要全栈AI解决方案助力模型开发和应用部署LLaMaLLaMaGPTFalconStarcoderChatGLMMPT…ImagefromMooler0410/LLMsPracticalGuideASurveyonChatGPTandBeyond.arXiv[Cs.CL].Retrievedfrom/abs/2304.13712FP32FP16BF16FP8(E5M2)FP8(E4M3)FP8相较于FP16/BF162倍的吞吐signmultiplyaccumulateintoTCRangePrecisionexponentsignmultiplyaccumulateintoTCRangePrecisionexponentmantissaDD——s—D——DD—— bias/act/… convertFP32|FP16|BF16|FP8TransformerEngineFP8混合精度训练其它操作仍在高精度参数更新 TransformerEngine算子融合混合精度训练thehistoryandJax•Hyperparameter:GBS2048_SEQ4096_MBS2_TP2_PP1_DP128•Hyperparameter:GBS1024_SEQ4096_MBS4_TP4_PP1_DP32HellaSwagWinoGrandeHellaSwagWinoGrande数据中心级别的计算机由网络定义数据中心AIEthernetFabric•TraditionalEthernetnetworkcansuffice•TraditionalEthernetnetworkforNorth-Southtraffic•NVIDIASpectrum-XEthernetforAINVIDIASpectrum-X-世界上第一款为AI而生的以太网平台融合标准以太网连接和专业的高性能架构PerfectEffectiveStandardEthernetSpectrum-4Spectrum-4EthernetSwitchsupportedsoftwarethatspeedsdeveloperexecutionofeverydeveloper’sjobssupportedsoftwarethatspeedsdeveloperexecutionofeverydeveloper’sjobsthousandsAccelerateend-to-performanceNVIDIAAIEnterpriseAIFrameworksNVIDIAAIWorkflowManagementandMLOpsJobScheduling&OrchestrationPROVISIONINGMONITORINGMANAGINGClusterManagementPROVISIONINGMONITORINGMANAGINGCLUSTERINGCLUSTERINGNETWORKIOSTORAGEIOIOMANAGEMENTNetwork/StorageNETWORKIOSTORAGEIOIOMANAGEMENTIN-NETWORKCOMPUTEIN-NETWORKCOMPUTEOperatingSystemNVIDIANeMo大语言模型应用落地全流程Acceleratedcitedresponses数据处理提升模型性能NeMoDataCuratorsteps:•Classifier-basedfilteringInternetscaledatasetsInternetscaledatasets.△.口△口口△口!!extracttextTextre- NVIDIAMegatron-CorePerformanceatScalePerformanceatScaleMemory,Compute,andCommunicationOptimizationParallelismTechniquesMemorySavingTechniquesSelectiveActivationRecomputeDistributedOptimizersPrecisionawareoptimizers(BF16,FP8)withFusedAdam●●●●●●●●●●●architectureFormalizedProductFormalizedProductSupportLatestresearchandNVIDIANeMo对大语言模型定制的套件根据场景定制大语言模型的不同方式Accuracyforspecificuse-casesPROMPTENGINEERINGPROMPTLEARNINGPARAMETEREFFICIENTFINE-TUNINGFINETUNINGPROMPTENGINEERING•LeastexpertiseBenefits••SFTNVIDIATriton推理服务器快速可扩展简单的推理服务开源软件(Jetson)Kubernetes,KServe,PlatformsLatencyLLM生产部署的极致性能TensorRT-LLMisanopen-sourcelibrarytooptimizeinferenceperformanceonthelatestLargeLanguageModelsforNVIDIAGPUs.ItisSoTAPerformancePerformanceTCOEaseExtension#defineanewactivationdefsilu(input:Tensor)→Tensor:#implementmodelslikeinDLFWsclassLlamaModel(Module)def__init__(…)self.layers=ModuleList([…])forlayerinself.layers:LLMBatchingwithTritonStaticInflightBatchingCost为RAG带来最佳的准确度和延时Acceleratedvectorsearch★ApplyforEarlyAccessStateoftheArtAccuracy•Latesttechniquesforchunkingandmetadata•Alldatasources,suchasunstructured(vectordatabases)andstructured(SQL,Tabular)LowestLatencyandHighestThroughputProductionReadywithEnterpriseSupport•ProductionreadyservicenterprisesupportNVIDIANeMoGuardrails保持LLM安全TopicalGuardrailsTopicalGuardrailsSafetyGuardrailsSecurityGuardrailsLLMs听取ChinaAIDay任一演讲，即可获取DL会后您将收到优惠码专属邮件任选右侧一门公开课使用优惠码（仅限一门）开启LLM实战之旅构建基于大语言模型（LLM）的应用利用大模型开源生态系统快速开发基于预训练大语言模型的应用基础基础高效定制大语言模型（LLM）学习提示工程和各类高效参数微调方法对预训练LLM模型进行定制以适应特定应用进阶进阶大语言模型（LLM）、生成式AI系列近期公开