CSA大型语言模型（LLM）威胁分类 Large Language Model (LLM) Threats Taxonomy

LargeLanguageModel(LLM)ThreatsTaxonomy

ThepermanentandofficiallocationfortheAIControlsFrameworkWorkingGroupis

/research/working-groups/ai-controls

©2024CloudSecurityAlliance–AllRightsReserved.Youmaydownload,store,displayonyour

computer,view,print,andlinktotheCloudSecurityAllianceat

subject

tothefollowing:(a)thedraftmaybeusedsolelyforyourpersonal,informational,noncommercialuse;(b)thedraftmaynotbemodifiedoralteredinanyway;(c)thedraftmaynotberedistributed;and(d)the

trademark,copyrightorothernoticesmaynotberemoved.Youmayquoteportionsofthedraftas

permittedbytheFairUseprovisionsoftheUnitedStatesCopyrightAct,providedthatyouattributetheportionstotheCloudSecurityAlliance.

©Copyright2024,CloudSecurityAlliance.Allrightsreserved.2

Acknowledgments

LeadAuthors

Reviewers

SiahBurke

PhilAlger

MarcoCapotondi

IlangoAllikuzhi

DanieleCatteddu

BakrAbdouh

KenHuang

VinayBansalVijayBolinaBrianBrinkley

Contributors

AnupamChatterjeeJasonClinton

MarinaBregkou

VidyaBalasubramanian

AlanCurranSandyDunnDavidGee

AvishayBar

ZackHamilton

MonicaChakrabortyAntonChuvakin

RicardoFerreiraAlessandroGrecoKrystalJackson

VicHargraveJerryHuang

RajeshKambleGianKapoorRicoKomenda

GianKapoor

VaniMittal

KushalKumar

AnkitaKumariYutaoMa

DannyManimboVishwasManralJesusLuna

MichaelRoza

LarsRuddigheit

JasonMorton

AmeyaNaik

GabrielNwajiakuMeghanaParwatePrabalPathak

RuchirPatwa

BrianPendletonKunalPradhan

DorSarig

Dr.MattRoldan

AmitSharma

RakeshSharmaKurtSeifried

CalebSima

EricTierling

JenniferToren

RobvanderVeerAshishVashishthaSounilYu

DennisXu

OmarSantos

Dr.JoshuaScarpino

NataliaSemenova

BhuvaneswariSelvaduraiJamillahShakoor

TalShapira

AkramSheriff

SrinivasTatipamula

Maria(MJ)SchwengerMahmoudZamani

RaphaelZimme

©Copyright2024,CloudSecurityAlliance.Allrightsreserved.3

TableofContents

Acknowledgments 3

TableofContents 4

ObjectivesandScope 5

RelationshipwiththeCSAAIControlFramework 6

1.LargeLanguageModelAssets 7

1.1.DataAssets 7

1.2.LLM-OpsCloudEnvironment 9

1.3.Model 10

1.4.OrchestratedServices 11

1.5.AIApplications 13

2.LLM-ServiceLifecycle 15

2.1Preparation 16

2.2Development 17

2.3Evaluation/Validation 18

2.4Deployment 20

2.5Delivery 22

2.6ServiceRetirement 24

3.LLM-ServiceImpactCategories 26

4.LLMServiceThreatCategories 26

4.1.ModelManipulation 26

4.2.DataPoisoning 27

4.3.SensitiveDataDisclosure 27

4.4.ModelTheft 27

4.5.ModelFailure/Malfunctioning 27

4.6.InsecureSupplyChain 27

4.7.InsecureApps/Plugins 27

4.8.DenialofService(DoS) 28

4.9.LossofGovernance/Compliance 28

5.References/Sources 29

©Copyright2024,CloudSecurityAlliance.Allrightsreserved.4

ObjectivesandScope

ThisdocumentwasauthoredbytheCloudSecurityAlliance(CSA)ArtificialIntelligence(AI)Controls

FrameworkWorkingGroup,withinthecontextoftheCSAAISafetyInitiative.Itestablishesacommon

taxonomyanddefinitionsforkeytermsrelatedtoriskscenariosandthreatstoLargeLanguageModels(LLMs).ThegoalistoprovideasharedlanguageandconceptualframeworktofacilitatecommunicationandalignmentwithintheIndustryandtosupportadditionalresearchwithinthecontextoftheCSAAI

SafetyInitiative.Morespecifically,thesedefinitionsandtaxonomyareintendedtoassisttheCSAAIControlWorkingGroupandtheCSAAITechnologyandRiskWorkingGroupintheirongoingefforts.

Inthiseffort,wefocusonthedefinitionofthefollowingelements(SeeFigure1):

●LLMAssets

●LLM-ServiceLifecycle

●LLM-ServiceImpactCategories

●LLM-ServiceThreatCategories

Figure1:CSALLMThreatTaxonomy

©Copyright2024,CloudSecurityAlliance.Allrightsreserved.5

Thesedefinitionsandtaxonomyreflectanextensivereviewoftheavailableliterature,aswellasmeetingsanddiscussionsamongWorkingGroupmembersandco-chairs.Throughthiscollaborativeexercise,a

strongconsensusemerged,establishingafoundationalsetofcommonterminologiesguidingourcollectiveefforts.

Thisdocumentdrawsinspirationfromnumerousindustryreferencescitedattheendofthedocument,andmostnotablyfromNISTAI100-2E2023titled“AdversarialMachineLearning:ATaxonomyand

TerminologyofAttacksandMitigations”[Barrettetal.,2023].

Withthesedefinitionsandtaxonomy,conversationsregardingtheevaluationofAIthreatsandrisks,

developingappropriatecontrolmeasures,andgoverningresponsibleAIdevelopmentcanadvancewithgreaterclarityandconsistencyacrossdiverseCSAgroupsandamongstakeholders.Establishinga

commonnomenclaturereducesconfusion,helpsconnectrelatedconcepts,andfacilitatesmoreprecisedialogue.ThisdocumentconsolidateskeytermsintoacentralreferenceservingthepurposeofaligningboththeAIControlWorkingGroupandtheAITechandRiskWorkingGroupwithintheCSAAISafetyInitiative.

RelationshipwiththeCSAAIControlFramework

TheCSAAIControlFrameworkWorkingGroup’sgoalistodefineaframeworkofcontrolobjectivestosupportorganizationsintheirsecureandresponsibledevelopment,management,anduseofAI

technologies.TheframeworkwillassistinevaluatingrisksanddefiningcontrolsrelatedtoGenerativeAI(GenAI),particularlyLLMs.

Thecontrolobjectiveswillcoveraspectsrelatedtocybersecurity.Additionally,itwillcoveraspectsrelatedtosafety,privacy,transparency,accountability,andexplainabilityasfarastheyrelatetocybersecurity.

PleasereviewCSA’sblogposttoexplorethedifferencesandcommonalitiesbetween

AISafetyandAI

Security

.

Byfocusingonthebusiness-to-businessimplications,theCSAAIControlFrameworkcomplements

governmentefforts1inprotectingnationalsecurity,citizen’srightsandlegalenforcement,advocatingforsecureandethicalAIapplicationsthatcomplywithglobalstandardsandregulations.

1E.g.EUAIAct,U.S.ArtificialIntelligenceSafetyInstitute(USAISI),etc.

©Copyright2024,CloudSecurityAlliance.Allrightsreserved.6

1.LargeLanguageModelAssets

ThissectiondefinesthefoundationalcomponentsessentialforimplementingandmanagingLLM

systems,fromthedetaileddataassetscrucialfortrainingandfine-tuningthesemodels,tothecomplexLLM-Opsenvironment,ensuringseamlessdeploymentandoperationofAIsystems.Furthermore,this

sectionclarifiestheLLM'ssignificance,architecture,capabilities,andoptimizationtechniques(seeFigure2).Additionally,thissectionexploresthevitalaspectofassetprotection,leveragingtheResponsible,

Accountable,Consulted,Informed(RACI)matrixtodelineateresponsibilitieswithinbothopen-sourcecommunitiesandorganizationstowardsimplementationofAIservices.

Figure2:LLMAssets

1.1.DataAssets

InLLMservices,manyassetsplayanintegralroleinshapingaservice'sefficacyandfunctionality.Data

assetsareattheforefrontoftheseassetsandserveasthecornerstoneofLLMoperations.ThelistbelowdescribesthetypicalrangeofassetsconstitutinganLLMService:

©Copyright2024,CloudSecurityAlliance.Allrightsreserved.7

●Datausedfortraining,benchmarking,testing,andvalidation

●Datausedforfine-tunetraining

●DatausedforRetrieval-AugmentedGeneration(RAG)

●Datacardsthatdefinethemetadataofthedatainuse

●Inputdata

●Usersessiondata

●Modeloutputdata

●Modelparameters(weights)

●Modelhyperparameters

●LogdatafromLLMsystems

Thefollowingarethedefinitionsoftheseassets:

1.Training,benchmarking,testing,andvalidationdata:Thisencompassesthedatasetusedtotrain,benchmark,test,andvalidatethemodel,consistingoftextsourcesfromwhichthemodelderivesinsightsintolanguagepatterns,andsemanticsthatareimperativeforqualityofthemodel.Eachdataelementis

treatedandmanagedindividually.

2.Fine-tunetrainingdata:Additionaldataisemployedtofine-tuneorfurtherpre-trainthemodelpost-initialtraining.Thisfacilitatesadjustmentstothemodel’sparameterstoalignmorecloselywithspecificusecasesordomains,enhancingitsadaptabilityandaccuracy.

3.Retrieval-AugmentedGeneration(RAG):IntegratesexternalknowledgebaseswithLLMs.By

retrievingrelevantinformationbeforegeneratingresponses,RAGenablesLLMstoleveragebothmodelknowledgeandexternalknowledgeeffectively.RAGcanretrievesupplementarydatafromvarious

sources,includinginternalsystems,andpublicsources,suchastheInternet,enrichinginputpromptsandrefiningthemodel'scontextualunderstandingtoproducehigher-qualityresponses.

4.Datacards:MetadataofthedatasetsusedforvariouspurposesinLLMneedstobemaintained.ThishelpsgovernAIdataandprovideslineage,traceability,ownership,datasensitivity,andcompliance

regimesforeverydatasetused.Storingandthencontinuouslyupdatingdatacardsasthedata,ownership,orrequirementschangeisessentialtomaintaincomplianceandvisibility.

5.Inputdata(system-levelprompt):Theinputdataisprovidedtosetthecontextandboundaries

aroundLLMsystems.Thesedatasetsareadditionallyusedtosettopicboundariesandguardrailsincaseofadversarialgeneration.

6.Usersessiondata:InformationamassedduringuserinteractionswiththeAIsystems,encompassinginputqueries,model-generatedresponses,andanysupplementarycontextprovidedbyusers,facilitatingpersonalizedinteractions.

7.Modeloutputdata:Theresultantoutputgeneratedbythemodelinresponsetoinputprompts,encompassingtextresponses,predictions,orotherformsofprocesseddata,reflectiveofthemodel'scomprehensionandinferencecapabilities.

©Copyright2024,CloudSecurityAlliance.Allrightsreserved.8

8.Modelparameters(weights):Internalparametersorweightsacquiredbythemodelduringtraining,delineatingitsbehaviorandexertingaprofoundinfluenceonitscapacitytogenerateandcontextuallyrelevantresponses.

9.Modelhyperparameters:Configurationsorsettingsspecifiedduringmodeltraining,including

parameterssuchaslearningrate,batchsize,orarchitecturechoices,arepivotalinshapingthemodel'soverallperformanceandbehavior.

10.Logdata:Recordeddataencapsulatingvariouseventsandinteractionsduringthemodel'soperation,

includinginputprompts,modelresponses,performancemetrics,andanyencounterederrorsoranomalies,instrumentalformonitoringandrefiningthemodel'sfunctionalityandperformance.

1.2.LLM-OpsCloudEnvironment

TheLLM-OpsEnvironmentencompassestheinfrastructureandprocessesinvolvedinthedeploymentandoperationofLLMs.Thefollowingbulletpointsarethekeytermsassociatedwiththisenvironment:

●Cloudrunningthetrainingenvironment

●Cloudrunningthemodelinferencepoint

●CloudrunningtheAIapplications

●Hybridandmulti-cloudinfrastructure

●Securityofthedeploymentenvironment

●Continuousmonitoring

●Cloudtohosttrainingdata(Storage)

ThesignificanceandessenceofeachoftheaboveassetwithintheframeworkoftheLLM-OpsEnvironmentisdescribedbelow:

1.Cloudrunningthetrainingenvironment:Thisdenotesthecloudplatformorserviceproviderentrustedwithhostingandmanagingthecomputationalresources,storagefacilities,andancillaryinfrastructurepivotalfortrainingLLMs.Itservesasthedevelopmentspacewheremodelsundergoiterativerefinementandenhancement.

2.Cloudrunningthemodelinferencepoint:Thisencapsulatesthecloudplatformorserviceprovidertaskedwithhostingandadministeringthecomputationalresources,storagesolutions,andassociated

infrastructureindispensablefordeployingLLMsandfacilitatinginferenceprocesses.Itenablesthemodeltogenerateresponsesbasedonuserinputs,ensuringseamlessinteractionandresponsiveness.

3.Public/Private/HybridCloudRunningtheAIapplications:ThisreferstothecloudplatformorserviceproviderentrustedwithhostingandoverseeingtheinfrastructureessentialforrunningAI

applicationsorAIservices,harnessingthecapabilitiesoftrainedlanguagemodels.ItservesastheoperationalhubwhereAI-drivenapplicationsleveragetheinferenceprowessofmodelstodelivervalue-addedfunctionalitiesandservicestoend-users.

4.Securityofthedeploymentenvironment:ThisencompassesthearrayofmechanismsandpoliciesimplementedtogovernandfortifyaccesstotheassortedcomponentsoftheLLM-OpsEnvironment.It

©Copyright2024,CloudSecurityAlliance.Allrightsreserved.9

encompassesIdentityandAccessManagement(IAM)protocolsandnetworksecuritymeasures,safeguardingtheintegrityandconfidentialityofcriticalassetsandfunctionalities.

5.Continuousmonitoring:ThisdenotestheongoingprocessofvigilantlyscrutinizingtheLLM-OpsEnvironment'sperformance,securityposture,andoverallwell-being.Itencompassesthevigilant

surveillanceofthetrainingenvironment,inferenceendpoint,andapplicationcomponents,ensuringoptimalfunctionalitywhilepromptlyidentifyingandremedyinganyanomaliesorissuesthatmayarise.

6.Cloudtohosttrainingdata(Storage):Thissignifiesthecloudplatformorserviceprovidertaskedwithsecurelyhousingandmanagingtheextensivedatasetsrequisitefortraininglanguagemodels.Itentailsrobuststorageanddatamanagementcapabilitiestoaccommodatethevoluminousanddiversedatasetsfundamentalfornurturingandrefininglanguagemodels.

1.3.Model

Theconceptof"Model"inthecontextofMLreferstoamathematicalrepresentationoranalgorithmtrainedtomakepredictionsorperformaspecifictask.

Thechoiceoffoundationmodel,fine-tuningapproach,andthedecisiontouseopen-sourceor

closed-sourcemodelscansignificantlyaffectLLMs'capabilities,performance,anddeploymentflexibilitywithinvariousapplicationsanddomains.

Wedefinethefollowingmodelassetsinthissubsection:

●FoundationModel

●Fine-TunedModel

●OpenSourcevs.ClosedSourceModels

●Domain-SpecificModels

●Modelcards

1.FoundationModel:

TheFoundationModelisthebaseuponwhichfurtheradvancementsarebuilt.Thesemodelsaretypicallylarge,pre-trainedlanguagemodelsthatencapsulateabroadunderstandingoflanguage,obtainedfromextensiveexposuretounlabeledtextdatathroughself-supervisedlearningtechniques.Foundation

models,ingeneral,provideastartingpointforsubsequentfine-tuningandspecializationtocaterto

specifictasksordomains.Forsomeadvancedandinnovativefoundationmodels,anotherterm,

“Frontier

Model”

canbeusedtorepresentabrandnewfoundationmodelintheAIMarketplace.FromanAIperspective,sometimestheterm“BaseModel''representsfoundationmodelsintheapplicationtechnologystacks.

2.Fine-TunedModel:

DerivedfromtheFoundationModel,theFine-TunedModelundergoesrefinementandadaptationto

catertospecifictasksordomains.Throughtheprocessoffine-tuning,theparametersofthefoundationmodelareupdatedutilizingsupervisedlearningtechniquesandtask-specificlabeleddata.Thisiterativeprocessenablesthemodeltoenhanceitsperformanceontargettasksordomainswhileretainingthe

foundationalknowledgeandcapabilitiesinheritedfromtheFoundationModel.

©Copyright2024,CloudSecurityAlliance.Allrightsreserved.10

3.Open-Sourcevs.Closed-SourceModels:

Thisdichotomypertainstotheaccessibilityandlicensingofamodel'ssourcecode,modelweights,andassociatedartifacts.Open-sourcemodelsmayreleasesomeoralloftheirtrainingdataandsourcecode,datausedforthemodeldevelopment,modelarchitecture,weights,andtoolstothepublicunder

open-sourcelicenses,grantingfreeusagewithspecifictermsandconditions.However,closed-sourcemodelsmaintainproprietarystatus,withholdingtheirsourcecode,weights,andimplementationdetailsfromthepublicdomain,oftenmotivatedbyintellectualpropertyprotectionorcommercialinterests.

Closed-sourcemodelsthatallowuserstoaccessthemodelsforfinetuningorinferencepurposesarecalledOpenaccessmodels.

Thesemodelassetscollectivelyformthebackboneofmodeldevelopment,fosteringinnovation,adaptability,andaccessibilitywithinGenAI.

4.Domain-SpecificModels:

Domain-specificmodelsrefertomachinelearningmodelsthataredesignedandtrainedtoexcelonspecificdomainknowledge,suchasfinancial,medicines,andcoding.

5.Modelcards:

Thecharacteristicsofmodelscanbedescribedusingmodelcards.ModelcardsarefilesthatmaintainthecontextofthemodelwhichisessentialforGovernanceandmakingsureAImodelscanbeusedcorrectly.Modelcards2consistofmodelcontextdetailslikeownership,performancecharacteristics,datasetsthemodelistrainedon,orderoftrainingetc.Thisalsohelpswithtraceability,lineageandunderstandingthebehaviorofthemodel.Modelcardsneedtobecontinuouslymaintainedandupdatedasthecontext

metadatachanges.[CSA,2024]

Moredetailsofmodelcardscanbefound,forexample,atthe

HuggingFace

platform,wherethemachinelearningcommunitycollaboratesonmodels,datasets,andapplications.

1.4.OrchestratedServices

TheseservicesencompassarangeofcomponentsandfunctionalitiesthatenabletheefficientandsecureoperationofLLMs.

ThefollowingisthelistofOrchestratedServicesAssets:

●CachingServices

●SecurityGateways(LLMGateways)

●DeploymentServices

●MonitoringServices

●OptimizationServices

●Plug-insforSecurity

●Plug-insforCustomizationandIntegration

●LLMGeneralAgents

2Formoredetailson‘Modelcards’pleaseconsultthe‘AIModelRiskManagementFramework’ofthe

AIRiskandTechnology

workinggroup

.

©Copyright2024,CloudSecurityAlliance.Allrightsreserved.11

Definitionandsignificanceofeachoftheabovelistedassetswithinthecontextoforchestratedservicesfollowsbelow.

1.CachingServices:

CachingServicesrefertosystemsorcomponentsthatfacilitatethecachingofmodelpredictions,inputs,orotherdatatoenhanceperformancebyreducingredundantcomputations.Bytemporarilystoring

frequentlyaccesseddata,cachingserviceshelpminimizeresponsetimesandalleviatecomputationalstrainonLLMs.

2.SecurityGateways(LLMGateways):

SecurityGateways,alsoknownasLLMGateways,arespecializedcomponentsthatserveas

intermediariesbetweenLLMsandexternalsystems.Thesegatewaysbolstersecuritybyimplementingaccesscontrolmeasures,inputvalidation,filteringmaliciouscontent(suchaspromptinjections),

PII/privacyinformation,andsafeguardsagainstpotentialthreatsormisuse,ensuringtheintegrityandconfidentialityofdataprocessedbyLLMs.

3.DeploymentServices:

DeploymentServicesstreamlinethedeploymentandscalingofLLMsacrossdiverseenvironments,includingcloudplatformsandon-premisesinfrastructure.Theseservicesautomatedeployment

processes,facilitateversionmanagement,andoptimizeresourceallocationtoensureefficientandseamlessLLMdeployment.

4.MonitoringServices:

MonitoringServicesarepivotalinoverseeingLLMsecurity,performance,health,andusage.These

servicesemploymonitoringtoolsandtechniquestogatherreal-timeinsights,detectanomalies,misuse(suchaspromptinjections)andissuealerts,enablingsecurity,proactivemaintenance,andtimely

interventiontoupholdtheoptimaloperationofLLMs.

5.OptimizationServices:

OptimizationServicesaregearedtowardsoptimizingtheperformanceandresourceutilizationofLLMs.Theseservicesemployarangeoftechniquessuchasmodelquantization,pruning,efficientinference

strategiestoenhanceLLMefficiency,reductionofcomputationaloverhead,andimprovementofoverallperformanceacrossdiversedeploymentscenarios.

6.Plug-insforSecurity:

Securityplug-insextendLLMsecuritybyprovidingdataencryption,accesscontrolmechanisms,threatdetectioncapabilities,andcomplianceenforcementmeasures,thusincreasingcyberresiliency.

7.Plug-insforCustomizationandIntegration:

Plug-insforCustomizationandIntegrationenablethecustomizationofLLMbehaviorandseamless

integrationwithothersystems,applications,ordatasources.Theseplug-insprovideflexibilityintailoring

LLMfunctionalitiestospecificusecasesordomainsandfacilitateinteroperabilitywithexistinginfrastructure,fosteringenhancedversatilityandutilityofLLMdeployments.

©Copyright2024,CloudSecurityAlliance.Allrightsreserved.12

8.LLMGeneralAgents:

LLMGeneralAgentsareintelligentagentsorcomponentscollaboratingwithLLMstoaugmenttheirfunctionalitiesandcapabilities.Theseagentsmayperformvarioustasks,suchas

●planning,

●reflection,

●functioncalling,

●monitoring,

●dataprocessing,

●explainability,

●optimization,

●scaling,andcollaboration,

●andenhancingtheversatilityandadaptabilityofLLMdeploymentsindiverseoperationalcontexts.

1.5.AIApplications

AIapplicationshavebecomeubiquitous,permeatingvariousfacetsofourdailylivesandbusiness

operations.Fromcontentgenerationtolanguagetranslationandbeyond,AIapplicationsfueledbyLLMshaverevolutionizedindustriesandreshapedhowweinteractwithinformationandtechnology.However,withtheproliferationofAIapplicationscomestheimperativeneedforeffectivecontrolframeworksto

governtheirdevelopment,deployment,andusage.

AIapplicationsrepresentthepinnacleofinnovation,offeringmanycapabilitiesthatcatertodiverse

businessdomainsandusecases.TheseapplicationsleveragethepowerofLLMstodecipherandprocessnaturallanguageinputs,enablingfunctionalitiessuchascontentgeneration,questionanswering,

sentimentanalysis,languagetranslation,andmore.Essentially,AIapplicationsserveastheinterface

throughwhichusersinteractwiththeunderlyingintelligenceofLLMs,facilitatingseamlesscommunicationandtaskautomationacrossvariousdomains.

AsdownstreamapplicationsofLLMs,AIapplicationsareoneofthemostimportantassetstoconsiderinanAIcontrolframework.TheyrepresentthedirecttouchpointbetweenLLMtechnologyandend-users,shapinghowusersperceiveandinteractwithAIsystems.Assuch,AIapplicationshavethepotentialtoamplifythebenefitsorrisksassociatedwithLLMs.

AIapplicationscanhavesignificanteconomicimpacts.AsbusinessesincreasinglyrelyonAIapplicationstodriveinnovation,streamlineoperations,andgaincompetitiveadvantages,theresponsible

developmentanddeploymentoftheseapplicationsbecomecrucialformaintainingmarketintegrityandfosteringalevelplayingfield.

Giventheseconsiderations,anAIcontrolframeworkmustprioritizethegovernanceandoversightofAIapplications.ThisincludesestablishingguidelinesandstandardsforAIapplicationdevelopment,testing,deployment,operation,andmaintenance,ensuringcompliancewithrelevantregulations,andpromotingtransparencyandaccountabilitythroughouttheAIapplicationlifecycle.Additionally,theframework

shouldfacilitatecontinuousmonitoringandevaluationofAIapplications,enablingtimelyidentificationandmitigationofpotentialrisksorunintendedconsequences.

©Copyright2024,CloudSecurityAlliance.Allrightsreserved.13

ByprioritizingAIapplicationsintheAIcontrolframework,organizationscanproactivelyaddressthechallengesandrisksassociatedwithLLM-poweredapplicationswhileunlockingtheirtransformativepotentialtodriveinnovationandimprovelives.

AIapplicationcardsarefilesthatmaintaintheAIcontextoftheapplicationwhichisessentialfor

governanceoftheapplication.AIapplicationcardsconveytheAIdataoftheapplications,including

modelsused,datasetsused,applicationandAIusecases,applicationowners(seedifferentkindsof

ownersfromtheRACImodelinthenextsection),andguardians.AIapplicationcardsareaneasywayto

conveyandshareAIdataforapplications,tohelpAIgovernanceexecutives,AIcouncils,andregulatorstounderstandtheapplicationandtheAIituses.TheAIapplicationcardsmayinturnpointtomodeland

datacards.

©Copyright2024,CloudSecurityAlliance.Allrightsreserved.14

2.LLM-ServiceLifecycle

TheLLM-ServiceLifecycleoutlinesdistinctphases,eachcrucialinensuringtheservice'sefficiency,

reliability,andrelevancethroughoutitslifespan.Fromthepreparatorystagesofconceptualizationand

planningtotheeventualarchivinganddisposal,eachphaseisintricatelyintegratedintoacomprehensiveframeworkdesignedtoimproveservicedeliveryandmaintainalignmentwithevolvingneedsand

standards.Organizationscanmanageservicedevelopment,evaluation,deployment,delivery,andretirementthroughthisstructuredapproachwithclarityandeffectiveness.

DrawinguponemergingstandardslikeISO/IEC5338onAIsystemlifecycles,andreviewsfrom

organizationsliketheUK'sCentreforDataEthicsandInnovation(CDEI),thislifecyclecoverstheend-to-endprocess,fromearlypreparationanddesignthroughtraining,evaluation,deployment,operation,andeventuallyretirement.

Thefollowingisthehigh-levelbreakdownofthelifecyclewewilldefineinthissection.

●Preparation:

。Datacollection

。Datacuration

。Datastorage

。Resourceprovisioning。Teamandexpertise

●Development:

。Design。Training

。Keyconsiderationsduringdevelopment。Guardrails

●Evaluation/Validation:

。Evaluation

。Validation/RedTeaming。Re-evaluation

。Keyconsiderationsduringevaluation/validation

●Deployment:

。Orchestration

。AIServicessupplychain。AIapplications

●Delivery:

。Operations。Maintenance

。Continuousmonitoring。Continuousimprovement

©Copyright2024,CloudS