揭开人工智能在交通网络安全中的神秘面纱（英文）

上传人：策*** IP属地：山西上传时间：2024-12-29 格式：DOCX 页数：128 大小：5.38MB 积分：19.9 举报 版权申诉

已阅读5页，还剩123页未读，继续免费阅读

版权说明：本文档由用户提供并上传，收益归属内容提供方，若内容存在侵权，请进行举报或认领

文档简介

DEMYSTIFYINGARTIFICIALINTELLIGENCEINTRANSPORTATIONCYBERSECURITY

URBANJONSON,SVPITANDCYBERSECURITYSERVICES,SERJON

URBANJONSON

ujonson@

Current

SVPInformationTechnologyandCybersecurityServices,SERJON,LLC

USFBIInfraGardTransportationSubjectMatterExpert

FBIAutomotiveSectorSpecificWorkingGroup(SSWG)

BoardofDirectors,CyberTruckChallenge

ProgramCommittee,ESCARUSA

SAEVehicleElectricalSystemSecurityCommitteeMember

Technology&MaintenanceCouncil(TMC)S.5andS.12StudyGroupMember

Experience

Over35yearsofexperienceinITandCybersecurity,includingstrategicplanning,assessments,projectmanagement,andprogrammanagement

Variouspapers,talks,andresearchonhacking,aswellasdefendingtrucksandtransportationingeneral

Abusinganddefendingsystemssincethe1980s

AGENDA

•AIOverview

•AITaxonomy

•ChallengeswithAI

•CommonAImistakes

•TransportationApplications

•AttackingAI

•DefendingAI

•Wrap-up

ImagegeneratedbyBingImageCreator

AIOVERVIEW

HISTORICALOVERVIEW

•1950s–BirthofAI

•AlanTuringandotherslaidthegroundworkformachineintelligence

•1960s–EarlyApplications

•Problem-solvingandsymbolicreasoning,e.g.playingchess

•1980s–ExpertSystems

•Programsdesignedtomimichumanexpertise,e.g.taxsoftware

•1990s–MachineLearningResurgence

•NeuralnetworksandnewalgorithmsrevitalizeinterestinAI

HISTORICALOVERVIEW

•2000s–RiseofBigData

•Availabilityoflargedatasetsandimprovedcomputingpower

•2010s–DeepLearningDominance

•Multi-layeredneuralnetworkscapableofimagerecognition,speechrecognition

•2020s–GenerativeAI

•NewLargeLanguageModels(LLM)builtonmassivedatacloudplatformscapableofgeneratingimages,code,andother

content(ChatGPT,BingImageCreator,etc.)basedoninputprompts

TECHNICALOVERVIEW

UNDERSTANDABLEVSPREDICTIVEPOWER

Image:NationalCyberSecurityCentre(UK)-Principlesforthesecurityofmachinelearning

[.uk/collection/machine-learning]

MACHINELEARNINGAPPLICATIONS

Commonmachine

learninganalytic

applications

Image:

/science/article/pii/S0951832021003835

ARTIFICIALNEURALNETS

Thecorecomponent

ofneuralnetsisthe

artificialneuron

Conceptuallycanbethoughtofasamini

linearregressionmodel

Image:

/blog/artificial-neural-networks-basics-guide/

TRAININGMETHODS

Image:

/science/article/pii/S0951832021003835

MODELTRAINING

Simplesupervisedlearning

Imagesareconvertedintonumericaldatausuallybyflatteningintoavector

Image:

/@MITIBMLab/estimating-information-flow-in-deep-neural-networks-b2a77bdda7a7

AITAXONOMY

OVERVIEW

NISTAIUseTaxonomy*:

•Decomposescomplexhuman-AItasksintoactivitiesthatareindependentoftechnologicaltechniques(e.g.,neural

network,largelanguagemodel,reinforcementlearning)anddomains(e.g.,finance,medicine,law).

•ProvidesaflexiblemeansofclassifyinganAIsystem’scontributiontoaspecifiedhuman-AItask.

•Intendedtobealivingdocumentthatisupdatedperiodicallywithfeedbackfromstakeholders,suchasthoseintheAI

evaluationandhumanfactorscommunities.

*NISTTrustworthyandResponsibleAINISTAI200-1,AIUseTaxonomy:AHuman-CenteredApproach,byTheofanos,Choong,andJenson,March2024,

/10.6028/NIST.AI.200-1

AITAXONOMY-TRANSPORTATION

•Connectionist–Learningalgorithmsbasedonneuralnetworks

•Bayesians–Probability-basedinferencesystems

•Symbolists–Logic-basedalgorithmssuchasrules-based

programming,decisiontrees,fuzzylogic,andrationalagents

•Analogizers–Similarity-basedclassifiers,suchassupportvectormachines

•Optimizations–Algorithmsperformingiterativeupdatesand

comparisonstodiscoveroptimumsolutions,e.g.GeneticAlgorithm(GA)

JonPerez-Cerrolaza,JaumeAbella,MarkusBorg,CarloDonzella,JesúsCerquides,FranciscoJ.Cazorla,CristoferEnglund,MarkusTauber,GeorgeNikolakopoulos,andJoseLuisFlores.

2024.ArtificialIntelligenceforSafety-CriticalSystemsinIndustrialandTransportationDomains:ASurvey.ACMComput.Surv.56,7,Article176(July2024),40pages.

/10.1145/3626314

CHALLENGESWITHAI

NON-DETERMINISTIC

•Thesameinputswillnotalwaysgeneratethesameoutputs

•Randomdataselectionbasedonprobabilitycurves

•Anytimeyouaddrandomdataselectioninconnectionistmodels,youruntheriskofnon-deterministicoutcomes

•Ifyourmodelcontinuestolearn,forexamplelinearregression,theoutputswillvaryovertimeasmodellearns

•Expertsystemsgenerallydonotsufferfromthisproblem,butanything

thathasaneuralnetworkwillruntherisk

INSCRUTABLE

•ThemathandcodeforAIiscompletelyunderstandable…..and“human-ish”readable

•Thedatathatyouusetotrainthemodelis

understandable(hopefully,ifyouhavedoneyourjobright)

•Theproblemiswhenyouusethecodetogenerateamodelbasedonthedata

•Duetohowthemodellearns(developingacomplexwebofprobabilisticweights)andisexpressed,itisnot

possibletolookatthemodelandunderstandhowitworks

UNEXPLAINABLE

•Sincethemodelisnon-deterministicandinscrutable,itisnoteasilyunderstood

•Makesexplaining“why”amodelproducedtheexactoutputexceedinglydifficultfor

neuralnetworks

•ExplainableandTrustworthyAIisanareaofintenseresearch

•TrustworthyAIcangenerateatrusted

explanationthathumanscanunderstand

WHYTHEPROBLEM?

•Layeredneuralnetworks

•Randomlygeneratedvalues

•Probabilisticevaluations

•Deeplearningisstatisticswithlinearalgebra

Image:

/tutorial/introduction-to-deep-neural-networks

DATAPROBLEM

•Ourmodelsareonlyasgoodasourdata

•Transportationdatasetsareintheirinfancy

•Wearestillinthegreat“dataownership”battle

•Ourvehicleplatformslackthesensorstocollectthenecessaryinformation(possible

exception…Tesla)

•Modelsareverylimitedinwhattheycando

TRANSPORTATIONDATASAMPLES

•Lackofinstrumentation

•Teslaprobablyhasbestdataset

•VehicleISasensorplatform

•Designedtocollectalldata

•Driversprovidingexperience

•Robottaxifleetdata2ndplace

•Cruise

•Waymo

•ProprietaryDataSources

•OEMdata

•Telematicsdata

•Vehicle/Fleetoperatordata

•OpenData

•PIVOT(

)

•EUDataAct

•ColoradoStateUniversity

TRAININGDATAPROBLEM

•DeeplearningandLLMsrequiremassivedatasetsforlearningandvalidation

•LLMs,suchasChatGPT,haveusedagreatdealofinternetcontent

•Manyimages,text,books,etc.usedinlearningmodelsarecopyrightedmaterials

•IsgeneratinganAImodelbasedonsomeoneelse’swork

a“fairuse”ofcopyrights?

•WhatiftheresultingAIismonetized?

•Howdoyouremoveonepartorsegmentoftrainingdataonceamodelhasbeencreated?

EDGECASES

•Edgecasesarestatisticaloutliereventsthatarenotpartofthetrainingdata

•Thoughtheymayberare,theycan

resultinunexpectedandundesirableoutcomes

•Edgecasesarewheretragedylives

Source:Projectguru.in

UBER

•UberAutonomousCrashMarch2018

•Pedestrianwalkingbicycleacrosstheroad

•Vehicleidentifiesandtrackspedestrian

•Vehicledoesnotbreak

•Safetydriverwasdistractedanddidnotact

•Factoryauto-breakingsystemdisabledsoasnottointerferewith

automateddrivingsoftware

•Exampleoflimitationsofdataandwhathappenswhennotfollowingfunctionalsafetybestpractices

CRUISE

•CruiseaccidentOctober2023

•Pedestriancrossesroadagainstdonotwalksignal

•Pedestriangetshalfwayacrossbeforecrosstrafficforcespedestriantowalkback

•Pedestrianhitbyacarandthrownintothepathofcruisevehicle

CRUISE

•Cruisevehicleisacceleratingeventhoughit“sees”pedestrian

•Vehicledoesnotrecognizescenario(edgecase)

•PedestriangetstrappedundertheCruisevehicle

•Vehiclesystemrecognizessomethingiswrong

•Insteadofstopping,thevehicledrivesforwardandpullsover,draggingthepedestrianunderthecar

•Whynotstop?

•Exampleoflimitationsofdataandwhathappenswhennotfollowingfunctionalsafetybestpractices

COMMONAIMISTAKES

LACKOFUNDERSTANDINGMODELS

•Mathishard,andlibrariesareeasy

•ThereareTONSofdifferentAImodelsandapproaches

•

TensorFlowiseasyandrequiresalmostnothinking

•

AnacondaextendsaccesstoeveryonewhocanPythonAmodelorimplementationinsearchofaproblem

•

Sometimes,neuralnetworksarenotthebestsolution

•

OftenseeproblemssolvedwithMLthatshouldbesolvedbyexpertsystems

•

Lackofunderstandingoflimitations

•

DisconnectbetweenAIandfunctionalsafety

DATASCIENCEDISCIPLINE

•Traditionalprogrammingisbasedonrequirements

•AIisbasedonDATA,whichmakesdatasciencecritical

•“Garbagein,garbageout”x1000

•Acommonerroristhatdatasciencelifecyclenotfollowed

•Smalldatasetsforlearningandvalidationareproblematic

•Lackofproperlysizeddatasetsleadto:

•Overfitmodelschasingaccuracy

•Falseminimaandmaximaforoptimizationproblems

•Higherprobabilityeventsarenotincludedinthemodel

•Significantmisclassificationscomparedtoreal-worlddata

Experimentationandprediction

DATASCIENCELIFECYCLE

preparation

Data

Explorationandvisualization

•Datacollectionandstorage

•Defineprojectobjectives

•Collectdata

•Normalizestorageandformat

•Explorationandvisualization

•Statisticaldataanalysis

•Datalabeling

•Datapreparation

•Missingorinconsistentdata

•Cleaningandaugmentingdata

•Removingduplicates

•Normalization

•Datalabeling

•Datatypeconversation

•Graphsandchartsforunderstanding

•Experimentationandprediction

•Trydifferentmodelsandapproaches

•Identifydatatrendsandpatterns

•Discoverinsights

•Buildmodel

Data

collection/storage

Source:

/blog/what-is-data-science-the-definitive-guide

TRANSPORTATIONAPPLICATIONS

ASSISTINGDRIVERS

•Safety-related“assistant”applicationstoreduceimpairedordistracteddriving

•Lane-keepingassist

•Intelligentcruisecontrol

•Automaticemergencybraking

•Limitcellphoneusewhiledriving(cellphone“drivingmode”)

•Limitcellphonedistractionsviapredictivebehaviororvehicleintegration

•Bettermusicplaylistprediction

•Bettermapsanddirections

•Recognizeimpaireddriving

•Inallthesescenarios,thedriverisstillthemaincontrollingactor

ANOMALY/ERRORDETECTION

•Vehiclepredictivemaintenance

•Batterylife

•Tirewear

•Many,manymore…..

•MotorfreightcarriersandTSPs

•Analyzebatteryvoltages

•Exhaustsensors

•Noiseandvibrationsensors

•CANbusmessages

•IntermittentDTCs

ANOMALY/ERRORDETECTION

•Vehiclecybersecurity[atscale]

•VSOC

•Faultpatterns

•Geolocationtrends

•IndividualvehicleIDSstillproblematic

•Lackoftrainingandvalidationdata

•Rule-basedexpertsystemsareprobablymoreeffective

•CompanionroleforML(seefunctionalsafety“safetybag”examples)

GENERALTRANSPORTATION

•TherearemanyapplicationsofAIinTransportationManagementSystems(TMS)andTrafficManagementSystems(TMS)

•Freightmovementoptimization

•Fuelconservation

•Mostefficientroutecalculations

•Trafficmanagement

•Parkingefficiencyandoptimization

FUNCTIONAL-SAFETY

•Functionalsafetyiswell-knownpracticewithspecificrulesandknownapproachestoachievingsafety

•Mitigationtechniquestodealwithuncertainty

•Safetybag(thinkinput/outputparametervalidation)

•Safetymonitors

•Diagnostics

•Formalmethods

•Functionalsafetysystemslikecrashavoidanceandlanedepartureassistthatcontainclassifiermodelsarenotprimarysafetysystems

•Driverremainstheprimarysafetysystemincontrolofthevehicle

FUNCTIONAL-SAFETY

•Neuralnetwork-basedAIisapoorchoiceforfunctionalsafetysystems

•Testingmassivelycomplexnon-deterministicsystemsisalmostimpossible

•Caveat:FormalmethodscombinedwithML

•Impossibletoexplainwhyamodelbehavesinacertainway

•Introducessafetyrisksandmassivelegalliabilities

AUTONOMOUSVEHICLES

•ExistingMLmodelsareunsuitableforSAELevel3–5automation

•Wedonothavethedatatousethemeffectively

•CurrentMLmodelsarenotexplainableortrustworthy

•MoreadvancedMLmodelsarenon-deterministic

•MLissuitableforclassifiersandpreceptorsbutnotatthe

accuracyrequiredforfunctionalsafety

•Closedandcontrolledenvironmentsarepossible

•Real-worldpublicroadsandadversarialenvironmentsaretoocomplexwheresafetystandardscannotbemet

Image:createdbymonkik

ATTACKINGAI

•Traditionaltechniquesarestillapplicable

•Denialofserviceattacks

•Softwarestackvulnerabilitiesandexploits

•Hostingandruntimeenvironmentexploitation

•Socialengineering

•TherearenewattackmethodstargetingAI

•MITREAdversarialThreatLandscapeforAI

•OWASPTop10MachineLearningRisks

Systems(ATLAS™)

CLASSIFICATIONINPUTMANIPULATION

•Modificationofaninput(e.g.image,sensorvalue)tocause:

•Misclassification

•Triggererrorconditions

•Alterintendedbehavior(inference-basedsystems)

•Acommonexampleisstopsign“modification”:

•Addingtapetocausemisclassification

•Shiningbrightlightsalteringgradientanalysis

•Alargenumberofacademicpapersonhowtomessuptrafficsignalinputs

•FewMLmodelsareimmune

•Distinctfrompromptmanipulation(coveredlater)

EXPLOITEDGECASES

•Limitationsofavailabledataallowedgecaseexploitation

•Analyzethemodelanddeterminelow-probabilityinputs

•UseanAItofuzzanotherAImodeltodeterminelimitations

•Causethemodeltobehaveincorrectlyorevencrash

•Especiallyeffectiveifinputandoutputvaluesarenotvalidatedandboundschecked

•Maycausesystemfailureorsoftwarestackmalfunction

•Errorconditionscancauseremotecodeexecutionordataexfiltrationopportunities

PROMPTINJECTION/MANIPULATION

•ApplicabletoLLMmodelswhichgenerateoutputbasedonprompts

•OnewaytothinkofthisisasSQLInjectionattacks,butinsteadoftargetingaSQLDB,theunderlyingmodelistargeted

•Injection/promptattackscancause:

•Datadisclosure

•Modelcorruption

•Hostingsystemcorruption

•Bypasssafetyorcontentrestrictions

•Wecanalsousesocialengineeringtrickstogetthemodeltodothingsitisnotsupposedtodo(hardastrickinga4yrold)

TRAININGDATAPOISONING

•AImodelsarebuiltfromthetrainingdata

•Duetolackoftrainingdata,manytrainingsetsarebasedonpublicdatasources

•Poisoningapublicdatasetcanintroduce

•Backdoors

•Remotecodeexecutionerrors

•Anynumberofmalwarescenarios

•Feedingmaliciousdataintoacontinuouslearningmodelcancausemodeldriftandeventualmodelfailure

•ContinuouslylearningIDSsystems

•Self-optimizingmodels

HACKING

EXAMPLE

HACKINGADASMODEL

•BlackhatAsia2024-TheKeytoRemoteVehicleControl:AutonomousDrivingDomainController

•ShupengGao,SeniorSecurityResearcher,Baidu

•YingtaoZeng,SeniorSecurityResearcher,Baidu

•JieGao,SeniorSecurityResearcher,Baidu

•YimiHu,SeniorSecurityResearcher,Baidu

•Analyzedover30ADASdevices

•50%hadSHHenabled

•Somedeployedwithoriginalmodelfiles(*.onnx)

•Littleornodiskencryption

/asia-24/briefings/schedule/index.html#the-key-to-remote-vehicle-control-autonomous-driving-domain-contr

oller-38089

HACKINGADASMODEL

•ADASunitcanbeapathwaytototalvehiclecompromiseasitneedstobeaccessibleonCANnetworkandupdateable

•Researchers:

•Abletooffloadentiremodelfiles,includingoriginalmodelfiles

•Abletoreadthemodelanddeployina$50toycar

•Possibletoupdatemodelonoriginaldevice

•PoorlysecuredADASmodulecanleadtototalvehiclecontrol

•Steering

•Braking

•Powertrain

DEFENDINGAI

AI-WHATISTHESAME?

Themorethingschange,themoretheyremainthesame:

•Softwarestackvulnerabilities

•Operatingsystemvulnerabilities

•Softwaresupplychainattacks

•Hardwarefirmware

•PlatformOS

•Browservulnerabilities

•ITandDevSecOpsbestpracticesstillapply

•Sanitizeandvalidateinputsandoutputs

AI-WHAT’SDIFFERENT?

•Systemsarebuiltgroundupfromdata,notrequirements

•Awholenewmindsetforadversarialattackvectors

•Increasedsupplychaincomplexity

•Inputvalidationbecomesharderandmoreimportant

•Datahandlingproceduresaremoreimportant

•Erroneous,mislabeled,orincompletedatacanhavebigimpact

•DevSecOpsneedstoincorporatedataandmachinelearningmodels

•Datascientistsneedtounderstandcybersecurity

AI–DEFENSIVEBESTPRACTICES

Whilenotanexhaustivelist,herearesomebestpractices:

•ProtectsystemboundariesbetweenITandAI

•Identifyandprotectallproprietarydata

•StrongaccessandauthorizationcontrolsforfinalAImodelweights

•Hardenthedeploymentenvironment

•Applyversionlabelstomodels(changestoweights)

•Validateallinputsforedgecasesandattacks

•Validatealloutputstoensureoperationinsideboundaries(safetybag)

DEFENSIVERESOURCES

ThereareseveralrecentgoodpublicationsoncybersecuritybestpracticesfordeployingmachinelearningandAIingeneral:

•NationalCyberSecurityCentre(UK)-Principlesforthesecurityofmachine

learning

.uk/collection/machine-learning

•NationalCyberSecurityCentre(UK)–GuidelinesforsecureAIsystem

development

.uk/collection/guidelines-secure-ai-system-development

•JointCybersecurityInformation-DeployingAISystemsSecurely

/2024/Apr/15/2003439257/-1/-1/0/CSI-DEPLOYING-AI-

SYSTEMS-SECURELY.PDF

WRAP-UP

AIFUTURE

•Improvementsincustomerservice

•Improvementsinoperationalefficiency

•Developingbetterdesigns

•Assistingindevelopmentofnewmaterials

•Inspectingandevaluatinginfrastructure

•Improvingsafetythroughnewdriverassistancefeatures

•Increasefleetuptime

……ExplainableandtrustworthyAIwillbringmoreapplications

FURTHERREADING

IfyouarenewtoAI/ML,Icanhighlyrecommendthefollowingbookasagoodstartingpoint

•Ozdemir,S.,Kakade,S.,&Tibaldeschi,M.(2018).PrincipalsofDataScience(2nded.).PacktPublishing.

/product/principles-of-data

-science-second-edition/9781789804546

Ifyouarelookingforamorein-depthfoundationalbook,Iwouldrecommendthefollowingbook:

•Kelleher,J.D.,MacNamee,B.,&D’Arcy,A.(2015).FundamentalsofMachineLearningforPredictiveDataAnalytics.TheMITPress.

FURTHERREADING

Forthosewhowanttogettotheheartofthemathandbuildyourownmodels,includingdeepregressionlearning,Iwouldrecommendthefollowingbook:

•Goodfellow,I.,Bengio,Y.,&Courville,A.(2017).DeepLearning.TheMITPress.Iwouldalsorecommendthefollowingpaperformoreinformationaboutsafety-criticalAIapplicationsintransportationtohelptietogethertheuseofAIintransportation:

•JonPerez-Cerrolaza,JaumeAbella,MarkusBorg,CarloDonzella,Jesús

Cerquides,FranciscoJ.Cazorla,CristoferEnglund,MarkusTauber,George

Nikolakopoulos,andJoseLuisFlores.2024.ArtificialIntelligenceforSafety-CriticalSystemsin

人人文库> 全部分类> 应用文书 > 研究报告

温馨提示

1. 本站所有资源如无特殊说明，都需要本地电脑安装OFFICE2007和PDF阅读器。图纸软件为CAD,CAXA,PROE,UG,SolidWorks等.压缩文件请下载最新的WinRAR软件解压。
2. 本站的文档不包含任何第三方提供的附件图纸等，如果需要附件，请联系上传者。文件的所有权益归上传用户所有。
3. 本站RAR压缩包中若带图纸，网页内容里面会有图纸预览，若没有图纸预览就没有图纸。
4. 未经权益所有人同意不得将文件中的内容挪作商业或盈利用途。
5. 人人文库网仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对用户上传分享的文档内容本身不做任何修改或编辑，并不能对任何下载内容负责。
6. 下载文件中如有侵权或不适当内容，请与我们联系，我们立即纠正。
7. 本站不保证下载资源的准确性、安全性和完整性, 同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。

揭开人工智能在交通网络安全中的神秘面纱（英文）

文档简介

温馨提示

最新文档

评论

揭开人工智能在交通网络安全中的神秘面纱（英文）

文档简介

温馨提示

最新文档

评论

相关文档