数据与模型安全 课件 第11周：联邦学习

Federated

Learning姜育刚，马兴军，吴祖煊/2017/04/federated-learning-collaborative.htmlRecap:week10口 CommonTamperingandDeepfakes口 ImageManipulationDetection口 VideoManipulationDetectionThisWeek口 FederatedLearning口 PrivacyinFederatedLearning口 RobustnessinFederatedLearning口 ChallengesandFutureResearchTraditionalMachineLearningDataModelDataandmodelinonesingleplaceTraditionalMachineLearningDataModelWhat

if

we

need

more

data?DataGatheringUsingmultipleGPUsFederatedLearning:Whatisit?Google:FederatedLearning:CollaborativeMachineLearningwithoutCentralizedTrainingDataFederatedLearning:Challenges,Methods,andFutureDirections,/pdf/1908.07873.pdfNextwordpredictiononmobile.FederatedMachineLearning:ConceptandApplications,/pdf/1902.04885.pdfHorizontalFL（横着切）:samefeatures,differentsamplesFederatedLearning:TypesVerticalFL（纵着切）:samesamples,differentfeaturesFederatedLearning:TypesFederatedMachineLearning:ConceptandApplications,/pdf/1902.04885.pdfFederatedLearning:TypesFederatedTransferLearning:differentsamples,differentfeaturesFederatedMachineLearning:ConceptandApplications,/pdf/1902.04885.pdfCompareDifferentParadigmsFederatedMachineLearning:ConceptandApplications,/pdf/1902.04885.pdfCompareDifferentParadigmshttps:///projects/distributed-learning-and-collaborative-learning-1/overview/SplitLearningvsFederatedLearningFederatedLearningFrameworksHE:homomorphicencryption SS:secretSharingObjectivesandUpdatesinFLGlobalobjectiveLocalobjective:LocalUpdates:GlobalAggregation(e.g.FedAvg):FederatedLearning–MajorChallengesExpensiveCommunicationSystemsHeterogeneityStatisticalHeterogeneityPrivacyandSecurityConcernsFederatedLearning:Challenges,Methods,andFutureDirections,/pdf/1908.07873.pdfFederatedLearning-HorizontalFederatedLearning:Challenges,Methods,andFutureDirections,/pdf/1908.07873.pdfHFLcanfurtherbedividedinto…?PrivacyandSecurityThreatsLyuetal.“Privacyandrobustnessinfederatedlearning:Attacksanddefenses.”TNNLS,2022.SummaryofThreatModelsFLserver(insider)FLparticipants(insider)Eavesdroppers(outsider)Serviceusers(outsider)□InsidervsOutsider □InsiderAttacksByzantine:theworstattacker,knowseverythingaboutthesystem,doesnotobeytheprotocol,sendarbitraryupdates,evencolludewitheachother.Sybil:takingoverthenetworkbysimulatingmanydummyparticipants,out-votethehonestusersSemi-honestvsMaliciousSemi-honestsettingMalicioussettingTraining-timevsTest-timeStealprivatedata,stealmodel,corruptthemodel(trainingtime)Adversarialattack(testtime)SummaryofAttacksExistingattacksagainstserver-basedFLPoisoningAttacksDatapoisoningvsmodel(weight)poisoningDataPoisoningAttacksinTraditionalML□Dirty-labelPoisoningLabelflipping(onlychangelabels)Dirty-labelbackdoor(changeinputsandlabels)Clean-labelPoisoningClean-labelbackdoor(onlychangeinputs)DataPoisoningAttacksinTraditionalMLAsimplepatterncanmakethemodeltomemorizeFLPoisoningAttacks–ModelPoisoningMaincharacteristics:ChangelocalmodelweightsMostlyByzantineattack(attackercandoanythingtotheweights)CanattackByzantine-robustaggregationmechanismssuchasKrumandcoordinate-wisemedianinsteadofweightedaveragingKrum:PrivacyAttacksForeverycommunicationround,localclientshavethechancetoreverseengineerothers’gradients.Fromthereversedgradients,reverseengineer:RepresentationsMembershipPropertiesSensitiveattributesInVFL:featuresPrivacyAttacks–InferenceAttacksDeepmodelsundertheGAN:informationleakagefromcollaborativedeeplearning,CCS2017InferenceclassrepresentationsusingGANsCIFAR-10horseclassReconstructAlice’sfaceimagePrivacyAttacks–InferenceAttacksComprehensiveprivacyanalysisofdeeplearning:Passiveandactivewhite-boxinferenceattacksagainstcentralizedandfederatedlearning,S&P,2019Inferencemembership:Passiveattacks:observeandinference.Activeattacks:influencethetargetmodelinordertoextractmoreinformation.WeaknessofFL:FLcreatesanenvironmentfor(almost)white-boxattacksPrivacyAttacks–InferenceAttacksOtherinferenceattacks:inferringproperties,trainingdata,labels...DeepLeakagefromGradient(DLG)ImprovedDeepLeakagefromGradient(iDLG)…Defenses–PrivacyDefenseHomomorphic

Encryption:RSAEl

GamalPaillier…Homomorphic

properties:Allows

computation

directly

onencrypted

data(“可算不可见”)Needs

to

be

designed

for

eachalgorithmA

side

note:

attacking

encrypted

FL

is

challengingbut

still

possible!Defenses–PrivacyDefense2.

SecureMultipartyComputation(SMC,Yaosharing):SecureML(data-independentofflinephase+fastonlinephase)Offlinemultiplicationtriplets,truncate,sharingCharacteristics:HighlevelprivacyHighcomputationandcommunicationcostYao'sMillionaires'problemProtocolsforSecureComputations,AndrewChi-ChihYao,1982,UCBerkeleyDefenses–PrivacyDefense2.DifferentialPrivacy(DP):TypesofDP:LocalDPCentralizedDPDistributedDPDefenses–PrivacyDefenseDataflowofstatisticsunderLDP2.DifferentialPrivacy(DP):Defenses–PrivacyDefense2.DifferentialPrivacy(DP):TypesoffrequencyestimationDefenses–PrivacyDefense2.DifferentialPrivacy(DP):Real-worldapplications.Vanilla

FLM:ADPmechanismCentralized

DPM:ADPmechanismLocal

DPM:ADPmechanismE:encryptionD:decryptionDistributed

DPDefenses–ByzantineDefenseAlgorithm:Krum(forByzantinerobustness)Setting:nparticipants,fareByzantine,with𝒏≥𝟐𝒇+𝟑Atcommunicationroundt,𝟏 𝟐 𝒏serverreceives{𝜹𝒕,𝜹𝒕,…,𝜹𝒕}foreach𝜹𝒕:𝒊selecttheclosest(L2distance)n-f-2intoset𝑪𝒊compute𝒔𝒄𝒐𝒓𝒆𝜹𝒕=∑𝒊 𝜹∈𝑪𝒊 𝒊𝜹𝒕−𝜹𝜹𝟏 𝒏𝜹𝒌𝒓𝒖𝒎=𝜹∗=argmin{𝒔𝒄𝒐𝒓𝒆𝜹𝒕 ,…,𝒔𝒄𝒐𝒓𝒆𝜹𝒕}updateglobalparameter:𝒘𝒕.𝟏=𝒘𝒕+𝜹𝒌𝒓𝒖𝒎Blanchardetal.“Machinelearningwithadversaries:Byzantinetolerantgradientdescent.”NeurIPS,2017.Defenses–ByzantineDefenseAlgorithm:Krum(forByzantinerobustness)Blanchard

et

al.

“Machine

learning

with

adversaries:

Byzantine

tolerant

gradient

descent.”

NeurIPS,

2017.红色：攻击梯度蓝色：真实梯度黑色：本地梯度黑色曲线：损失函数Defenses–ByzantineDefenseMorerobustaggregationmethods:Multi-Krum=Krum+Averaging=Krumrobustness+increasedconvergencespeedcoordinate-wisemedian,coordinate-wisetrimmedmeanmedianisnotgoodforconvergenceBulyan=Krum+trimmedmedianMedianandgeometric-median(RobustFederatedAggregation)RFA:approximategeometricmedian(notrobusttoByzantineattacks)Defenses–ByzantineDefenseModelpoisoningattackcanbreakKrumandcoordinate-wisemedianAnalyzingfederatedlearningthroughanadversariallens,ICML2019.𝜏/:adversarialtargetclassr:numberofpoisonedsamples𝐷0:cleandata1𝑤?2:estimationoftheglobalparametersReversedgradientsfromthelastround.Defenses–SybilDefenseFromtraditionalML:RejectonNegativeInfluence(RONI)WithacleanvalidationdatasetItrequiresuniformdistributioninnon-IIDsetting,notgood.FoolsGold:Sybilsharethesameobjective,driftsawayfromtheoriginalobjectiveCoreidea:cosinesimilarityFoolsGold:MitigatingSybilsinFederatedLearningPoisoning,/abs/1808.04866Defenses–SybilDefenseDistributedbackdoorattack(DBA)canbypassbothRFAandFoolsGold.DBA:Distributed

Backdoor

Attacks

against

Federated

Learning,

ICLR

2020.

Defenses

-

SummaryDefenseagainstFederatedLearningPoisoning.n:numberofparticipants.RemainingChallengesandFutureResearch□ CurseofdimensionalityLargermodelsaremorevulnerableSharingweights/gradientsmaynotbeagoodidea□ WeaknessesofcurrentattacksGANattackassumestheclassofdataisfromonesingleparticipantDLG/iDLGworkwithsecond-ordergradientmethod(expensive)andsmallminibatch-gradients(B=8)□ Vulnerabilitytofreeriders:pretendtohavedatabutnot.□ WeaknessofCurrentPrivacy-preservingTechniquesSecureaggregationismorevulnerabletopoisoningattacks