数据与模型安全 课件 第4周：对抗样本检测

Adversarial

Example

Detection姜育刚，马兴军，吴祖煊Recap:

week

3

1.

Adversarial

Examples

2.

Adversarial

Attacks

3.

Adversarial

Vulnerability

UnderstandingIn-class

Adversarial

Attack

Competitionhttps://codalab.lisn.upsaclay.fr/competitions/15669?secret_key=77cb8986-d5bd-4009-82f0-7dde2e819ff8

In-class

Adversarial

Attack

CompetitionIn-class

Adversarial

Attack

CompetitionAdversarial

attack

competition（account

for

30%）必须使用学校邮箱注册比赛（否则无成绩）比赛时间：Phase

1：10月1号–

10月28号Phase

2：评估阶段，学生不参与没卡的同学可以使用Google

Colab：/

按排名算分：第一名30分最后一名15分Adversarial

Example

Detection

(AED)A

binary

classification

problem:

clean

(y=0)

or

adv

(y=1)?An

anomaly

detection

problem:

benign

(y=0)

or

abnormal

(y=1)?

Principles

for

AEDAll

binary

classification

methods

can

be

applied

for

AEDPrinciples

for

AEDAll

anomaly

detection

methods

can

be

applied

for

AEDPrinciples

for

AEDUse

as

much

information

as

you

canInput

statisticsManual

featuresTraining

dataAttention

mapTransformationMixupDenoising…ActivationsDeep

featuresProbabilitiesLogitsGradientsLoss

landscapeUncertainty…Principles

for

AEDLeverage

unique

characteristics

of

adversarial

examplesTwinsStrangersExtremely

close

to

the

clean

sampleFar

away

in

predictionPrinciples

for

AEDBuild

detectors

based

on

existing

understandingsHigh

dimensional

pocketsLocal

linearityTilting

boundaryPrinciples

for

AEDIt’s

is

still

feature

engineering!Challenges

in

AEDThe

diversity

of

adversarial

examples

used

for

training

the

detectors

determine

the

detection

performanceDetectors

are

also

machine

learning

models:

they

are

also

vulnerable

to

adversarial

attacks

The

detectors

need

to

detect

both

existing

and

unknown

attacksThe

detectors

need

to

be

robust

to

adaptive

attacksExisting

MethodsSecondary

Classification

Methods

(二级分类法)Principle

Component

Analysis

(主成分分析法，PCA)Distribution

Detection

Methods

(分布检测法)Prediction

Inconsistency

(预测不一致性)Reconstruction

Inconsistency

(重建不一致性)Trapping

Based

Detection

(诱捕检测法)Existing

MethodsSecondary

Classification

Methods

(二级分类法)Principle

Component

Analysis

(主成分分析法，PCA)Distribution

Detection

Methods

(分布检测法)Prediction

Inconsistency

(预测不一致性)Reconstruction

Inconsistency

(重建不一致性)Trapping

Based

Detection

(诱捕检测法)Secondary

Classification

MethodsTake

adversarial

examples

as

a

new

classAdversarialRetraining

(对抗重训练)Grosse

et

al.

Onthe(Statistical)DetectionofAdversarialExamples,

arXiv:1702.06280Secondary

Classification

MethodsClean

samples

as

class

0,

adversarial

as

class

1AdversarialClassification

(对抗分类)Gong

et

al.

Adversarialandcleandataarenottwins,

arXiv:1704.04960Secondary

Classification

MethodsTraining

a

detector

for

each

intermediate

layerCascade

Classifiers

(级联分类器)Metzen,JanHendrik,etal."Ondetectingadversarialperturbations."

arXivpreprintarXiv:1702.04267

(2017).Existing

MethodsSecondary

Classification

Methods

(二级分类法)Principle

Component

Analysis

(主成分分析法，PCA)Distribution

Detection

Methods

(分布检测法)Prediction

Inconsistency

(预测不一致性)Reconstruction

Inconsistency

(重建不一致性)Trapping

Based

Detection

(诱捕检测法)Principle

Component

Analysis

(PCA)The

last

few

components

differentiate

adversarial

examplesHendrycks,Dan,andKevinGimpel.“Earlymethodsfordetectingadversarialimages.”

arXiv:1608.00530

(2016);

Carlini

and

Wagner."Adversarialexamplesarenoteasilydetected:Bypassingtendetectionmethods."

AISec.2017.Blue:

a

clean

sampleYellow:

an

adv

exampleAn

artifact

caused

by

the

black

backgroundDimensionality

ReductionBhagoji,ArjunNitin,DanielCullina,andPrateekMittal."Dimensionalityreductionasadefenseagainstevasionattacksonmachinelearningclassifiers."arXiv:1704.02654

2.1(2017).Train

on

PCA

reduced

dataExisting

MethodsSecondary

Classification

Methods

(二级分类法)Principle

Component

Analysis

(主成分分析法，PCA)Distribution

Detection

Methods

(分布检测法)Prediction

Inconsistency

(预测不一致性)Reconstruction

Inconsistency

(重建不一致性)Trapping

Based

Detection

(诱捕检测法)Distribution

DetectionGrosse

et

al.

Onthe(Statistical)DetectionofAdversarialExamples,

arXiv:1702.06280MaximumMeanDiscrepancy

(MMD)Two

datasets:

Distribution

DetectionFeinman,Reuben,etal."Detectingadversarialsamplesfromartifacts."

arXivpreprintarXiv:1703.00410

(2017).KernelDensityEstimation

(KDE)Adversarial

examples

are

in

low

density

spaceDistribution

DetectionFeinman,Reuben,etal."Detectingadversarialsamplesfromartifacts."

arXivpreprintarXiv:1703.00410

(2017).KernelDensityEstimation

(KDE)Adversarial

examples

are

in

low

density

space

Bypassing

10

Detection

MethodsAdversarialExamplesAreNotEasilyDetected:BypassingTenDetectionMethods.

Carlini

and

Wagner,

AISec

2017.Local

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018Definition(LocalIntrinsicDimensionality)AdversarialexamplesareinhighdimensionalsubspacesLocal

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018AdversarialSubspacesandExpansionDimension:

Local

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018Estimatinglocalintrinsicdimensionality.Amsaleg

et

al.KDD

2015EstimationofLID:

Hill(MLE)estimator(Hill1975,Amsalegetal.2015):BasedonExtremeValueTheory:Nearestneighbordistancesareextremeevents.LowertaildistributionfollowsGeneralizedParetoDistribution(GPD).

Local

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018InterpretationofLIDforAdversarialSubspaces:LIDdirectlymeasuresexpansionrateoflocaldistancedistributions.Theexpansionofadversarialsubspaceishigherthannormaldatasubspace.LIDassessesthespace-fillingcapabilityofthesubspace,basedonthedistancedistributionoftheexampletoitsneighbors.Local

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018LID

of

adversarial

examples

(red)

are

higherLID

at

deeper

layers

are

more

differentiableLocal

Intrinsic

Dimensionality

(LID)Local

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018Experiments&Results:DatasetFeatureFGMBIM-aBIM-bJSMAOptMNISTKD78.1298.1498.6168.7795.15BU32.3791.5525.4688.7471.30LID96.8999.6099.8392.2499.24CIFAR-10KD64.9268.3898.7085.7791.35BU70.5381.6097.3287.3691.39LID82.3882.5199.7895.8798.94SVHNKD70.3977.1899.5786.4687.41BU86.7884.0786.9391.3387.13LID97.6187.5599.7295.0797.60Local

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018Experiments&Results:Train\TestattackFGMBIM-aBIM-bJSMAOptFGSMKD64.9269.1589.7185.7291.22BU70.5381.672.6586.7991.27LID82.3882.3091.6189.9393.32Detectors

trained

on

simple

attacks

FGSM

can

detect

complex

attacksAn

Improved

Detector

of

LID/pdf/2212.06776.pdf

An

Improved

Detector

of

LID/pdf/2212.06776.pdfMahalanobisDistance

(MD)Mahalanobis,PrasantaChandra."Onthegeneralizeddistanceinstatistics."NationalInstituteofScienceofIndia,1936.

The

MD

of

between

two

data

points:MahalanobisDistance

(MD)Leeetal.“Asimpleunifiedframeworkfordetectingout-of-distributionsamplesandadversarialattacks.”

NeurIPS

2018.

MahalanobisDistance

(MD)Leeetal.“Asimpleunifiedframeworkfordetectingout-of-distributionsamplesandadversarialattacks.”

NeurIPS

2018.MahalanobisDistance

(MD)Leeetal.“Asimpleunifiedframeworkfordetectingout-of-distributionsamplesandadversarialattacks.”

NeurIPS

2018.Experiments&Results:Existing

MethodsSecondary

Classification

Methods

(二级分类法)Principle

Component

Analysis

(主成分分析法，PCA)Distribution

Detection

Methods

(分布检测法)Prediction

Inconsistency

(预测不一致性)Reconstruction

Inconsistency

(重建不一致性)Trapping

Based

Detection

(诱捕检测法)Bayes

UncertaintyBayesianUncertainty(BU)

Feinman,Reuben,etal."Detectingadversarialsamplesfromartifacts."

arXivpreprintarXiv:1703.00410

(2017).Feature

SqueezingXu

et

al."Featuresqueezing:Detectingadversarialexamplesindeepneuralnetworks."

arXiv:1704.01155

(2017).Bit

depth

reductionSqueezing

clean

and

adv

examplesReducing

input

dimensionality

improves

robustnessThe

prediction

inconsistency

before

and

after

squeezing

can

detect

advsRandom

TransformationTian

et

al."Detectingadversarialexamplesthroughimagetransformation."

AAAI2018.The

prediction

of

advs

will

change

after

random

transformationsLog-OddsRoth

et

al.“Theoddsareodd:Astatisticaltestfordetectingadversarialexamples.”

ICML2019.Add

random

noise

to

the

input

Log-OddsHuetal.“Anewdefenseagainstadversarialimages:Turningaweaknessintoastrength.”

NeurIPS

2019.原则1：对抗样本的梯度更均匀原则2：对抗样本难以被攻击第二次测试准则1：随机噪声不会改变预测结果测试准则1：再次攻击需要更多的扰动Existing

MethodsSecondary

Classification

Methods

(二级分类法)Principle

Component

Analysis

(主成分分析法，PCA)Distribution

Detection

Methods

(分布检测法)Prediction

Inconsistency

(预测不一致性)Reconstruction