真实世界中的鲁棒目标检测简单高效的目标检测域泛化方法

TowardsRobustObjectDetectionInvarianttoReal-WorldDomainShifts(ICLR2023)

HKUST&MPIIÐZurich

QiFan,MattiaSegu,Yu-WingTai,FisherYu,Chi-KeungTang,

BerntSchiele,DengxinDai

DomainShifts&Generalization

DomainShifts

•Sameclassbutdifferentvisualstyles

SolvingDomainShifts

•DomainAdaptation

•Labeledsourcedomain

•Labeled/unlabeledtargetdomain

•Usebothdomainimagestotrainthemodeltogeneralizeonthetargetdomain

•DomainGeneralization

•Multiple/singlesourcedomain

•Onlyusethesourcedomaintotrainthemodeltogeneralizeontargetdomains

Real-worldDomainShifts

OurTarget

Trainamodel

ononesourcedomain

Applythemodel

ondiversetargetdomains

Real-WorldDomainShifts

ProblemAnalysis

DomainShiftsObservation

•Domainshiftsaremainlyreflectedbystyleshifts.

DomainShiftsObservation

•Domainshiftsaremainlyreflectedbystyleshifts.

•Wecantraindomain-invariantmodelswithdiversesynthesizeddomainstyles.

DomainShiftsObservation

•Domainshiftsaremainlyreflectedbystyleshifts.

•Wecantraindomain-invariantmodelswithdiversesynthesizeddomainstyles.

•Domainstylesareencodedbyfeaturechannelstatistics.

DomainShiftsObservation

•Domainshiftsaremainlyreflectedbystyleshifts.

•Wecantraindomain-invariantmodelswithdiversesynthesizeddomainstyles.

•Domainstylesareencodedbyfeaturechannelstatistics.

•Perturbingfeaturechannelstatisticscansynthesizenewstyles.

(TheResNetbackboneblock1)

PreviousClassificationDG

MethodsFails

ClassificationDGsourcedomain:PACS

DetectionDGsourcedomain:Cityscapes

High

Low

DomainStyleVariance

ImageContextDiversity

Low

High

PreviousClassificationDGMethodsFails

Trainamodel

ononesourcedomain

•Smalldomainstylevariancerestrictsfeature-leveldomainsynthesis.

•Largecontextdiversityrestrictsimage-leveldomainsynthesis.

ImageStyleMatters

x

μ,σ

Replacethefeaturechannelstatistics

μ!ew,σ!ew

ProblemAnalysis

•MixstyleandDSUaresuboptimalforrobustobjectdetectionwhentheinter-imagestylevarianceissmall.

ProblemAnalysis

•MixstyleandDSUaresuboptimalforrobustobjectdetectionwhentheinter-imagestylevarianceissmall.

OurSimpleMethod

NormalizationPerturbation

•AdaptiveInstanceNormalization(AdaIN)

•NormalizationPerturbation

NormalizationPerturbation

.NormalizationPerturbation:y=ax+(β−a)μc

x∈ℛBXCXHXWistheCNNfeatures.

μc∈ℛBXCisthechannelstatistics(mean),estimatedontheinputfeatures.{a,β}∈ℛBXCarerandomnoisesdrawnfromtheGaussiandistribution.

δisthenormalizedstatisticvarianceofthemini-batchofmultiplefeaturechannelstatistics.

NormalizationPerturbation

.NormalizationPerturbation:y=ax+(β−a)μc

xERBXCXHXWistheCNNfeatures.

μcERBXCisthechannelstatistics(mean),estimatedontheinputfeatures.fa,β}ERBXCarerandomnoisesdrawnfromtheGaussiandistribution.

δisthenormalizedstatisticvarianceofthemini-batchofmultiplefeaturechannelstatistics.

21

NormalizationPerturbation

.NormalizationPerturbation:y=ax+(β−a)μc

.NormalizationPerturbationPlus:y=ax+δ(β−a)μc

xERBXCXHXWistheCNNfeatures.

μcERBXCisthechannelstatistics(mean),estimatedontheinputfeatures.fa,β}ERBXCarerandomnoisesdrawnfromtheGaussiandistribution.

δisthenormalizedstatisticvarianceofthemini-batchofmultiplefeaturechannelstatistics.

22

NormalizationPerturbationPlus

•Motivation:somechannelssignificantlyvaryasthedomainchanges

NormalizationPerturbation

Advantages

EffectiveDomainBlending

HighContentFidelity

•Image-leveldomainsynthesis

•maydestroythecontentstructuresoftheoriginalimages

•stylesaredeterministicandlimited

•thestyleaugmentationisonlyperformedonthelow-dimensionalimagespace.

DiverseLatentStyles

BenefitsOtherMethods

AblationStudies

AblationStudies

ComparisonResults

RobustObjectDetection

UnsupervisedDomainAdaptiveObjectDetec