扩散模型医学图像分析 综述

1

/amirhossein-kz/Awesome-Diffusion-Models-in-Medical-Imaging

arXiv:2211.07804v1[eess.IV]14Nov2022

DIFFUSIONMODELSFORMEDICALIMAGEANALYSIS:ACOMPREHENSIVESURVEY

AmirhosseinKazerouni

SchoolofElectricalEngineeringIranUniversityofScienceandTechnology

Tehran,Iran

amirhossein477@

EhsanKhodapanahAghdam

DepartmentofElectricalEngineering

ShahidBeheshtiUniversity

Tehran,Iran

ehsan.khpaghdam@

MoeinHeidari

SchoolofElectricalEngineeringIranUniversityofScienceandTechnology

Tehran,Iranmoein_heidari@elec.iust.ac.ir

RezaAzad

InstituteofImagingandComputerVision

RWTHAachenUniversity

Aachen,Germany

azad@lfb.rwth-aachen.de

MohsenFayyaz

Microsoft

Berlin,Germanymohsenfayyaz@

IlkerHacihaliloglu

DepartmentofRadiology

DepartmentofMedicine

UniversityofBritishColumbia

BritishColumbia,Canada

ilker.hacihaliloglu@ubc.ca

DoritMerhof

FacultyofInformaticsandDataScience

UniversityofRegensburg

Regensburg,Germany

dorit.merhof@ur.de

ABSTRACT

Denoisingdiffusionmodels,aclassofgenerativemodels,havegarneredimmenseinterestlatelyinvariousdeep-learningproblems.AdiffusionprobabilisticmodeldefinesaforwarddiffusionstagewheretheinputdataisgraduallyperturbedoverseveralstepsbyaddingGaussiannoiseandthenlearnstoreversethediffusionprocesstoretrievethedesirednoise-freedatafromnoisydatasamples.Diffusionmodelsarewidelyappreciatedfortheirstrongmodecoverageandqualityofthegeneratedsamplesinspiteoftheirknowncomputationalburdens.Capitalizingontheadvancesincomputervision,thefieldofmedicalimaginghasalsoobservedagrowinginterestindiffusionmodels.Withtheaimofhelpingtheresearchernavigatethisprofusion,thissurveyintendstoprovideacomprehensiveoverviewofdiffusionmodelsinthedisciplineofmedicalimageanalysis.Specifically,westartwithanintroductiontothesolidtheoreticalfoundationandfundamentalconceptsbehinddiffusionmodelsandthethreegenericdiffusionmodelingframeworks,namely,diffusionprobabilisticmodels,noise-conditionedscorenetworks,andstochasticdifferentialequations.Then,weprovideasystematictaxonomyofdiffusionmodelsinthemedicaldomainandproposeamulti-perspectivecategorizationbasedontheirapplication,imagingmodality,organofinterest,andalgorithms.Tothisend,wecoverextensiveapplicationsofdiffusionmodelsinthemedicaldomain,includingsegmentation,anomalydetection,image-to-imagetranslation,2/3Dgeneration,reconstruction,denoising,andothermedically-relatedchallenges.Furthermore,weemphasizethepracticalusecaseofsomeselectedapproaches,andthenwediscussthelimitationsofthediffusionmodelsinthemedicaldomainandproposeseveraldirectionstofulfillthedemandsofthisfield.Finally,wegathertheoverviewedstudieswiththeiravailableopen-sourceimplementationsatour

GitHub1

.

Weaimtoupdatetherelevantlatestpaperswithinitregularly.

KeywordsGenerativemodels.Diffusionmodels.Denoisingdiffusionmodels.Noiseconditionedscorenetworks.Score-basedmodels.Medicalimaging.Medicalapplications.Survey

2

Generator

True/

False

Decoder

Generator

SoftMax

Flow

DiffusionModelsforMedicalImageAnalysis:AComprehensiveSurvey

Discriminator

(a)

GAN

Encoder

(c)VAE

Discriminator

(b)Energy-basedModels

Inverse

(d)Flow-basedmodels

(e)

Diffusionmodels

Figure1:Inthisfigure,Wedisplaytherevolutionofgenerativemodelsandtheinsightsbehindthem.(

a

)GeneralAdversarialNetwork(GAN)[

1

]isanend-to-endpipelinethattrainsthegeneratorinanadversarialmannertogeneratesamplesthatthediscriminatoriscapableofdistinguishingfromtherealdatasample.(

b

)Energy-basedModel(EBM)[

2

],alsoknownasnon-normalizedprobabilisticmodels,trainsinthesamewayasGANswithtwomajormodifications.First,thediscriminatorlearnsaproperenergy-basedfunctionthatmapsthedatasampletoadistributionspace.Second,thegeneratorutilizesapriorinputtoenhancethesamplegenerationperformance.(

c

)VariationalAutoEncoder(VAE)[

3

]isastandalonenetworkthatfollowsaprojectionfromadatasampletoalow-dimensionallatentspacebytheencoderandgeneratesbysamplingfromitviaadecoderpath.(

d

)Normalizingflow(NF)[

4

]utilizesaninvertibleflowfunctiontotransforminputtolatentspaceandgeneratesampleswiththeinverseflowfunction.(

e

)DiffusionModelsinterminglethenoisewiththeinputinsuccessivestepsuntilitbecomesanoisedistributionbeforeapplyingareverseprocesstoneutralizethenoiseadditionineachstepinthesamplingprocedure.

1Introduction

Generativemodelingusingneuralnetworkshasbeenaformidableforceinthepastdecadeofdeeplearning.Sincetheiremergence,generativemodelshavemadeatremendousimpactinvariousdomainsrangingfromimages[

5

,

6

],audio[

7

,

8

],totext,[

9

]andpointclouds[

10

].Fromaprobabilisticmodelingviewpoint,thekeydefiningcharacteristicofagenerativemodelisthatitistrainedinsuchawaysothatitssamples~pθ-(comefromthesamedistributionasthetrainingdatadistribution,z~pd-z(.Thepioneeringenergy-basedmodelsachievethisbydefininganunnormalizedprobabilitydensityoverastatespace;however,thesemethodsrequireMarkovChainMonteCarlo(MCMC)samplingduringbothtrainingandinference,whichisaslowiterativeprocess[

11

].Followingtheunprecedentedsurgeofavailabledatasets,aswellasadvancesingeneraldeeplearningarchitectures,therehasbeenarevolutionaryparadigmshiftingenerativemodeling.Specifically,thethreemainstreamgenerativeframeworksinclude,namely,Generativeadversarialnetworks(GANs)[

1

],variationalautoencoders(VAEs)[

12

,

3

],andnormalizingflows[

13

](see

Figure1

).Generativemodelstypicallyentailkeyrequirementstobeadoptedinreal-worldproblems.Theserequirementsinclude(i)high-qualitysampling,(ii)modecoverageandsamplediversity,and(iii)fastexecutiontimeandcomputationallyinexpensivesampling[

14

](see

Figure2

).Generativemodelsoftenmakeaccommodationsbetweenthesecriteria.Specifically,GANsarecapableofgeneratinghigh-qualitysamplesrapidly,buttheyhavepoormodecoverageandarepronetolacksamplingdiversity.Conversely,VAEsandnormalizingflowssufferfromtheintrinsicpropertyoflowsamplequalitydespitebeingwitnessedincoveringdatamodes.GANsconsistoftwomodels:ageneratorandacritic(discriminator),whichcompetewitheachotherwhilesimultaneouslymakingeachotherstronger.Thegeneratortriestocapturethedistributionoftrueexamples,whilethediscriminator,whichistypicallyabinaryclassifier,estimatestheprobabilityofagivensamplecomingfromtherealdataset.Itworksasacriticandisoptimizedtorecognizethesyntheticsamplesfromtherealones.AcommonconcernwithGANsistheirtrainingdynamics

3

FastSampling

DiffusionModelsforMedicalImageAnalysis:AComprehensiveSurvey

whichhavebeenrecognizedasbeingunstable,resultingindeficienciessuchasmodecollapse,vanishinggradients,andconvergence[

15

].Therefore,animmenseinteresthasalsoinfluencedtheresearchdirectionofGANstoproposemoreefficientvariants[

16

,

17

].Variationalauto-encoders(VAEs)optimizethelog-likelihoodofthedatabymaximizingtheevidencelowerbound(ELBO).Despitetheremarkableachievements,thebehaviorofVariationalAutoencodersisstillfarfromsatisfactoryduetosometheoreticalandpracticalchallengessuchasbalancingissue[

18

]andvariablecollapsephenomenon[

19

].Aflow-basedgenerativemodelisconstructedbyasequenceofinvertibletransformations.Specifically,anormalizingflowtransformsasimpledistributionintoacomplexonebyapplyingasequenceofinvertibletransformationfunctionswhereonecanobtainthedesiredprobabilitydistributionforthefinaltargetvariableusingachangeofvariablestheorem.UnlikeGANsandVAEs,thesemodelsexplicitlylearnthedatadistribution;therefore,theirlossfunctionissimplythenegativelog-likelihood[

20

].Despitebeingfeasiblydesigned,thesegenerativemodelshavetheirspecificdrawbacks.SincetheLikelihood-basedmethodhastoconstructanormalizedprobabilitymodel,aspecifictypeofarchitecturemustbeused(AutoregressiveModel,FlowModel),orinthecaseofVAE,analternativeLosssuchasELBOisnotcalculateddirectlyforthegeneratedprobabilitydistribution.Incontrast,thelearningprocessofGANsisinherentlyunstableduetothenatureoftheadversariallossoftheGAN.Recently,diffusionmodels[

21

,

22

]haveemergedaspowerfulgenerativemodels,showcasingoneoftheleadingtopicsincomputervisionsothatresearchersandpractitionersalikemayfinditchallengingtokeeppacewiththerateofinnovation.Fundamentally,diffusionmodelsworkbydestroyingtrainingdatathroughthesuccessiveadditionofGaussiannoiseandthenlearningtorecoverthedatabyreversingthisnoisingprocess.

Todate,diffusionmodelshavebeenfoundtobeusefulinawideva-rietyofareas,rangingfromgenerativemodelingtaskssuchasimagegeneration[

23

],imagesuper-resolution[

24

],imageinpainting[

25

]todiscriminativetaskssuchasimagesegmentation[

26

],classification[

27

],andanomalydetection[

28

].Recently,themedicalimagingcommu-nityhaswitnessedexponentialgrowthinthenumberofdiffusion-basedtechniques(see

Figure4

).Asshownin

Figure4

,awealthofresearchisdedicatedtotheapplicationsofdiffusionmodelsindiversemedicalimagingscenarios.Thus,asurveyoftheexistingliteratureisbothbeneficialforthecommunityandquitetimely.Tothisend,thissurveysetsouttoprovideacomprehensiveoverviewoftherecentadvancesmadeandprovidesaholisticoverviewofthisclassofmodelsinmedicalimaging.Athoroughsearchoftherelevantliteraturerevealedthatwearethefirsttocoverthediffusion-basedmodelsexploitedinthemedicaldomain.Wehopethisworkwillpointoutnewpaths,providearoadmapforresearchers,andinspirefurtherinterestinthevisioncommunitytoleveragethepotentialofdiffusionmodelsinthemedicaldomain.Ourmajorcontributionsinclude:

●Thisisthefirstsurveypaperthatcomprehensivelycoversapplicationsofdiffusionmodelsinthemedicalimagingdomain.Specifically,wepresentacomprehensiveoverviewofallavailablerelevantpapers(untilOctober2022).

●Wedeviseamulti-perspectivecategorizationofdiffusionmodelsinthemedicalcommunity,providingasystematicaltaxonomyofresearchindiffusionmodelsandtheirapplications.Wedividetheexistingdiffusionmodelsintothreecategories:denoisingdiffusionprobabilisticmodels,noise-conditionedscorenetworks,andstochasticdifferentialequations.Moreover,wegrouptheapplicationsofdiffusionmodelsintosevencategories:anomalydetection,denoising,reconstruction,segmentation,image-to-imagetranslation,imagegeneration,andotherapplications.

Denoising

Diffusion

Models

Generative

AdversarialNetworks

High

Quality

Samples

Mode

Coverage/Diversity

,

VariationalAutoencoders

NormalizingFlows

Figure2:Generativelearningtrial[

14

].De-spitetheabilityofGANstoquicklygeneratehigh-fidelitysamples,theirmodecoverageislim-ited.Inaddition,VAEsandnormalizingflowshavebeenrevealedtohaveagreatdealofdiver-sity;however,theygenerallyhavepoorsamplingquality.Diffusionmodelshaveemergedtocom-pensateforthedeficiencyofVAEsandGANsbyshowingadequatemodecoverageandhigh-qualitysampling.Nevertheless,duetotheiritera-tivenature,whichcausesaslowsamplingprocess,theyarepracticallyexpensiveandrequiremoreimprovement.

●Wedonotrestrictourattentiontoapplicationandprovideanewtaxonomy(see

Figure4

)whereeachpaperisbroadlyclassifiedaccordingtotheproposedalgorithmalongwiththeorganconcernedandimagingmodality,respectively.

●Finally,wediscussthechallengesandopenissuesandidentifythenewtrendsraisingopenquestionsaboutthefuturedevelopmentofdiffusionmodelsinthemedicaldomaininbothalgorithmsandapplications.

PaperOrganization.In

Section2.1

,wepresentadetailedoverviewoftheconceptsandtheoreticalfoundationsbehinddiffusionmodels,coveringthreesub-categorieswithasimilarbaselinedefinition.

Sections3.1

to

3.7

comprehensivelycovertheapplicationsofdiffusionmodelsinseveralmedicalimagingtasks,asshownin

Figure3

,whereas

Section3.8

providesacomparativetask-specificoverviewofdifferentliteraturework.Weconcludethissurveybypinpointingfuturedirectionsandopenchallengesfacingdiffusionmodelsinthemedicalimagingdomainin

Section4

.

4

Numberofarticles

DiffusionModelsforMedicalImageAnalysis:AComprehensiveSurvey

econstructionImageGeneration

│egmentationOtherApplications

Image-to-ImageTranslation

11%

8%

11%

34%

16%

13%

AnomalyDetectionDenoising

8%

(a)

OpticalbasedX-raybased

63%

2%

9%

MRI

NuclearImaging

26%

(b)

16

14

12

10

8

6

4

2

0

AnomalyDetection

Denoising

Segmentation

Image-to-ImageTranslation

Reconstruction

ImageGeneration

OtherApplications

2021-Q3

2021-Q4

2022-Q1Quarters

2022-Q2

2022-Q3

(c)

Figure3:Thediagram(a)showstherelativeproportionofpublishedpaperscategorizedaccordingtotheirapplicationand(b)accordingtotheirimagingmodalities.(c)indicatesthenumberofdiffusion-basedresearchpaperspublishedinthemedicaldomain.Thegrowthrateperyearrevealstheimportanceofdiffusionmodelsforfuturework.

2Taxonomy

Generativeapproacheshaveundergonesignificantadvancesinmedicalimagingoverthepastfewdecades.Therefore,therehavebeennumeroussurveypaperspublishedondeepgenerativemodelsformedicalimaging[

29

,

30

,

31

].Someofthesepapersfocusonaspecificapplicationonly,whileothersconcentrateonaspecificimagemodality.Thereis,however,alackofcomprehensivesurveysontheapplicationsofdiffusionmodelsinmedicalimaging.Tothisend,inthissurvey,wedeviseamulti-perspectivevisionofdiffusionmodelsinwhichwediscussexistingliteraturebasedontheirapplicationsinthemedicaldomain.Nonetheless,wedonotrestrictourinteresttotheapplicationsbutdescribetheunderlyingworkingprinciples,theorgan,andtheimagingmodalityoftheproposedmethod.Wefurtherdiscusshowthisadditionalinformationcanhelpresearchersattempttoconsolidatetheliteratureacrossthespectrum.Abriefoutlookofourpaperisdepictedin

Figure4

.

2.1Algorithm

Thereareatleastthreesub-categoriesofdiffusionmodelsthatcomplywiththebaselinedefinitionofdiffusionmodels[

60

].First,therearedenoisingdiffusionprobabilisticmodels(DDPMs)[

21

,

22

],whichareaclassoflatentvariablemodelsinspiredbyconsiderationsfromnonequilibriumthermodynamics.Thesecondsub-categoryisrepresentedbynoise-conditionedscorenetworks(NCSNs)[

61

],whichisfundamentallybasedonestimatingthederivativeofthelogdensityfunctionoftheperturbeddatadistributionatdifferentnoiselevels.Stochasticdifferentialequations(SDEs)[

62

]formthethirdsub-category,whichencapsulatespreviousapproachesandcanbeviewedasageneralizationoverDDPMsandNCSNs.Wehereinafterelaborateonthedetailsofeachcategory.

2.1.1DenoisingDiffusionProbabilisticModels(DDPMs)

ForwardProcess.DDPMdefinestheforwarddiffusionprocessasaMarkovChainwhereGaussiannoiseisaddedinsuccessivestepstoobtainasetofnoisysamples.Considerg-z0(astheuncorrupted(original)datadistribution.Givenadatasamplez0华g-z0(,aforwardnoisingprocesspwhichproduceslatentz1throughzTbyaddingGaussiannoiseattimetisdefinedasfollows:

g-zt|zt—1(=八╱zt﹔^1_8t.zt—1,8t.t、,Ate{1,...,7}(1)

where7and81,...,8Te[〇,1(representthenumberofdiffusionstepsandthevariancescheduleacrossdiffusion

stepsrespectively.tistheidentitymatrixand八-z﹔m,7(representsthenormaldistributionofmeanmandcovariance7.

Consideringat=1_8tandt=!=0as,onecandirectlysampleanarbitrarystepofthenoisedlatentconditionedon

theinputz0asfollows:

g-xt|x0(=N╱xt﹔^tx0,-1_t(t、

(2)

xt=^tx0十^1_al9(3)

5

Modality:PET

Modality:CT/MRI

Organ:Brain

1.MT-Diffusion

Organ:Multi-organ

Algorithm:DDPM

Algorithm:DDPM/SDE

Modality:OCT

Modality:MRI

16.DenoOCT-DDPM

Organ:Eye

2.UMM-CSGM

Organ:Brain

Algorithm:DDPM

Algorithm:SDE

Modality:MRI

Organ:Brain/Abdominal

3.SynDiff

Organ:Brain

Algorithm:DDPM

Modality:Microscopic

Modality:CT/MRI

Organ:Giloma

4.SIM-SGM

Organ:Multi-organ/Brain

Algorithm:SDE

Modality:MRI

19.DISPR

Organ:BloodCell

5.MC-DDPM

Organ:Knee

Algorithm:DDPM

Modality:MRI

Modality:MRI

Organ:Brain

6.Score-MRI

Organ:Knee

Algorithm:DDPM

Modality:PET

Modality:MRI

Organ:Heart

7.DiffuseRecon

Organ:Knee

Algorithm:DDPM

Algorithm:DDPM

Reconstruction

Modality:MRI

Modality:MRI

8.AdaDiff

Organ:Brain

Organ:Brain

Algorithm:DDPM

Modality:MRI

Organ:Brain

9.MRI-DDMC

Organ:Brain

Algorithm:DDPM

Algorithm:SDE

Modality:MRI

Modality:MRI

Organ:Knee/Brain

10.Self-Score

Organ:Brain

Algorithm:DDPM

Modality:MRI/X-ray

Organ:Multi-organ

11.MCG

Organ:Brain/Chest

Algorithm:DDPM

Modality:Microscopic

Organ:Brain

Organ:Skin

26.Multi-scaleDDAM

Algorithm:DDPM

Modality:X-ray

Segmentation

13.DARL

Organ:Heart/Eye

Organ:Aminoacids

Algorithm:DDPM

Algorithm:DDPM

Modality:MRI

Modality:MRI

Organ:Brain

14.IISE

Organ:Knee/Liver

Algorithm:DDPM

Organ:Brain

DiffusionModelsforMedicalImageAnalysis:AComprehensiveSurvey

15.PET-DDPM

Denoising

Image-to-ImageTranslation

Modality:MRI/CT

17.BrainGen

Algorithm:DDPM

18.MFDPM

Algorithm:DDPM

Modality:Microscopic

ImageGeneration

Algorithm:DDPM

20.3D-DDPM

Algorithm:SDE

21.DDM

DiffusionModels

Taxonomy

22.AnoDDPM

Algorithm:DDPM

Modality:MRI

23.CDPM

AnomalyDetection

24.IITM-Diffusion

Algorithm:NCSN

Modality:CT

25.AnoDDIM

Algorithm:SDE

Modality:MRI

12.brainSPADE

Algorithm:DDPM

Modality:X-ray

crystallography

27.SMCDiff

OtherApplications

28.R2D2+

Algorithm:SDE

Modality:CT/MRI

29.BAnoDDPM

Algorithm:DDPM

Figure4:Theproposedtaxonomyfordiffusion-basedmedicalimageanalysisresearchisbuiltonsevensub-fields:I)Image-to-ImageTranslation,II)MedicalImageReconstruction,III)ImageSegmentation,IV)MedicalImageDenoising,V)ImageGeneration,VI)AnomalyDetectionandVII)multi-disciplinaryapplications,namedOtherApplications.Forthesakeofbrevity,weutilizetheprefixnumbersinthepaper’snameinascendingorderanddenotethereferenceforeachstudyas

follows:1.[

32

],2.[

33

],3.[

34

],4.[

35

],5.[

36

],6.[

37

],7.[

38

],8.[

39

],9.[

40

],10.[

41

],11.[

42

],12.[

43

],13.[

44

],14.[

45

],15.[

46

],16.[

47

],17.[

48

],18.[

49

],19.[

50

],20.[

51

],21.[

52

],22.[

53

],23.[

54

],24.[

55

],25.[

28

],26.[

56

],27.[

57

],28.[

58

],29.[

59

].

6

DiffusionModelsforMedicalImageAnalysis:AComprehensiveSurvey

ReverseProcess.Leveragingtheabovedefinitions,wecanapproximateareverseprocesstogetasamplefromg(z0).Tothisend,wecanparameterizethisreverseprocessbystartingatp(xT)=c(xT;φ,t)asfollows:

T

pθ(x0:T)=p(xT)(pθ(xt—1lxt)

t=1

pθ(xt—1lxt)=c(xt—1;mθ(xt,t),θ(xt,t))

(4)

(5)

Totrainthismodelsuchthatp(z0)learnsthetruedatadistributiong(z0),wecanoptimizethefollowingvariationalboundonnegativelog-likelihood:

E[_logpθ(x0)]<Bq┌_log┐

=Eq_logp(xT)_log(6)

=_LVL.B

Hoetal.[

22

]founditbetternottodirectlyparameterizemθ(zt,t)asaneuralnetwork,butinsteadtotrainamodel9θ(zt,t)topredict9.Hence,byreparameterizing

Equation(6)

,theyproposedasimplifiedobjectiveasfollows:

Lsimple=Et,x0,e)|9_9θ(zt,t)|2」(7)

wheretheauthorsdrawaconnectionbetweenthelossin

Equation(6)

togenerativescorenetworksinSongetal.[

61

].

2.1.2NoiseConditionedScoreNetworks(NCSNs)

Thescorefunctionofsomedatadistributionp(z)isdefinedasthegradientofthelogdensitywithrespecttotheinput,Vxlogp(z).Toestimatethisscorefunction,onecantrainasharedneuralnetworkwithscorematching.Specifically,thescorenetworks9isaneuralnetworkparameterizedby9,whichistrainedtoapproximatethescoreofp(z)(sθ(z)sVxlogp(z))byminimizingthefollowingobjective:

Ex~p(x)|sθ(z)_Vxlogp(z)|(8)

However,duetothecomputationalburdenofcalculatingVxlogp(z),scorematchingisnotscalabletodeepnetworksandhighdimensionaldata.Tomitigatethisproblem,theauthorsof[

61

]proposetoexploitdenoisingscorematching[

63

]andslicedscorematching[

64

].Moreover,Songetal.[

61

]highlightmajorchallengesthatpreventanaiveapplicationofscore-basedgenerativemodelinginrealdata.Thekeychallengeisthefactthattheestimatedscorefunctionsareinaccurateinlow-densityregionssincedataintherealworldtendtoconcentrateonlow-dimensionalmanifoldsembeddedinahigh-dimensionalspace(themanifoldhypothesis).TheauthorsdemonstratethattheseproblemscanbeaddressedbyperturbingthedatawithGaussiannoiseatdifferentscales,asitmakesthedatadistributionmoreamenabletoscore-basedgenerativemodeling.Theyproposetoestimatethescorecorrespondingtoallnoiselevelsbytrainingasinglenoise-conditionedscorenetwork(NCSN).TheyderiveVxlog(pσt(z))asVxtlogpσt(ztlz)=_xxbychoosingthenoisedistributiontobepσt(ztlz)=c╱zt;z,7.t、whereztisanoisedversionofz.Thus,foragivensequenceofGaussiannoisescales71<72<...<7T,

Equation(8)

canbewrittenas:

u(7t)Ep(x)Ext~pσt(xt|x)sθ(zt,7t)+(9)

whereu(7t)isaweightingfunction.Theinferenceisdoneusinganiterativeprocedurecalled"Langevindynamics"[

65

,

66

].LangevindynamicsdesignanMCMCproceduretosamplefromadistributionp(x)usingonlyitsscorefunctionVxlogp(x).Specifically,tomovefromarandomsamplex0~π(x)towardssamplesfromp(x),ititeratesthefollowing:

zi=zi—1+2Vxlogp(z)+′e.wi(10)

e

7

■x_f-x,t(■t十g-t(■w(11)

Figure5:TensynthetichistopathologyimagesgeneratedbyMFDPM[

49

].

wherewi~c-〇,t(,andi∈u1,...,N}.Whene二〇andN二o,xisamplesobtainedfromthisprocedureconvergetoasamplefromp-x(.Theauthorsof[

61

]proposeamodificationofthisalgorithmnomenclatureastheannealedLangevindynamicsalgorithmsincethenoisescale7idecreases(anneals)graduallyovertimetomitigatesomepitfallsandfailuremodesofscorematching[

67

].

2.1.3StochasticDifferentialEquations(SD