Texture-Synthesis-by-Non-parametric-Sampling：通过非参数化采样的纹理合成-课件

1、Texture Synthesis by Non-parametric Sampling Alexei Efros and Thomas LeungUC BerkeleyTexture Synthesis by Non-paraGoal of Texture SynthesisGiven a finite sample of some texture, the goal is to synthesize other samples from that same texture. The sample needs to be large enoughTrue (infinite) texture

2、SYNTHESISgenerated imageinput imageGoal of Texture SynthesisGivenThe ChallengeTexture analysis: how to capture the essence of texture? Need to model the whole spectrum: from repeated to stochastic textureThis problem is at intersection of vision, graphics, statistics, and image compressionrepeatedst

3、ochasticBoth?The ChallengeTexture analysis:Some Previous Workmulti-scale filter response histogram matching Heeger and Bergen,95sampling from conditional distribution over multiple scales DeBonet,97filter histograms with Gibbs sampling Zhu et al,98matching 1st and 2nd order properties of wavelet coe

4、fficients Simoncelli and Portilla,98N-gram language model Shannon,48clustering pixel neighbourhood densities Popat and Picard,93Some Previous Workmulti-scale Our ApproachOur goals:preserve local structuremodel wide range of real texturesability to do constrained synthesisOur method:Texture is “grown

5、” one pixel at a timeconditional pdf of pixel given its neighbors synthesized thus far is computed directly from the sample imageOur ApproachOur goals:Motivation from LanguageShannon,48 proposed a way to generate English-looking text using N-grams:Assume a generalized Markov modelUse a large text to

6、 compute probability distributions of each letter given N-1 previous letters precompute or sample randomlyStarting from a seed repeatedly sample this Markov chain to generate new lettersOne can use whole words instead of letters too:WE NEEDTOEATCAKEMotivation from LanguageShannMark V. Shaney (Bell L

7、abs)Results (using alt.singles corpus): “As Ive commented before, really relating to someone involves standing next to impossible.”One morning I shot an elephant in my arms and kissed him.”I spent an interesting evening recently with a grain of saltNotice how well local structure is preserved!Now le

8、ts try this in 2D.Mark V. Shaney (Bell Labs)ResuSynthesizing One PixelInfinite sample imageGenerated imageAssuming Markov property, what is conditional probability distribution of p, given the neighbourhood window?Instead of constructing a model, lets directly search the input image for all such nei

9、ghbourhoods to produce a histogram for p To synthesize p, just pick one match at randomSAMPLEpSynthesizing One PixelInfiniteReally Synthesizing One Pixelfinite sample imageGenerated imagepHowever, since our sample image is finite, an exact neighbourhood match might not be presentSo we find the best

10、match using SSD error (weighted by a Gaussian to emphasize local structure), and take all samples within some distance from that matchSAMPLEReally Synthesizing One PixelfGrowing TextureStarting from the initial configuration, we “grow” the texture one pixel at a timeThe size of the neighbourhood win

11、dow is a parameter that specifies how stochastic the user believes this texture to beTo grow from scratch, we use a random 3x3 patch from input image as seedGrowing TextureStarting from tSome DetailsGrowing is in “onion skin” orderWithin each “layer”, pixels with most neighbors are synthesized first

12、If no close match can be found, the pixel is not synthesized until the endUsing Gaussian-weighted SSD is very important to make sure the new pixel agrees with its closest neighborsApproximates reduction to a smaller neighborhood window if data is too sparseSome DetailsGrowing is in “oniRandomness Pa

13、rameterRandomness ParameterMore Synthesis ResultsIncreasing window sizeMore Synthesis ResultsIncreasiBrodatz Resultsaluminum wirereptile skinBrodatz Resultsaluminum wirereMore Brodatz Resultsfrench canvasrafia weaveMore Brodatz Resultsfrench canMore ResultswoodgraniteMore ResultswoodgraniteMore Resu

14、ltswhite breadbrick wallMore Resultswhite breadbrick wConstrained SynthesisConstrained SynthesisVisual ComparisonDeBonet, 97Our approachSimple tilingSynthetic tilabletextureVisual ComparisonDeBonet, 97Failure CasesGrowing garbage Verbatim copyingFailure CasesGrowing garbage VHomage to ShannonHomage

15、to ShannonConstrained Text SynthesisConstrained Text SynthesisApplicationsOcclusion fill-infor 3D reconstructionregion-based image and video compressiona small sample of textured region is storedTexturing non-developable objectsgrowing texture directly on surfaceMotion synthesisApplicationsOcclusion

16、 fill-inTexturing a sphere2D3DSample imageTexturing a sphere2D3DSample iImage ExtrapolationImage ExtrapolationSummaryAdvantages:conceptually simplemodels a wide range of real-world texturesnaturally does hole-fillingDisadvantages:its greedyits slow its a heuristicNot an answer to texture analysis, b

17、ut hopefully some inspiration!SummaryAdvantages:AcknowledgmentsThanks to:Alex BergElizaveta LevinaJitendra MalikYair WeissFunding agenciesNSF Graduate FellowshipBerkeley FellowshipONR MURICalifornia MIRCOAcknowledgmentsThanks to:FundiTexture-Synthesis-by-Non-parametric-Sampling：通过非参数化采样的纹理合成-课件Texture-Synthesis-by-Non-par