a multiband approach to unsupervised scale parameter selection for multiscale image segmentation

1、A multi-band approach to unsupervised scale parameter selectionfor multi-scale image segmentationJian Yanga,c, Peijun Lib, Yuhong HecaDepartment of Geography, University of Toronto, 100 St. George Street, Toronto, ON M5S 3G3, CanadabInstitute of Remote Sensing and GIS, School of Earth and Space Scie

2、nces, Peking University, Beijing 100871, ChinacDepartment of Geography, University of Toronto Mississauga, 3359 Mississauga Rd North, Mississauga, ON L5L 1C6, Canadaa r t i c l ei n f oArticle history:Received 20 March 2013Received in revised form 28 February 2014Accepted 8 April 2014Available onlin

3、e 10 May 2014Keywords:Object based image analysisMulti-scale segmentationAppropriate scale parameter selectionMulti-bandSpectral angleSegmentation evaluationa b s t r a c tImage segmentation is one of key steps in object based image analysis of very high resolution images.Selecting the appropriate s

4、cale parameter becomes a particularly important task in image segmentation.In this study, an unsupervised multi-band approach is proposed for scale parameter selection in themulti-scale image segmentation process, which uses spectral angle to measure the spectral homogeneityof segments. With the inc

5、reasing scale parameter, spectral homogeneity of segments decreases until theymatch the objects in the real world. The index of spectral homogeneity is thus used to determine multipleappropriate scale parameters. The performance of the proposed method is compared to a single-bandbased method through

6、 qualitative visual interpretation and quantitative discrepancy measures. Bothmethods are applied for segmenting two images: a QuickBird scene of an urban area within Beijing, Chinaand a Woldview-2 scene of a suburban area in Kashiwa, Japan. The proposed multi-band based segmen-tation scale paramete

7、r selection method outperforms the single-band based method with the better rec-ognition for diverse land cover objects in different urban landscapes.? 2014 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS) Published by ElsevierB.V. All rights reserved.1. IntroductionThe fide

8、lity provided by very high resolution (VHR) data fromIKONOS, QuickBird, GeoEye-1 and WorldView-2, has proven usefulfor numerous applications, such as impervious surface mapping(Lu et al., 2011; Lu and Weng, 2009; Yuan and Bauer, 2006) andtree crown delineation (Ardila et al., 2012; Mallinis et al.,

9、2008;Song et al., 2010). Furthermore, object based image analysis (OBIA)yields better accuracy compared to traditional pixel-based imageanalysis (Cleve et al., 2008; Yan et al., 2006; Yuan and Bauer,2006)whenhighwithin-classspectralvariabilityoccurs(Johansen et al., 2010). Of the first stage of OBIA

10、, the aforemen-tioned literature suggests that image segmentation is the mostinfluential on land cover object recognition. Consequently, severalimage segmentation algorithms have been proposed includingregion-growingsegmentation (Benz et al., 2004), watershedsegmentation (Li et al., 2010; Li and Xia

11、o, 2007; Wang et al.,2004), and mean-shift segmentation (Comaniciu and Meer, 2002).Moreover,manyimagesegmentationalgorithmshavebeenexpanded to consider objects at multiple scales, since groundobjects generally show multi-scale features in high resolution image(Blaschke, 2010; Bruzzone and Carlin, 20

12、06; Hay et al., 2003). Forinstance, at fine scales a grass field may have spectral variabilityor patchiness related to micro-moisture regimes or worn patchesdues to sporting events while at coarser scales the field in itsentirety stands out from the surrounding urban environment. Assuch, multi-scale

13、 image segmentation addresses some of the defi-ciencies associated with single scale segmentation in complex landcover environments (Akay and Aksoy, 2008; Carleer and Wolff,2006; De Roeck et al., 2009; Li et al., 2011; Tilton et al., 2012)In image segmentation, the appropriate scale parameter is not

14、readily apparent and is currently chosen by time consuming andsubjective trial-and-error (Meinel and Neubert, 2004; Zhang et al.,2008). In lieu of trail-and-error approaches, several scale parameterselection methods have been proposed typically utilizing measuresof the dissimilarity between a segmen

15、tation result and a referenceimage in which the optimal scale parameter is the best match to thereference image (Carleer et al., 2005; Chabrier et al., 2006; Liu et al.,2012; Mller et al., 2007; Neubert et al., 2008; Tong et al., 2012). Inaddition, Espindola et al. (2006) and Johnson and Xie (2011)s

16、elected the optimal segmentation scale parameter by assessing/10.1016/j.isprsjprs.2014.04.0080924-2716/? 2014 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS) Published by Elsevier B.V. All rights reserved.Corresponding author at: Department of Geography, Uni

17、versity of Toronto, 100St. George Street, Toronto, ON M5S 3G3, Canada. Tel.: +1 416 978 3375; fax: +1 416946 3886.E-mail address: (J. Yang).ISPRS Journal of Photogrammetry and Remote Sensing 94 (2014) 1324Contents lists available at ScienceDirectISPRS Journal of Photogrammetry and Remote Sensingjour

18、nal homepage: homogeneity using weighted variance of the NIRband and inter-segment heterogeneity using spatial autocorrela-tion (Global Morans I). Kim et al. (2008, 2009) identified the opti-malscaleparameterforsegmentingforeststandsthroughcomputing unweighted variance and Global Morans I. In contra

19、st,Dragut et al. (2010) developed an automatic single-band methodfor defining meaningful segmentation scale parameters across arange of different image types and landscapes. Further, a few stud-ies have also paid attention on selecting more than one appropriatescale parameter for multi-scale image s

20、egmentation (dOleire-Oltmanns et al., 2013; Dragut and Eisank, 2011; Dragut et al.,2011; Dragut et al., 2010; Trias-Sanz, 2005; Trias-Sanz et al., 2008).Single-band based segmentation scale parameter selectionapproaches may be suitable for some environments featuringstrong spectral contrasts such as

21、 forests. The objects that comprisea forest consistently feature strong gradients in NIR reflectancesuch that the choice of band to base heterogeneity upon is obvious.Since single-band scale parameter selection assumes a contrastingreflectance between objects it may have difficulty discriminatingobj

22、ects with spectrally similar signatures, especially within com-plex environments where multiple objects with different spectralcharacteristics dominate the scenes. In an urban context, the vari-ety of materials, diversity of forms and prevalence of mixed pixelsthe likelihood of all objects contrasti

23、ng in a sole band is substan-tially decreased. Furthermore, we may expect that man-madematerials, such as metal, asphalt and concrete, which have minimalvariability in the NIR region, may feature strong variability in thevisible light region on account of painting, special coatings or evenage. Howev

24、er, the utility of multi-band methods to determine theappropriate segmentation scale parameter has not been examinedand a comprehensive review indicated that few studies have exam-ined multiple band image segmentation of a VHR image. This studyproposes a new unsupervised method of scale parameter se

25、lectionfor multi-scale image segmentation, which simultaneously usesmultiple bands of a VHR image. A series of appropriate segmenta-tion scale parameters are identified to delineate various scales ofland cover objects. Segmentation performance is qualitatively andquantitatively assessed in compariso

26、n with a unsupervised sin-gle-band approach (Espindola et al., 2006; Johnson and Xie, 2011).2. MethodsThe appropriate scale parameters were determined by evaluat-ing the spectral homogeneity post image segmentation conductedacross a range of tested scale parameters. The image segmentationmethod adop

27、ted in this study was multi-resolution non-hierarchi-cal segmentation (MRS) algorithm, a commonly used segmentationalgorithm implemented in eCognition Developer 8.7. Segmentationoccurs by defining small groups of pixels as segments and mergingsimilar neighboring segments together in subsequent steps

28、 until aheterogeneity threshold, set by the scale parameter is reached(Benz et al., 2004). Optimally, the final segments will have the geo-metricalshapeandboundaryaspertherealworldobjectspresentinthe image. As the segments grow, the spectral homogeneitydecreases till the point they match the objects

29、 in the real world size.Spectralhomogeneityofthesegmentswasmeasuredasthespectralangle (Kruse et al., 1993) between each pair of two pixels within thesegment. The index (HAMEAN) measuring the spectral homogeneityof segments were used for segmentation scale parameter selection.2.1. Spectral homogeneit

30、y of segments measured by spectral angleSpectral angle is a common distance metric for two spectracomparison (Kruse et al., 1993). The reflectance spectra ofindividual pixel can be described as vectors in an n-dimensionalspace, where n is the number of spectral bands. Each pixel vectorhas a certain

31、length and direction. The length of the vector repre-sents brightness of the pixel while the direction represents thespectral characteristics of the pixel. Difference of illuminationmainly influences the length of the vector, while spectral distancebetween different pixels is measured by the angle b

32、etween theircorresponding vectors (Luc et al., 2005). The more similar thetwo spectra are, the smaller the spectral angle between them.The spectral angle is defined by the expression given in Eq. (1):ha;b cos?1Pni1aibiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffif

33、fiffiffiffiPni1a2iPni1b2iq0B1CA1where n represents the number of spectral bands, aiand birepresentthe reflectance spectra component i of two different pixels,respectively.Since the spectral angle quantifies the spectral differencebetween two pixels, it is reasonable to use it to measure spectralhomo

34、geneity. Spectral angles between each pair of two pixels ina segment were calculated to determine the mean value. The meanHAMEANfor all segments was produced by averaging the value foreach segment in the entire image. In particular, the validation ofspectral homogeneity measurement using the mean sp

35、ectral anglewas quantitatively compared with a single-band based metric (var-iance) before appropriate segmentation scale parameter selection.2.2. Appropriate segmentation scale parameter selectionWith the increasing scale parameter, spectral homogeneity of asegment thus decreases while the index of

36、 mean spectral angleincreases. When a segment nearly matches the object in the image,the decreasing spectral homogeneity will accelerate, because ofedge effects and the increased likelihood of mixed pixels causegreater spectral heterogeneity at the objects edge in comparisonto the objects center. Fu

37、rthermore, the object boundaries will bepreserved in segmentation at a number of higher levels, wherethe homogeneity of this object remains the same (Dragut et al.,2010). When segments just match the representative objects, theincreasing tendency ofHAMEANwill suddenly reduce or stop,because spectral

38、 homogeneity of segments almost remains thesame right after the appropriate scale parameter is reached. Inorder to discover the variation of spectral homogeneity from ascale level to other, the derivative ofHAMEANwith respect to scaleparameter l were calculated using Eq. (2):_HAMEANl dHAMEANldlHAMEA

39、Nl ?HAMEANl ?DlDl2whereHAMEAN(l) is the average of mean spectral angles at segmen-tation scale parameter l.Based on above statement, when segments at segmentationscale parameter l ?Dl nearly match the appropriate segmentationat scale parameter l,_HAMEANl is usually greater than both_HAMEANl ?Dl and_

40、HAMEANl Dl. Thus, a local peak (LP) in thecurves of_HAMEANl is generated. An LP was considered as a signalof the appropriate scale parameters for multi-scale image segmen-tation. To highlight the expression of each peak, an LP index ILPwasderived as given by Eq. (3):ILP_HAMEANl?_HAMEANl?Dl?_HAMEANl?

41、_HAMEANlDl?3The scale parameters at which relatively greater LPs (i.e. withhigh ILPvalues) occur were considered as the more suitable seg-mentation scale parameters in this study. Since ground objectswith different size co-exist in an image, it is very likely that severalLPs will be identified for t

42、he image.14J. Yang et al./ISPRS Journal of Photogrammetry and Remote Sensing 94 (2014) 13242.3. Segmentation evaluationThe proposed method was evaluated against the traditionalunsupervised single-band based method of Espindola et al.(2006) and Johnson and Xie (2011). The Global Score (GS) index,whic

43、h integrates both global intra-segment homogeneity andinter-segment heterogeneity with weighted variance (wVar) andglobal MoranI (MI), is determined to indicate the optimal scaleparameter based on the lowest GS index. The NIR band was usedsince the NIR band has been found to be the most effective an

44、d reli-able compared to the visible bands (Johnson and Xie, 2011).Performances of the single and multiple band methodologies wereevaluatedintermsofqualitativevisualinterpretationandquantitativediscrepancy measures. Visual interpretation is one of the most com-monly used evaluation methods (Meinel an

45、d Neubert, 2004; Pal andPal, 1993; Pesaresi and Benediktsson, 2001; Zhang et al., 2008)although it is highly subjective, time and labor-intensive. Because thehuman eye is acknowledged as a strong and experienced source forevaluation of segmentation techniques, the quality of a segmentationresult is

46、not acceptable if it could not pass visual interpretation.Forquantitativesegmentationevaluation,anumberofindicesareavailable to evaluate the segmentation performance. However, nosingle index can reliably indicate the goodness of fit between twoobjects. This study employed a more robust approach in t

47、hat a num-ber of discrepancy measures between a reference polygon and thecorresponding final segmented objects were used to evaluate scaleparameter suitability. As the segments that comprise a referencepolygonareunclear,thisstudyconsideredacandidatesegmentacor-responding segment if the intersection

48、area between the referencepolygon and the candidate segment is over half the area of eitherthereferencepolygonorthecandidatesegment(Clintonetal.,2010).One measure of discrepancy detailed in Clinton et al. (2010)defined OverSegmentation (OS1), UnderSegmentation (US1) andEuclidean Distance (ED1) to re

49、flect the geometric relationshipsbetween reference polygons and corresponding segments, shownas Eq. (4):OS1 1 ?areaR SareaRareaR ? SareaRUS1 1 ?areaR SareaSareaS ? RareaSED1 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiOS12US122q4where R is the set of reference polygons and S is the set of cor

50、re-sponding segments. Especially, zero values for OS1 and US1 indicatethat there are no over-segmented and under-segmented objects,respectively. Since both OS1 and US1 are normalized indicesbetween 0 and 1, the composite index ED1 is reasonable enoughto equally value these two indices. Thus, a lower

51、 value of ED1reflects a higher overall segmentation quality considering bothover-segmentation and under-segmentation.Another method considers both the geometric matches and thearithmetic relationship between reference polygons and corre-sponding segment. Liu et al. (2012) proposed three new indices

52、toguarantee desirable segmentation, the Potential Segmentation Error(PSE), Number-of-Segments Ratio (NSR) and ED2, shown as Eq. (5):PSE areaS ? RareaRNSR m ?vjjmED2 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPSE2 NSR2p5where R and S are defined as above, m is

53、 the number of referencepolygons andvis the number of corresponding segments. Zerovalue for PSE means there are no under-segmented objects whilezero value for NSR means an optimal one-to-one relationshipbetween reference polygons and corresponding segments.Values of ED2 are most reliable when PSE an

54、d NSR have a sim-ilar order of magnitude, which does not occur when many objectsare over-segmented. To address the issue, this study proposedthree new indices consisting of OS2, US2 and ED3 based on theaverage value of local OverSegmentation and UnderSegmentationmetrics, shown as Eq. (6):OS2XiXj1?ar

55、ea risj?areari?;sj2SUS2XiXj1?arearisjareasj?;sj2SED3XiXjffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffif

56、fiffi1?arearisjareari?21?arearisjareasj?22vuut;sj2S6where riis an arbitrary element of R and sjis the corresponding seg-ment, as an element of S. Both OS2 and US2 are normalized between0 and 1 so that ED3 can indicate global geometric relationship aswell as arithmetic relationship.In this study, ED1

57、, ED2 and ED3 were utilized as discrepancymeasures between all the hand-digitized reference polygons andthe corresponding segments, and then used to evaluate the qualityof image segmentation at the different scale parameters.3. Experiments3.1. DatasetsWe performed two datasets to examine the perform

58、ance of theproposed multi-band based segmentation scale parameter selec-tion method. The two experiments were designed to demonstratethe effectiveness of the proposed method over two types of VHRimages from different urban areas. One is a QuickBird scene col-lected from urban area of Beijing, China

59、on September of 2003,and the other is a Woldview-2 scene from suburban area of Kas-hiwa, Japan on March of 2011.Boththe QuickBird andWorldView-2images containfour multi-spectral bands (Blue, Green, Red and NIR) and a panchromatic band.The multispectral and panchromatic images were fused to producea four-band pan-sharpened multispectral image with a pixel size of0.61 m for the QuickBird and a pixel size of 0.5 m for the World-View-2. Image fusion was performed using the Gram-Schmidt pro-cedure(LabenandBrower,2000)implementedintheENVIsoftwarepackage. Two subsets of 1500 ? 1500 pixels were clipp