60an independent component analysis-based disturbance separation

1、J Intell Manuf (2012) 23:561573 DOI 10.1007/s10845-010-0394-3An independent component analysis-based disturbanceseparation scheme for statistical process monitoringChi-Jie LuReceived: 20 April 2009 / Accepted: 19 February 2010 / Published online: 8 March 2010 Springer Science+Business Media, LLC 201

2、0Abstract In this paper, an independent component analysis (ICA)-based disturbance separation scheme is proposed for statistical process monitoring. ICA is a novel statistical signal processing technique and has been widely applied in medical signal processing, audio signal processing, feature extra

3、ction and face recognition. However, there are still few applica- tions of using ICA in process monitoring. In the proposed scheme, ICA is first applied to in-control training process data to determine the de-mixing matrix and the correspond- ing independent components (ICs). The IC representing the

4、 white noise information of the training data is then identi- fied and the associated row vector of the IC in the de-mixing matrix is preserved. The preserved row vector is then used to generate the monitoring IC of the process data under mon- itoring. The disturbances in the monitoring process can

5、be effectively enhanced in the monitoring IC. Finally, the tradi- tional exponentially weighted moving average control chart is used to the monitoring IC for process control. For eval- uating the effectiveness of the proposed scheme, simulated manufacturing process datasets with step-change disturba

6、nce are evaluated. Experiments reveal that the proposed moni- toring scheme outperforms the traditional control charts in most instances and thus is effective for statistical process monitoring.IntroductionStatistical process control (SPC) has been extensively used to monitor and improve the quality

7、 of manufacturing processes. Control charts, such as Shewhart, exponentially weighted moving average (EWMA) and cumulative sum (CUSUM) charts, are effective SPC tools which can help reduce varia- tion in manufactured products and increase the competitive- ness by improving product quality, and hence

8、 are basic and powerful SPC tools to deal with real world process control problems.Traditional SPC control charting techniques developed for discrete manufacturing operations are often not applicable in continuous and batch process industries because the manu- facturing process data violate a basic

9、statistical assumption used to develop the SPC charts. The basic assumption of SPC charts is that the processing datasets are independently and identically distributed (IID) about the process mean (Montgomery 2001). However, the process datasets collected from most continuous and batch process opera

10、tions are usu- ally serially correlated (Alwan and Roberts 1988; Chiu et al. 2001, 2003; Zhang 1998). It has been found that the perfor- mance of traditional control charts is significantly affected by autocorrelated data (Altiok and Melamed 2001; Alwan 1992; Harris and Ross 1991; Woodall and Faltin

11、 1993; Zhang 1998). When traditional SPC techniques of control charts are applied for monitoring autocorrelated process data, the most important effect of autocorrelation is in the generation of a large number of false out-of-control signals. False alarms can cause unnecessary inspection costs and e

12、fforts for in- control process and, hence, monitoring effective schemes are required for correlated processes.Among the commonly discussed traditional statistical process control methodologies, the time series modeling techniques have been recommended to monitor correlatedKeywordsStatistical process

13、 control Independent component analysis EWMA control chart Correlated processC.-J. Lu (B)Department of Industrial Engineering and Management, Ching YunUniversity, 320 Jhong-Li City, Taoyuan County, e-mail: .tw, , ROC1 3562J Intell Manuf (2012) 23:561573the model. Fina

14、lly, the potential mixed effects of modeling errors and process mean changes may render the developed control chart useless or misleading (Hwarng 2004; Lu and Reynolds 1999; Zhang 1998). Therefore, the performance of the time-series based control charts depends highly on the adequacy of the fitted m

15、odel.In this paper, a process monitoring scheme based on inde- pendent component analysis (ICA), one of the most widely used techniques for signal separation, is proposed for moni- toringthe meanofcorrelated process. Unlike the above-men- tioned residual charts, the proposed scheme does not require

16、estimating the parameter(s) of time series model and should be a possible alternative in monitoring correlated processes. Independent component analysis is a novel statistical sig- nal processing technique. The basic idea of it is to extract the hidden information fromtheobserved multivariate statis

17、tical data (Cichocki and Amari 2002; Hyvrinen et al. 2001; Lee 1998). This hidden information, which is called the indepen- dent component (IC) of the observable data, is assumed to be non-Gaussian and mutually independent. In other words, ICA is seeking a set of independent components from the meas

18、ured process variables. Therefore, it is quite natural to infer that monitoring based on the ICA solution may give bet- ter results compared with monitoring the original data (Lee et al. 2003a) because the original data could be a mixture of noise and process characteristics such as process distur-b

19、ances and/or autocorrelation (Lu et al. 2008).The basic ICA model has been widely applied in signal processing, face recognition, feature extraction and data min- ing (David and Sanchez 2002; Hyvrinen et al. 2001; Lu and Tsai 2008; Lu et al. 2009; Wang et al. 2006). However, there are still few appl

20、ications of using ICA in process monitor- ing (Wang et al. 2009). Kano et al. (2003) have successfully demonstrated the idea of process monitoring based on the observation of ICs instead of the original measurements. In their work, a set of devised SPC charts have been developed effectively for each

21、 IC. Lee et al. (2003a,b) investigated the utilization of kernel density estimation to define the con- trol limits of ICs that do not satisfy Gaussian distribution. In order to monitor the batch processes which combine ICA and kernel estimation, Lee et al. (2004) extended their original method to mu

22、lti-way ICA. Xia and Howell (2003) developed a spectral ICA approach to transform the process measure- ments from the time domain to the frequency domain and to identify major oscillations.Shannon et al. (2003, 2004) used ICA in monitoring a semiconductor manufacturing process. They applied ICA on m

23、icroelectronic parametric test (E-test) data to generate inde- pendent components for isolating the sources of variation in the test data. Then, each IC was used to represent different fundamental physical mechanisms of the process and was charted using traditional Xbar chart for process control and

24、 diagnosis. He et al. (2004) combined ICA and multi-wayNoThe adequacy of the fitted model?YesFig. 1The flowchart of the traditional residual chart proposedprocess data (Alwan and Roberts 1988; Harris and Ross 1991; Wardell et al. 1992, 1994). The time series model- ing techniques firstly use an appr

25、opriate time series model to fit the autocorrelated data, then apply standard control charts to the residuals to generate residual control charts for controlling correlated process. If the process quality param- eter data do not undergo any shift in the mean, variance or autocorrelative structure, t

26、he residuals of the time series model will exhibit a mean of approximately zero and non- significant autocorrelations at all lags (Yourstone and Mont- gomery 1989). Thus, the autocorrelative structure of the data is accounted for using the time series model, and determi- nations concerning the proce

27、ss state of control can be made from monitoring the residual values for non-random behav- ior. The flowchart of a traditional residual control chart, such as the residual EWMA chart used in this paper, is illustrated in Fig. 1.Several researchers have evaluated the performance of time series modelin

28、g techniques (Harris and Ross 1991; Lu and Reynolds 1999; Wardell et al. 1994). Even though time series modeling techniques are widely adopted, the major limitation is that a time series model has to be identified before residuals can be obtained. Besides, time series mod- eling of the process data

29、is not always straightforward. More- over, it requires regular validity checks of the fitted time series model because the underlying structure of the mon- itoring process may change. If the autocorrelated character- istic of the monitoring process is changed, we need to rebuild1 3Training phaseMoni

30、toring phaseUse traditional control chart, such as EWMA chart, to the residual data for process monitoring.Use the fitted model to the monitoring autocorrelated process data to generate the residual process dataPreserve the fitted model.Use an appropriate time series model to fit the autocorrelated

31、dataCollect historical autocorrelated process data from the monitoring processesJ Intell Manuf (2012) 23:561573563principal component analysis (MPCA) for batch process monitoring. They used ICA and MPCA on the multivariate historical data collected from a normal operated process to estimate ICs and

32、calculate the latent value of Hotelling T2, respectively. If the negentropy values of the ICs are larger then the predefined threshold, each ICs was monitored for process control, else the T2 value is monitored. Al-bazzaz and Wang (2004) developed an ICA-based control charts for batch process. In th

33、eir research, ICA was used in unfold- ing batch process data to estimate ICs that are non-Gaussian distributed. Instead of using the original variables, each IC was charted with time varying control limits for batch pro- cess monitoring. Shannon and McNames (2007) developed an ICA based disturbance

34、specific control charts for multivar- iate process monitoring. In their work, first, ICA was applied on the E-test data containing known disturbances and recy- cled from factorial design experiments to estimate ICs for disturbance-source estimation. Then, the ICs and de-mixing matrix were used to es

35、timate a factor model that is utilized to develop control charts for controlling particular types of process disturbance.The above papers mainly apply ICA to deal with the prob- lemof monitoring multivariable systems consisting of alarge number of correlated/uncorrelated variables. However, the util

36、ized methods can not be applied directly in monitoring univariate process consisting of only one key measurement. Besides, the existing researches did not apply their methods on autocorrelated process data and/or evaluate the perfor- mance of the methods in monitoring the correlated process data wit

37、h different levels of autocorrelation and disturbance. In univariate process control, there is only one key vari- able/measurement to be monitored. However, the basic ICA method assumes that the observed variable is multiple. In order to perform ICA, in the proposed monitoring scheme, the training d

38、ata matrix for ICA is generated by combining a in-control AR(1) process and a pseudo variable.The AR(1) process data is collected from the key variable and is Gauss- ian distributed. Thepseudovariable is a Gaussianwhitenoisedata with zero mean and unit variance.This paper focused on monitoring an un

39、ivariate process. The proposed ICA monitoring scheme first uses ICA to the in-control training correlated process data to estimate the de-mixing matrix and ICs. The IC representing the white noise information of the training data is then identified and the associated row vector of the IC in the de-m

40、ixing matrix is preserved. The preserved row vector containing the least autocorrelation information of the process data is then used to generate the IC (called monitoring IC) of the process data under monitoring. The autocorrelation of the monitoring pro- cess will be depressed and the disturbances

41、 can be effectively enhanced in the monitoring IC. Finally, the EWMA chart is applied to the monitoring IC for process control. Since many autocorrelation cases collected from manufacturing opera-tions can be modeled using first order autoregressive (AR(1) model (Alwan 1992; English et al. 2000; Mon

42、tgomery et al. 1994; Yourstone and Montgomery 1989), the correlated pro- cess simulated and adopted in this study is AR(1) process.The rest of this paper is organized as follows. Sec- tion “Independent component analysis” provides a brief introduction to ICA. Section “ICA-based disturbance separa- t

43、ion scheme” presents the AR(1) process and step-change dis- turbance, followed by reviewing the EWMA control charts. Then the proposed ICA monitoring scheme and the effect of training sample are thoroughly described. Section “Experi- mental results” presents the experimental results. The paper is co

44、ncluded in section“Conclusions”.Independent component analysisIn the basic ICA algorithm, it is assumed that m measured variables, x(t) = x1(t), x2(t), . . . , xm(t)T at time t can be expressed as linear combinations of n unknown latent source components s1, s 2 ,., sn:nx(t) =a jsj (t) =As(t )(1)j =

45、1where s(t) = s1(t), s2(t), . . . , sn(t)T and a j is the jth row of unknown mixing matrix A. Here, we assume m n for A to be full rank matrix. The vector s is the latent source datathat cannot be directly observed from the observed mixture data x. The basic problem of ICA is to estimate the latent

46、source components s(t) and unknown mixing matrix A from x(t) withappropriateassumptions on thestatisticalproperties of the source distribution. Thus, ICA model aims at finding a de-mixing matrix W such thatny(t) =w jxj (t) = Wx(t),(2)j =1where y(t) = y1(t), y2(t),., yn(t)T is the independent compone

47、nt vector. The elements of y(t) must be as statis-tically independent as possible, and are called independent components (ICs). The ICs are used to estimate the latent source components s(t). The vector w j in Eq. (2) is the j throw of the de-mixing matrix W, j = 1, 2 ,., n. It is used to filter the

48、 observed data x(t) to generate the correspond-ing independent component yj (t), i.e., yj (t) = w j x(t), j =1, 2 ,., n.The ICA modeling is formulated as an optimization prob- lem by setting up the measure of the independence of ICs as an objective function followed by using some optimiza- tion tech

49、niques for solving the de-mixing matrix W. Several existing algorithms can be used for performing ICA mod- eling (Bell and Sejnowski 1995; David and Sanchez 2002;1 3564J Intell Manuf (2012) 23:5615732Hyvrinen and Oja 1997). In general, the ICs are obtained by using the de-mixing matrix W to multiply

50、 the observed datax(t), i.e., y(t) = Wx(t). The de-mixing matrix W can be determined using an unsupervised learning algorithm withthe objective of maximizing the statistical independence of ICs. The ICs with non-Gaussian distributions imply the sta- tisticalindependenceamongthem(Hyvrinen and Oja 200

51、0).Normally, non-Gaussianity of ICs can be measured by the negentropy (Hyvrinen and Oja 2000):f = 0 + (f 0) + , t N(6)(0, )tt 1tThe standardized AR(1) process is used as the basis for sim- ulation of datasets with process shifts.In manufacturing process control, minimization of the deviation of the

52、process output from the target quality char- acteristic is always one of the goals. However, unexpected incidences or deliberate adjustments are frequently happened in the industrial processes. Changes in product design, vacil- lations in material properties, or unanticipated events may all upset th

53、e production process and demand correction (Shao 1998). Normally, all these disruptions are considered as tran- sient disturbances. Since the step-change disturbance is one of the commonly encountered disturbances (MacGregor et al. 1984; Shao 1998; Shao and Chiu 1999; Shao et al. 1999; Wang and Chen

54、 2002; Wu and Shamsuzzaman 2006; Yang 2009 ), the proposed method will focus on monitoring AR(1) process data with a step-change disturbance.When standardized AR(1) process data ft has a step- change disturbance occurring at an unknown time point J(y) = H(ygauss) H(y)(3)where ygauss is a Gaussian ra

55、ndom vector having the same covariance matrix as y . H is the entropy of a random vector ywith density p(y) defined as H(y) = p(y)log p(y)dy.The negentropy is always non-negative and is zero ifand only if y has a Gaussian distribution. Since the prob- lem in using negentropy is computationally very

56、difficult, an approximation of negentropy is proposed (Hyvrinen and Oja 2000) as follows:J(y) E G(y) E G(v)2(4)with mean shift size (measured by ), f would be trans-tferred to xt as followsxt = ft + ht0if t where v is a Gaussian variable of zero mean and unit vari- ance, and y is a random variable w

57、ith zero mean and unit var-iance. G is a nonquadratic function, and is given by G(y) = exp(y2/2) in this study. The FastICA algorithm proposed by Hyvrinen et al. (2001) is adopted in this paper to solvefor the de-mixing matrix W.where ht=(7)1if t .EWMA control chartICA-based disturbance separation s

58、chemeThe EWMA chart is designed by giving exponential weight to past observations and hence the more recent data points receiving heavier weights. The value of the EWMA statistic at time t, denoted by ct, is expressed asAR(1) process and step-change disturbanceAn AR(1) model is used for simulating process data with and without shifts in the mean parameter value for testing the per- formance of the proposed ICA-based monitoring scheme. It can be described as the follow