文献翻译-基于最小二乘支持向量回归的短期混沌时间序列预测

1、外文文献翻译Short Term Chaotic Time Series Prediction using Symmetric LS-SVM RegressionAbstractIn this article, we illustrate the effect of imposing symmetry as prior knowledge into the modelling stage, within the context of chaotic time series predictions. It is illustrated that using Least-Squares Suppo

2、rt Vector Machines with symmetry constraints improves the simulation performance, for the cases of time series generated from the Lorenz attractor, and multi-scroll attractors. Not only accurate forecasts are obtained, but also the forecast horizon for which these predictions are obtained is expande

3、d.1. IntroductionIn applied nonlinear time series analysis, the estimation of a nonlinear black-box model in order to produce accurate forecasts starting from a set of observations is common practice.Usually a time series model is estimated based on available data up to time t, and its final assessm

4、ent is based on the simulation performance from t + 1 onwards. Due to the nature of time series generated by chaotic systems, where the series not only shows nonlinear behavior but also drastic regime changes due to local instability of attractors, this is a very challenging task. For this reason, c

5、haotic time series have been used as benchmark in several time series competitions.The modelling of chaotic time series can be improved by exploiting some of its properties. If the true underlying system is symmetric, this information can be imposed to the model as prior knowledge , in which case it

6、 is possible to obtain better forecasts than those obtained with a general model . In thisarticle, short term predictions for chaotic time series are generated using Least-Squares Support Vector Machines (LS-SVM) regression. We show that LS-SVM with symmetry constraints can produce accurate predicti

7、ons. Not only accurate forecasts are obtained, but also the forecast horizon for which these predictions are obtained is expanded, when compared with the unconstrained LS-SVM formulation.This paper is structured as follows. Section 2 describes the LS-SVM technique for regression, and how symmetry ca

8、n be imposed in a straightforward way.Section 3 describes the applications for the cases of the xcoordinate of the Lorenz attractor, and the data generated by a nonlinear transformation of multi-scroll attractors.2. LS-SVM with Symmetry ConstraintsLeast-Squares Support Vector Machines (LS-SVM) is a

9、powerful nonlinear black-box regression method,which builds a linear model in the so-called feature space where the inputs have been transformed by means of a (possibly infinite dimensional) nonlinear mapping . This is converted to the dual space by means of the Mercers theorem and the use of aposit

10、ive definite kernel, without computing explicitly the mapping. The LS-SVM formulation, solves a linear system in dual space under a least-squares cost function , where the sparseness property can be obtained by e.g. sequentially pruning the support valuespectrum or via a fixed-size subset selection

11、approach. The LS-SVM training procedure involves the selection of a kernel parameter and the regularization parameter of the cost function, which can be done e.g. by cross-validation, Bayesian techniques or others. The inclusion of a symmetry constraint (odd or even) to the nonlinearity within the L

12、S-SVM regression framework can be formulated as follows . Given the sampleof N points xk, y Nkk = 1, with input vectorsxkÎ Rp and output valuesykÎ R , the goal is toestimate a model of the formy = wTj (x) + b + ej (·) : Rp ® Rnhis the mapping to a high dimensional (and possibly i

13、nfinite dimensional) featurespace, and the residuals e are assumed to be i.i.d. with zero mean and constant (and finite) variance. The following optimization problem with a regularized cost function is formulated:1minwT w + g1 åNe 2w,b,ek 22kk =1ì y= wTj(x ) + b + es.t. íkkk, k = 1,.N

14、 ,îwTj(x ) = awTj(-x), k = 1,.N ,kkwhere a is a given constant which can take either -1 or 1. The first restriction is the standard model formulation in the LS-SVM framework. The second restriction is a shorthand for the cases where we wantto impose the nonlinear functionwTj (xk)to be even (res

15、p. odd) by using a = 1 (resp. a = 1). Thesolution is formalized in the KKT lemma.3. Application to Chaotic Time SeriesIn this section, the effects of imposing symmetry to the LS-SVM are presented for two cases of chaotic time series. On each example, an RBF kernel is used and the parameterssand gare

16、 found by10-fold cross validation over the corresponding training sample. The results using the standard LS-SVM are compared to those obtained with the symmetry-constrained LS-SVM (S-LS-SVM) from (2). The examples are defined in such a way that there are not enough training datapoints on every regio

17、n of the relevant space; thus, it is very difficult for a black-box model to ”learn” thesymmetry just by using the available information. The examples are compared in terms of the performance in the training sample (cross-validation mean squared error, MSE-CV) and the generalization performance (MSE

18、 out of sample, MSE-OUT).Foreachcase,aNonlinearAutoRegressive (NAR)black-boxmodelisformulated:y( t)=g( y(-t1) ,y -(t2 ) , .y.-t,(p+)e) twhere g is to be identified by LS-SVM and S-LS-SVM. The order p is selected during the cross-validation process as an extra parameter. After each model is estimated

19、, they are used in simulation mode, where the future predictions are computed with the estimated model using past predictions:ÙÙÙÙÙy(t) = g( y(t -1), y(t - 2),., y(t - p)3.1. Lorenz attractorThis example is taken from 1. The xcoordinateof the Lorenz attractor is used as an e

20、xample of a time series generated by a dynamical system. A sample of 1000 datapoints is used for training, which corresponds to an unbalanced sample over the evolution of the system, shown on Figure 1 as a time-delay embedding.Figure 2 (top) shows the training sequence (thick line) and the future ev

21、olution of the series (test zone). Figure 2 (bottom) shows the simulations obtained from both models on the test zone. Results are presented on Table 1. Clearly the S-LS-SVM can simulate the system for the next 500 timesteps, far beyond the 100 points that can be simulated by the LS-SVM.3.2. Multi-s

22、croll attractorsThis dataset was used for the K.U.Leuven Time Series Prediction Competition . The series was generatedby·x = h(x)y = W tanh(Vx)where h is the multi-scroll equation, x is the 3-dimensional coordinate vector, and W,V are the interconnection matrices of the nonlinear function (a 3-

23、units multilayer perceptron, MLP). This MLP function hides the underlying structure of the attractor .A training set of 2,000 points was available for model estimation, shown on Figure 3, and the goal was to predict the next 200 points out of sample. The winner of the competition followed a complete

24、 methodology involving local modelling, specialized many-steps ahead cross-validation parameters tuning, and the exploitation of the symmetry properties of the series (which he did by flipping the series around the time axis).Following the winner approach, both LS-SVM and S-LS-SVM are trained using

25、10-step-ahead cross-validation for hyperparameters selection. To illustrate the difference between both models, the out of sample MSE is computed considering only the first n simulation points, where n = 20, 50, 100, 200. It is important to emphasize that both models are trained using exactly the sa

26、me methodology for order and hyperparameter selection; the only difference is the symmetry constraint for the S-LS-SVM case. Results are reported on Table 2. The simulations from both models are shown on Figure 4.Figure 1: The training (left) and test (right) series from the xcoordinate of the Loren

27、z attractorFigure 2: (Top) The series from the xcoordinateLoofrentzhaettractor, part of which is used for training (thick line).(Bottom) Simulations with LS-SVM (dashed line), S-LS-SVM (thick line) compared to the actual values (thin line).Figure 3: The training sample (thick line) and future evolut

28、ion (thin line) of the series from the K.U.Leuven Time Series CompetitionFigure 4: Simulations with LS-SVM (dashed line), S-LSSVM (thick line) compared to the actual values (thin line) for the next 200 points of the K.U.Leuven data.Table 1: Performance of LS-SVM and S-LS-SVM on the Lorenz data.Table

29、 2: Performance of LS-SVM and S-LS-SVM on the K.U.Leuven data.4. ConclusionsFor the task of chaotic time series prediction, we have illustrated how to use LS-SVM regression with symmetry constraints to improve the simulation performance for the cases of series generated by Lorenz attractor and multi

30、-scroll attractors. By adding symmetry constraints to the LS-SVM formulation, it is possible to embed the information about symmetry into the kernel level. This translates not only in better predictions for a given time horizon, but also on a larger forecast horizon in which the model can track the

31、time series into the future.基于最小二乘支持向量回归的短期混沌时间序列预测摘要：本文解释了在混沌序列预测范围内，先验知识到模型阶段任意使用对称性的作用效果。以洛伦兹吸引器和多涡卷吸引子为例，表明了使用具有对称约束的最小二乘支持向量机能够提高仿 真性能，其不仅能够获得的预测结果精确，而且所获得的预测范围能够得到扩展。1. 介绍应用非线性时间序列分析，对一个非线性的黑箱模型的预测，要得到准确的预测，从一开始的 一系列观测是比较常见的做法。通常一个时间序列模型是根据可用的时间 t 数据预测，他的最后评估时基于 t+1 时刻起的仿真性能。由于时间序列的本质产生于混沌系统，那

32、里的时间序列不仅显示非线性行为而且激烈的范围变化引起了吸引子的局部不稳定，这是一个非常具有挑战性的任务。因 此，混沌时间序列已经在几次时间序列的竞争中被看做基准。混沌时间序列的模型可以利用它的一些属性来改善。如果真正的底层系统是对称的，那么这个信 息可以强加到模型中作为先验知识。在这篇文章中，短期预测混沌时间序列是利用最小二乘支持向 量机（LS-SVM）的回归来产生。我们的分析表明，与无约束的最小二乘支持向量机算法相比，利用 对称约束的最小二乘支持向量机可以产生精确的预测结果：不仅得到了精确的预测结果，而且这些 预测的预测界限还得到了扩展。这篇文章的摘要组织如下。第二部分描述了最小二乘支持向量

33、回归的技术，以及对称可以以一种 简便的方法实行。第三部分描述了以 x 坐标下的洛伦兹吸引器为例的应用，以及多轴吸引子的非线性转变所产生的数据。2. 利用对称约束的最小二乘支持向量机最小二乘支持向量机（LS-SVM）是一个强大的非线性黑箱模型的回归方法，它在所谓的输入是 一系列非线性映射的特征空间（可能是无限的空间）里，构建了一个线性模型。利用mercer 定理和使用正定内核转变在二维空间进行分析，并不用计算明确的映射。最小二乘支持向量机解决了二维 空间下线性系统的最小二乘成本函数，稀疏属性可以连续地修剪保价频谱，或者通过一个固定大小 的子集选取来接近。最小二乘支持向量机训练的过程涉及到核函数参

34、数的选取和定向聚合价值函数 的参数选择，可以用交叉验证，贝叶斯方法或者其他方法来实现。包括对称约束（奇数或偶数）内 的非线性特征，可以利用最小二乘支持向量机回归框架来制定如下：给出N 个实例点x, y N，输入向量xÎ Rp 和输出向量 yÎ R ，目标是来预测如下的模kkk = 1kk型:y = wTj (x) + b + ej (·) : Rp ® Rnh 是映射到一个高维空间（并可能无限维）的特征空间，和其余额 e，被认为是 i.i.d 与灵均值和方差常数。用正规化函数的制定来解决以下优化问题：1minwT w + g1 åNe 2w,b

35、,ek 22kk =1ì y= wTj(x ) + b + es.t. íkkk, k = 1,.N ,îwTj(x ) = awTj(-x), k = 1,.N ,kka 是一个给定的常数，可以取 1 或者-1，第一个限制是最小二乘支持向量机框架中的标准模型框架，第二个限制是我们希望对非线性函数变换wTj (xk) 成为偶数（或者奇数）利用 a 的取值为 1（奇数为-1）。在 KKT 定理中引出这个正式的解决方案。 3.混沌时间序列的应用在这部分中，最小二乘支持向量机的对称约束的影响通过两个混沌时间序列的例子来表示。在每个例子中，使用径向基核函数，径向基核函数参

36、数s 和g 在相应的训练样本中进行十倍的交叉验证选取。利用最小二乘支持向量机的标准进行了结果对比，与那些获得对称的最小二乘支持向量机（S-LS-SVM）进行对比。实例以这样一种方式定义，在相对空间的每一个范围里没有得到足够的训练，因此，对一个黑箱模型来说，只用过可用的信息来学习对称是一件特别困难的事。这些示例在训练样本中利用性能来对比（交叉验证表示性能错误， MSE-CV）和普遍性能（MSE-OUT）。在每个 例 子 中 ， 一 个 非 线 性 自 回 归 （ NAR ） 黑 箱 模 型 这 样 设 定 ：y(t) = g( y(t -1), y(t - 2),., y(t - p) + e(t)g 用来鉴别 S-LS-SVM 和 LS-SVM。p 命令在交叉验证时被选作是额外的参数。在每个模型都被预 测 之 后 ， 它 们 被 用 于 模 拟 模 式 ， 那 里 的 未 来 预 测 利 用 过 去 的 预 测 来 计