Improvement of the Simplified Fast Transversal Filter Type Algorithm for Adaptive Filtering 1

Problem statement: In this study, we proposed a new algorithm M-SMFTF for adaptive filtering with fast convergence and low complexity. Approach: It was the result of a simplified FTF type algorithm, where the adaptation gain was obtained only from the forward prediction variables and using a new recursive method to compute the likelihood variable. Results: The computational complexity was reduced from 7L-6L, where L is the finite impulse response filter length. Furthermore, this computational complexity can be significantly reduced to (2L+4P) when used with a reduced P-size forward predictor. Conclusion: This algorithm presented a certain interest, for the adaptation of very long filters, like those used in the problems of echo acoustic cancellation, due to its reduced complexity, its numerical stability and its convergence in the presence of the speech signal.


INTRODUCTION
In general the problem of system identification involves constructing an estimate of an unknown system given only two signals, the input signal and a reference signal. Typically the unknown system is modelled linearly with a Finite Impulse Response (FIR) and adaptive filtering algorithms are employed to iteratively converge upon an estimate of the response. If the system is time-varying, then the problem expands to include tracking the unknown system as it changes over time [1][2][3] . There are two major classes of adaptive algorithms. One is the Least Mean Square (LMS) algorithm, which is based on a stochastic gradient method. The LMS algorithm has been extensively studied and many theoretical results on its transient and steady state performances have been obtained [3] . The LMS algorithm has a computational complexity of O(L), L is the FIR filter length. The other class of adaptive algorithm is the Recursive Least-Squares (RLS) algorithm which minimizes a deterministic sum of squared errors [4] . The RLS algorithm solves this problem, but at the expense of increased computational complexity of O(L 2 ). A large number of Fast RLS (FRLS) algorithms have been developed over the years, but, unfortunately, it seems that the better a FRLS algorithm is in terms of computational efficiency, the more severe is its problems related to numerical stability [4] . Fast versions of these algorithms, namely, the fast Kalman [5] , the Fast A Posteriori Error Sequential Technique (FAEST) [6] and Fast Transversal Filter (FTF) [7] algorithms, are derived from the RLS by the introduction of forward and backward predictors. The FRLS algorithm shows a complexity of O(L). Several numerical solutions of stabilization, with stationary signals, are proposed in the literature [8][9][10][11][12] . Another way of reducing the complexity of the FRLS algorithm has been proposed in [13,14] : When the input signal can be accurately modelled by a predictor of order P, the fast Newton Transversal Filter (FNTF) avoids running forward and backward predictors of order L, which would be required by a FRLS algorithm. The required quantities are extrapolated from the predictors of order P (P<<L). Thus, the complexity of the FNTF falls down to (2L+12P) multiplications instead of 8L. Recently, the simplified FTF-type algorithm [15] developed for use in acoustic echo cancellers. This algorithm derived from the FTF algorithm where the adaptation gain is obtained only from the forward prediction variables. The computational complexity of this algorithm is 7L when used with a full size predictor which is less complex than the original numerically stable 8L FTF algorithm. In this study, we propose more complexity reduction of the simplified FTF-type algorithm by using a new recursive method to compute the likelihood variable. The computational complexity of the proposed algorithm is 6L and this computational complexity can be significantly reduced to (2L+4P) when used with a reduced P-size forward predictor. The M-SMFTF of the proposed algorithm outperforms the classical adaptive algorithms because of its convergence speed which approaches that of the RLS algorithm and its computational complexity which is slightly greater than the one of the NLMS algorithm. We describe the NLMS and numerically stable FRLS (NS-FRLS) algorithms. More complexity reduction for simplified FTF-type (M-SMFTF) algorithm is proposed. At the end, we present some simulation results of the proposed algorithms.
Adaptive algorithms: The main identification block diagram of a linear system with Finite Impulse Response (FIR) is represented in Fig. 1. The output a priori error L,n ε of this system at time n is: L,n n n d y ε = − where, T opt ,L opt,1 opt,2 opt,L w w , w , ..., w   =   represents the unknown system impulse response vector and v n is a stationary, zero-mean and independent noise sequence that is uncorrelated with any other signal. The superscript T describes transposition. The filter is updated at each instant by feedback of the estimation error proportional to the adaptation gain, denoted as g L,n and according to: The different algorithms are distinguished by the gain calculation.
The NLMS algorithm: Algorithms derived from the gradient [3] , for which the optimization criterion corresponds to a minimization of the mean-square error. For the Normalized LMS (NLMS) algorithm, the adaptation gain is given by: Where: µ = Referred to as the adaptation step c 0 = A small positive constant used to avoid division by zero in absence of the input signal The stability condition of this algorithm is 0<µ<2 and the fastest convergence is obtained for µ = 1 [16] . The power π x,n of input signal can alternatively be estimated using the following recursive equation [17] : 2 x,n x,n 1 n where, γ is a forgetting factor ( 1 / L γ ≈ ). The computational complexity of the NLMS algorithm is 2L multiplications per sample.
The NS-FRLS algorithm: The filter w L,n is calculated by minimizing the weighted least squares criterion according to [1] : where, λ denotes the exponential forgetting factor (0<λ≤1). The adaptation gain is given by: where, R L,n is an estimate of the correlation matrix of the input signal vector. The variables γ L,n and L,n k ɶ respectively indicate the likelihood variable and normalized Kalman gain vector. This latter is calculated, independently of the filtering part w L,n , by a FRLS algorithm using forward/backward linear prediction analysis over the signal x n [1] . The calculation complexity of a FRLS algorithm is of order L. This reduction of complexity, compared to that of RLS algorithms, which have a complexity of order L 2 , have made all FRLS algorithms numerically unstable.
The numerical stability is achieved by using a control variable, called also a divergence indicator ξ n [11] , theoretically equals to zero. Its introduction in an unspecified point of the algorithm modifies its numerical properties. This variable is given by: with ≤ µ ≤ . It is obtained by using some redundant formulae of the FRLS algorithms. We can calculate differently the backward a priori prediction errors in tree ways ( L,n r , 0 f L,n r and 1 f L,n r ). We define three backward a priori prediction errors, theoretically equivalents, which will be used to calculate the likelihood variable γ L,n , the backward prediction error variance β L,n and the backward prediction b L,n . We introduce these variables into the algorithm and we use suitably the scalar parameters b ( , , ) γ β µ µ µ and µ s , in order to obtain the numerical stability. It can be shown that the variance of the numerical errors in the backward predictor, with the assumption of a white Gaussian input signal, is stable under the following condition [11] : The resulting stabilized FRLS (NS-FRLS) algorithms have a complexity of 8L; it is shown in Algorithm 1. Note that numerical stabilization of the algorithm limits the range of the forgetting factor λ (10) and consequently their convergence speed and tracking ability.

Proposed algorithms:
The M-SMFTF algorithm: We propose more complexity reduction of the simplified FTF-type (M-SMFTF) algorithm by using a new recursive method to compute the likelihood variable. The Simplified FTFtype algorithm [15] derived from the FTF algorithm where the adaptation gain is obtained only from the forward prediction variables. The backward prediction variables, which are the main source of the numerical instability in the FRLS algorithms, are completely discarded.  ( r ) Filtering part: By using only forward prediction variables and adding a small regularization constant c a and a leakage factor η, we obtain a robust numerically stable adaptive algorithm that shows the same performances as FRLS algorithms.
By taking the expression of normalized Kalman gain: L,n 1 L,n L,n L,n L,n 1 L,n 1 L,n 1 L,n 1 and if we discard all backward prediction variables from (11) and use only the forward variables to compute the normalized Kalman gain: L,n L,n L,n 1 L,n 1 L,n 1 This algorithm is not very robust with nonstationarity input signal like speech signals. The first difficulty comes from The second difficulty is that the forward predictor is locked over its last values. It is known that the FRLS algorithms were developed in the prewindowing case and all vectors are initialised by zero so that the algorithm starts adapting. In these conditions, when the input signal vanishes and reappears after a long period of time, the algorithm may diverge because of these nonzero values of the predictor. In other words, the algorithm is not well initialised when the signal reappears. In such conditions, it might be preferable to have the forward predictor a L,n return back to zero by doing the following operation: where, η is a close to one constant often called the leakage factor [12] . The likelihood variable is given by using the definition directly: By exploiting certain invariance properties by shifting the vector input signal extended to the order (L+1), we obtain two writing manners of input vector: By multiplying on the left, the members of left and right of the expression (16) by Eq. 17a and b respectively, the following equality is obtained: The computational complexity of the M-SMFTF algorithm is 6L; it is shown in Algorithm 2.
The RM-SMFTF algorithm: The Reduced size predictors in the FTF algorithms have been successfully used in the FNTF algorithms [13][14][15] . The proposed algorithm can be easily used with reduced size prediction part. If we denote P the order of the predictor and L the size of adaptive filter, the forward predictor and the normalized Kalman gain are given respectively by: L,n P,n P,n 1 L,n 1 P,n 1 a L,n L P where, P is much smaller than L. The first (P+1) components of the L,n k ɶ are updated using the reduced size forward variables, the last components are just a shifted version of the (P+1) th component of L,n k ɶ . For this algorithm, we need two likelihood variables: The first one, γ P,n , is used to update the forward prediction error variance α P,n , where c P,n is (P+1) th component of L,n k ɶ . The second likelihood variable, γ L,n , is used to update the forward predictor a P,n of order P and the transversal filter w L,n . The computational complexity of this algorithm is (2L+4P); it is shown in Algorithm 3.

Analysis prediction part:
We study the propagation of errors in all recursive quantities of the prediction part of the M-SMFTF algorithm. Assuming that the numerical errors are small, the model of error propagation in the recursive variables can be approximated by the following linear model: where, e(n) represent the round-off noise. We can write the state vector of the errors at the time n as follows: ; P,n 1 L,n 1 P,n L,n k k (1: P);c k (P 1) 2 P,n P,n 1 P,n P,n n P P,n P,n 1 a P,n P,n 1 e c x ; c 1 represent respectively the errors cumulated up until the time n in the forward and Kalman recursive variables. The (2L+2) × (2L+2) dimensional matrix F(n) given by: represents the transition matrix. The system (22) is said to be stable, in the mean sense, if the eigenvalues of E{F(n)}, in the steady state, are all less than one in magnitude [9] . The operator E{.} denotes the expected value. After a propagation analysis of the numerical errors of the 1 st order and an asymptotic study of the equations of errors propagation, we approximate the errors in the forward variables (∆a L,n , ∆α L,n ) and the Kalman variables ( L,n 1 k − ∆ ɶ , ∆γ L,n-1 ) by the following linear first order models deduced from differentiating (a L,n , α L,n ) and ( L,n k ɶ , γ L,n ) respectively: Where: L,n L,n L,n L,n 1 By assuming that the perturbation terms (p a (n), p α (n), p k (n) and p γ (n)) remain limited.
In asymptotic mode, we can write: We can thus say that the system is numerically stable, in the mean sense, for λ and η between zero and one. It can be shown that the variance of the numerical errors in the forward predictor, with the assumption of a white Gaussian input signal, is stable under the following condition: We note that the lower bound of this condition is always smaller than the lower bound of condition (10) of the original numerically stable FRLS algorithm, which means that we can choose smaller values for the forgetting factor for the proposed algorithm and consequently have faster convergence rate and better tracking ability.

MATERIALS AND METHODS
To confirm the validity of our analysis and demonstrate the improved numerical performance, some simulations are carried out. All plots show the mean squared modeling versus the number of iterations. For the purpose of smoothing the curves, error samples are averaged over 256 points. The forgetting factor λ and the leakage factor η for the M-SMFTF algorithm are chosen according to (34) with the stationary input.

Comparative performances for stationary signals:
We used a stationary correlated noise with a spectrum equivalent to the average spectrum of speech, called USASI noise in the field of acoustic echo cancellation. This signal, with mean zero and variance equal to 0.32, sampled at 16 kHz is filtered by impulse response which represents a real impulse response measured in a car and truncated to 256 samples. We compare the convergence speed and tracking capacity of the proposed algorithm with NS-FRLS and NLMS algorithms. The NLMS (µ = 1) and NS-FRLS (λ = 1-1/3L) algorithms are tuned to obtain fastest convergence. The nonstationarity of the system to be modeled is simulated by introducing a linear gain variation on the desired signal.
The filter length is L = 256, the forgetting factor is (λ = 1-1/L) of the M-SMFTF algorithm. And for RM-SMFTF algorithm, the predictor order is P, the forgetting factor is (λ = 1-1/P). Figure 2 shows that better performances in convergence speed are obtained for the M-SMFTF algorithm. The differences in the final MSE(n) for the M-SMFTF and NS-FRLS algorithms are due to the use of different forgetting factors λ. It is observed that the proposed algorithm converges much faster and tracks better the variation of the system than both NS-FRLS and NLMS algorithms.
In this simulation, we compare the convergence performance of the NLMS algorithm and RM-SMFTF algorithm with different values for the leakage η. Figure 3 presents the results obtained with the stationary USASI noise for the filter order L = 256. We simulated an abrupt change in the impulse response at the 56320th samples. We use the following parameters: The predictor order is P = 40, the forgetting factor is λ = 1-1/P. The convergence speed of RM-SMFTF is much faster than NLMS. We notice, for the RM-SMFTF algorithm, that the more η approaches one and the better the speed.  Comparative performances for speech signals: The input signal used in the simulations is speech signal, sampled at 16 kHz. We compare the convergence speed of the proposed algorithm with NS-FRLS and NLMS algorithms. We simulated an abrupt change in the impulse response at the 56320th samples. The choice of the forgetting factor for NS-FRLS algorithm to ensure numerical stability is λ = 1-1/10L. The forgetting factor for M-SMFTF and RM-SMFTF algorithms are λ = 1-1/L and λ = 1-1/P respectively, but the leakage η and the constant c a must be carefully chosen.
In Fig. 4, we can see that the initial convergence is almost the same for both M-SMFTF and NS-FRLS algorithms. But the M-SMFTF achieves better reconvergence after the abrupt change in the impulse response. In Fig. 5, we observe that the re-convergence of RM-SMFTF is again faster than NLMS.

DISCUSSION
Different simulations have been done for different sizes L and P and all these results show that there is no degradation in the final steady-state MSE(n) of the reduced size predictor algorithm even for P<<L. The convergence speed and tracking capability of the reduced size predictor algorithm can be adjusted by changing the choice of the parameters λ, η and c a . The proposed algorithm outperforms the classical adaptive algorithms because of its convergence speed which approaches that of the RLS algorithm and its computational complexity which is slightly greater than the one of the NLMS algorithm.

CONCLUSION
We have presented a new algorithm M-SMFTF for adaptive filtering with fast convergence and low complexity. We have proposed more complexity reduction of simplified FTF type algorithm by using a new recursive method to compute the likelihood variable. The computational complexity of the M-SMFTF algorithm is 6L operations per sample and this computational complexity can be significantly reduced to (2L+4P) when used with a reduced P-size (P<<L) forward predictor. This can be very interesting for long filters. The low computational complexity of the M-SMFTF when dealing with long filters and it a performance capabilities render it very interesting for applications such as acoustic echo cancellation. The simulation has shown that the performances of proposed algorithm are better than those of the normalized least mean square algorithm.