1 Introduction

The Least Square Error (LSE) is usually used for finding ”the best fit” of measured data, which leads to a solution of an over-determined system of linear equations Ax=b. The LSE method is very often used in financially oriented applications using linear and non-linear regressions. In some specific cases, the total least square method is to be used, mostly related to the implicit representation [¹, ⁶, ⁹, ¹⁰]. However, the LSE method’s result depends on the physical units of the data domain used in the polynomial regression case.

2 Over-Determined Systems

In the case of the over-determined linear system, the matrix A is (n××m), n>>m, the vector b is (m××1), the LSE is usually used to obtain an approximate solution. However, in many cases, users are not aware of the LSE properties [¹⁷]. It is well known, that a result of the LSE approximation depends on physical units used, if polynomial regression is used, e.g. in the estimation of processing time, etc. Let us consider a regression function φφ(t):

If the time unit [s]] is used, the results are different from the case, when the unit [ms]] is used. Also, the element a0, which represents a value for t=0, causes some problems; detected also in interpolation and approximation using Radial Basis Function (RBF) [², ⁵, ¹³, ¹⁴].

In the case of the linear regression, the LSE method is usually applied directly to the data set using pseudo-inverse as follows:

Let us consider the LSE formulation as in Eq. 1, but modified for an over-determined system of linear equations. Then the LSE use leads to:

where P(n××n) and D(m××m) are non-singular diagonal matrices. Using algebraic operations:

As the matrix D is diagonal and non-singular, it is possible to multiply Eq. 5 from the left by (DT)−−1. It results to:

where Q=PTP is a diagonal matrix of pi2 row multipliers. If ξξ=D−−1x, then Eq. 6 can be rewritten as:

then the solution of Eq. 7 using LSE method:

Therefore in the case of linear regression, the LSE method, Eq. 3:

3 Example

Let us consider two simple examples of the LSE use for two different simple cases with a modification, when the first row of the extended matrix [A||b]] is multiplied by the value 10:

These elementary examples serve to understand the limitations of the LSE use. The results are valid for d-dimensional space, in general. In the first case, usually, users are not aware of that. The second case can be easily understood as the k represents a normal vector generally in a higher dimension, while q is related to a distance from the origin. The solution of Eq. 3 might be unstable, as the matrix AT  A is generally numerically ill-conditioned [³, ¹¹, ¹³, ¹⁴].

It should be noted, that in many cases the Total Least Square Error (TLSE) should be used instead. However, it leads to more complicated computation [⁹, ¹⁶].

A simple preconditioning [¹², ¹⁵] should be considered. Also, the modified Gauss elimination method [⁷, ⁸] can be used as a solution of a linear system is equivalent to the outer product use.

4 Conclusion

This contribution describes selected mostly un-known properties of the Least Square Error for the approximation of acquired data. The LSE method is non-invariant to the multiplication of a row of the extended matrix [A||b]]. Also, in the case of non-existent metric between parameters, like a distance and a normal vector, the LSE based approximation should not be used.