**5.4 Experimental example**

**5.4.2 A check for the derivatives s 0 (t), s 00 (t), and s 000 (t)**

It is desirable to perform a check on the derivatives of the smoothing spline, which I can do by comparing them to estimates made from the noisy experimental data. For this, we need a regression technique which behaves like a non-parametric model -- one in which the fitting parameters are free to change along the length of the curve. As a check of the first derivative, a line may be fit to a small window of data using least squares regression. The slope of this line represents the ‘slope’ of the data at the center of the window3. Mathematically, to find the first derivative of ˜y(t) data at time, ti, fit a line (a1t + a2) to the data within the window [ti−w, ti+w]. The width of

the window is 2w + 1 data points, where a larger w yields more smoothing of the data but a less localized estimate. The first derivative of this linear polynomial (namely a1) is the estimate of the first derivative at time, ti. This process would be repeated

with the window centered at each tw+1 ≤ ti ≤ tN −w to obtain the derivative estimate

for each time. Since this procedure involves performing a least squares fit to a small window of data, I call this windowed least squares (WLS).

Higher order derivatives can also be estimated using windowed least squares. At each discrete time, a least squares linear polynomial fit gives an estimate of the first derivative at that time, a quadratic polynomial fit gives an estimate of the second derivative, a cubic polynomial fit gives an estimate of the third derivative at that time, and so on. The windowed least squares fit types and derivative estimates are summarized in table 5.1, and the estimates of the first and second derivatives are shown in figure 5-10. These data agree quite well with the derivatives of the smoothing spline, as expected.

The windowed least squares method provides a good estimate of the derivatives of the function, because the general trend of the data surrounding each point is captured by the least squares regression technique. However, this method does not ensure that

Table 5.1: Windowed least squares estimates of the first, second, and third derivatives of noisy ˜y(t) data.

windowed least squares fit derivative estimate linear: a1t + a2 ywls0 (t) ≈ a1 quadratic: a1t2+ a2t + a3 ywls00 (t) ≈ 2 · a1 cubic: a1t3+ a2t2+ a3t + a4 ywls000 (t) ≈ 3 · 2 · a1 0 0.05 0.1 0.15 0.2 −6 −5 −4 −3 −2 −1 0 finite difference windowed least squares 3rd order polynomial 7th order polynomial smoothing spline time [s] velocity [m/s] 0 0.05 0.1 0.15 0.2 0 10 20 30 40 50 finite difference windowed least squares 3rd order polynomial 7th order polynomial smoothing spline time [s] acceleration [m/s 2] (a) (b)

Figure 5-10: Velocity, y0(t), and acceleration, y00(t), computed by: finite difference, ‘’; windowed least squares, ‘•’; third-order polynomial least squares fit to the entire data set, ‘-·-’; seventh-order polynomial least squares fit to the entire data set, ‘--’; and the selected smoothing spline, for which E = Ecr, ‘–’.

the derivative is a smooth function as the window is moved along the data set. It also fails to predict the derivative near the ends of the data interval (ti < tw+1 and

ti > tN −w), since the window would then extend beyond the interval of available data.

Two less accurate methods for estimating the derivatives are also shown in figure 5- 10: least squares regression to the entire data set, and finite differences. The derivatives of a least squares regression to all the data are inherently questionable, because the fitting parameters depend on the entire data set. Clearly, one cannot assume that the dynamics of our billiard ball during early times (e.g. during cavity formation) are the same as the dynamics during later times (e.g. after cavity collapse). Fitting a single polynomial to all of the data implicitly demands that the physics at

all times be the same, which is clearly not true in this experiment.

It would be appropriate to fit a polynomial to all of the data (using least squares) if the physics were the same throughout the experiment and the form of the true function is known (e.g. a quadratic polynomial fit to position data of a ball falling in a vacuum). However, if the form of the true function is unknown (which is usually the case in scientific research), then this method can give misleading results. For example, both 3rd-order and 7th-order polynomials fit well to all of the position data in the billiard ball example problem. However, their second derivatives are quite different, and neither agrees with the smoothing spline prediction or windowed least squares estimate (see figure 5-10b). From the present smoothing spline approach, it is clear that the acceleration of the sphere is not linear throughout its fall. The 7th order fit at least gives a closer approximation of the acceleration than the 3rd order fit, which (in spite of it implying a linear acceleration) is all too often used in these types of experiments.

Finite difference methods amplify measurement noise, yielding poor estimates of derivatives. For example, the central divided difference formula predicts

d˜y(ti)
dt =
˜
yi+1−˜yi−1
24t + O(4t
2_{)}

= y(ti+1)−y(ti−1)

24t +
˜
i+1−˜i−1
24t + O(4t
2_{)}
= dy(ti)
dt + O
4t
+ O (4t2) (5.4.2)

where O( ) denotes the order of magnitude of the error in the prediction. For a small timestep, 4t 1, the measurement error, , is amplified. The noise is amplified again upon taking each successive derivative, yielding derivatives with unsatisfactorily-large error on the order of

d˜y
dt ∼ O
4t
, d
2_{y}_{˜}
dt2 ∼ O
4t2
, d
3_{y}_{˜}
dt3 ∼ O
4t3
, . . .

Similarly, all finite difference methods amplify measurement noise, even when a larger
time step is used4_{. This error amplification is quite noticeable in the acceleration}

estimates in figure 5-10b.

### 5.5

### Conclusions

I have shown that performing data regression using smoothing splines is the best method for predicting instantaneous derivatives of noisy experimental data. It agrees well with the windowed least squares method, which is a good means to approximate these derivatives. Other methods, such as finite differences or fitting polynomials to the entire data set yield poor estimates.

Finding the derivative of noisy data amounts to fitting an analytic curve that best approximates the true function that the data represents. The Matlab function spaps(t, ˜y, E) fits a smoothing spline to given ˜y(t) data, with minimum roughness and error at most equal to E. I have presented a novel and robust method for selecting the value of the error tolerance, E, that produces the ‘best’ spline fit, one which follows the roughness of the true function but does not introduce roughness due to measurement error.

My method is based on two critical insights. First, by systematically exploring the R(E) relationship implicit in the ‘Reinsch problem’, I discovered that the R(E) frontier has a kink at a critical error tolerance, Ecr. Second, I showed both graphically

and with scaling arguments that Ecrcorresponds to the spline with the minimum error

to the data possible without introducing roughness due to the noise in the data. In
my analytical example, I also showed that the spline corresponding to Ecr has nearly
4_{Even if n timesteps are skipped on either side of the data point, the central difference formula}

predicts

d ˜y(ti)

dt =

y(ti+n) − y(ti−n)

2n4t + O

_{}

n4t

+ O n24t2 which may never have satisfactorily-small error.

the minimum possible predictive error, P , which supports my claim that choosing an error tolerance of Ecr produces the best possible spline fit.

The critical error tolerance, Ecr, corresponds to the point on the R(E) frontier

with the maximum positive curvature (in log space). I automate finding Ecrfor a given

data set by using the double-bisection procedure developed herein. For experimental measurements with high-precision (small ) and high-resolution (large N ), my method robustly fits the data and yields the desired derivatives.

One extension of this work is to apply my methodology to two-dimensional data (e.g. measurements made along two spatial dimensions or measurements made along one spatial dimension over several timesteps). My method can also be extended to more complicated types of smoothing splines (e.g. with non-uniform knot locations, or with non-uniform weighting on the roughness). Examining a roughness versus ‘fitting parameter’ frontier, however, will remain as the hallmark of my methodology. With the advent of high-speed, high-resolution imaging and data acquisition systems, researchers are able to acquire data with high temporal and spatial resolution, at very high precision. My method can be used to very-accurately regress these data and compute their first few derivatives.