NISTIR 89–4197
Applied and
Computational
Mathematics
Division
Center for Computing and Applied Mathematics
Orthogonal Distance Regression
Paul T. Boggs and Janet E. Rogers
November, 1989
(Revised July, 1990)
U.S. DEPARTMENT OF COMMERCE
National Institute of Standards and Technology
Orthogonal Distance Regression
∗
Paul T. Boggs
†Janet E. Rogers
‡Abstract
Orthogonal Distance Regresson (ODR) is the name given to the com-putational problem associated with finding the maximum likelihood esti-mators of parameters in measurement error models in the case of normally distributed errors. We examine the stable and efficient algorithm of Boggs, Byrd and Schnabel (SIAM J. Sci. Stat. Comput., 8 (1987), pp. 1052– 1078) for finding the solution of this problem when the underlying model is assumed to be nonlinear in both the independent variable and the pa-rameters. We also describe the associated public domain software package, ODRPACK. We then review the results of a simulation study that compares ODR with ordinary least squares (OLS). We also present the new results of an extension to this study. Finally we discuss the use of the asymptotic covariance matrix for computing confidence regions and intervals for the es-timated parameters. Our conclusions are that ODR is better than OLS for the criteria considered, and that ODRPACK can provide effective solutions and useful statistical information for nonlinear ODR problems.
1. Introduction
Much has been written about measurement error models and their properties. The definitive modern treatment of such problems, which are also known as er-rors in variables and generalized least squares, is given by Fuller [14]. Because
∗Contribution of the National Institute of Standards and Technology and not subject to
copyright in the United States.
†Applied and Computational Mathematics Division, National Institute of Standards and
Technology, Gaithersburg, MD 20899. INTERNET: [email protected]
‡Applied and Computational Mathematics Division, National Institute of Standards and
Technology, Boulder, CO, 80303-3328. INTERNET: [email protected]
Introduction 2 of the complexity of these problems, most of the work in this area has concerned linearmodels. In this paper, however, we are primarily concerned with nonlinear measurement error models and the stable and efficient algorithm of Boggs, Byrd and Schnabel [5] for the numerical solution of the resulting nonlinear optimization problem that we refer to as the orthogonal distance regression problem. We also discuss the implementation of this algorithm in the software library ODRPACK and the use of this software to perform certain simulation studies that help to illuminate the differences between using an orthogonal distance criterion for es-timating the parameters of a model and the use of the standard ordinary least squares (OLS) criterion. (See Boggs et al. [6], and Boggs, Rogers and Schnabel [8].) We conclude with another simulation study (Boggs and Rogers [7]) designed to test the effectiveness of the confidence intervals and regions obtained from the asymptotic covariance matrix.
The measurement error model can be defined in a number of ways. Here we assume that
(Xi, Yi), i = 1, . . . , n,
are observed random variables with underlying true values (xi, yi), i = 1, . . . , n,
where xi ∈ ℜm and yi ∈ ℜ. We also assume that δi is the random error associated
with xi, that ǫi is the random error associated with yi, and that
Xi = xi− δi
Yi = yi− ǫi.
Finally, we assume that yi is given as a function of xi and a set of parameters
β ∈ ℜp, i.e.,
yi = f (xi; β)
or
Yi = f (Xi+ δi; β) − ǫi.
(Throughout this paper, we use bold face to denote the matrix or column vec-tor whose components are the corresponding subscripted variables, e.g., β = (β1, β2, . . . , βp)′, where ′ means transpose.) The function f may be either linear
or nonlinear in its arguments, but must be a smooth function of these arguments. At this point, we make no restrictions on the distribution of x, i.e., we do not specify either a functional or structural model.
Introduction 3 Any procedure that estimates β for a measurement error model should take into account that both Xi and Yi have error. We do this by using an orthogonal
distance to define the distance ri from the point (Xi, Yi) to the curve f (˜x; ˜β),
where ˜ denotes a non-distinguished value of the variable. That is, we take ri2 = min
˜ ǫi,˜δi
{˜ǫ2i + ˜δi2}
subject to: Yi= f (Xi+ ˜δi; ˜β) − ˜ǫi, i = 1, . . . , n.
Note that here, and in the following derivation, we assume that Xi is one
dimen-sional to simplify the notation.
Under the assumptions that ǫi ∼ N(0, σǫ2i) and δi ∼ N(0, σ 2
δi), it follows that
the maximum likelihood estimate ˆβis that which minimizes the sum of the squares of the ri. Thus ˆβ is the solution of
min ˜ β,δ˜, ˜ǫ n X i=1 n ˜ǫ2i + ˜δ2i o (1.1) subject to: Yi = f (Xi+ ˜δi; ˜β) − ˜ǫi, i = 1, . . . , n. (1.2)
Since the constraints (1.2) can be used to eliminate ˜ǫ from (1.1), this constrained problem is equivalent to the unconstrained minimization problem
min ˜ β,δ˜ n X i=1 h f (Xi+ ˜δi; ˜β) − Yi i2 + ˜δ2i .
The final form of the optimization problem is obtained by allowing weights to be associated with the observations. We thus define the weighted Orthogonal Distance Regression Problem (ODR) to be
min ˜ β,δ˜ n X i=1 w2 i h f (Xi+ ˜δi; ˜β) − Yi i2 + d2 i˜δi2 (1.3) where wi ≥ 0 and di > 0. The problem in this form can arise in statistical
applications, and in curve fitting and other applications where the errors and their associated weights have no statistical connotation. In the remainder of this paper, however, we deal with problems where wi = 1/σǫi and di = σǫi/σδi.
In §2 we discuss the stable and efficient numerical algorithm in Boggs, Byrd and Schnabel [5] for solving this problem. This algorithm requires the same work
Algorithm and Software 4
per iteration as the corresponding procedure for solving an OLS problem. The algorithm has been implemented in a Fortran subroutine library called ODRPACK (Boggs et al. [3]) that is publically available. The features of this package are also discussed.
In §3 we review the results of a simulation study (Boggs et al. [6]) that was designed to examine the differences between ODR and OLS solutions. This “baseline” study assumes that the ratios di = σǫi/σδi are known and constant,
and wi = 1 for all i. Under the chosen criteria ODR is shown to be superior to
OLS. We then describe an extended study (Boggs, Rogers and Schnabel [8]) that relaxes the assumption that the di are known exactly. In this case, if the ratios
are known to within a factor of 10, ODR is still preferred.
Fuller [14] shows how to derive the asymptotic covariance matrix for the ODR problem. In §4 we discuss the results of a further simulation study (Boggs and Rogers [7]) designed to ascertain the effectiveness of confidence regions and inter-vals constructed based on this matrix.
The results reported here allow the conclusion that ODR is preferred over OLS when the ratios di are reasonably well known. Furthermore, the existence of
the algorithm and software described in §2 means that solving an ODR problem is just as easy as solving a nonlinear OLS problem, and that useful statistical information can be readily produced.
2. Algorithm and Software
The ODR problem (1.3) has been in the computing literature for several years, and several procedures for its solution have been proposed, cf., Britt and Luecke [9], Powell and MacDonald [15], Schnell [18], and Schwetlick and Tiller [19]. In this section, we review the recent stable and efficient algorithm of Boggs, Byrd and Schnabel [5], and its implementation in ODRPACK. (See Boggs et al. [3] and [4].) The algorithm is based on a trust-region strategy that yields a Levenberg-Marquardt type step. (See, e.g., Dennis and Schnabel [10].) The trust-region approach allows for a much more effective procedure for computing the so-called Levenberg-Marquardt parameter.
To describe the algorithm we again assume that Xi is one dimensional, which
keeps the notation simpler. (The extension to m dimensions is straightforward, and ODRPACK allows this.) We first show that (1.3) can be viewed as an ex-tended OLS problem. We then review the trust-region approach for solving non-linear OLS problems, and show how the special structure of the extended problem
Algorithm and Software 5
can be exploited to produce an efficient scheme. In fact the efficiency is compara-ble to that of a trust-region algorithm for solving the OLS procompara-blem obtained from the assumption that there are no errors in Xi. We conclude this section with a
brief description of ODRPACK. Let gi( ˜β, ˜δ) = wi h f (Xi+ ˜δi; ˜β) − Yi i i = 1, . . . , n gi+n( ˜β, ˜δ) = widiδ˜i i = 1, . . . , n,
and denote the 2-norm of g by
g( ˜β, ˜δ) 2 ≡ 2n X i=1 h gi( ˜β, ˜δ) i2 . Then (1.3) becomes min ˜ β,δ˜ g( ˜β, ˜δ) 2 ≡ min ˜ β,δ˜ 2n X i=1 h gi( ˜β, ˜δ) i2
which is an OLS problem with n + p parameters and 2n observations. Let
˜
θ = β˜˜ δ
!
and denote by J ∈ ℜ2n×(n+p) the Jacobian of
g(˜θ) = (g1(˜θ), g2(˜θ), . . . , g(˜θ)2n) ′ , i.e., Jij = ∂gi ∂ ˜θj .
Then the linearized model at the current iterate, θc, given by the first two terms of the Taylor series, is
g(θc) + J (θc)s ≡ gc+ Jcs (2.1) where
s= ˜θ− θc.
(2.1) can then be used to compute a step, sc, from which the next iterate, θ+, is
Algorithm and Software 6
is the so-called Gauss-Newton step where sc is simply the (linear least squares)
solution of
min s kg
c
+ Jcsk2.
But even when the length of this step is controlled, e.g., by a line search, the resulting algorithm has not proven itself to be competetive with other procedures. (See, e.g., Dennis and Schnabel [10].) If, however, we explicitly recognize that the length of the step sc must be controlled by the extent to which (2.1) is a good
model of g, then a much better procedure arises.
Specifically, the trust-region algorithm chooses the current step sc as the
solu-tion to
mins kgc + Jcs
k2
subject to: ksk ≤ τ (2.2) where τ is the trust-region radius. The value of τ defines the region where (2.1) is a good model of g and should not be thought of in a statistical sense. The value of τ is automatically adjusted at each iteration based on whether kg(θc + sc)k <
kgck and on a comparison of kg(θc+ sc)k with kgc + Jcsck. The actual tests
employed in modern trust-region algorithms have been empirically determined over the years. (See Dennis and Schnabel [10] for a general discussion, and Boggs, Byrd and Schnabel [5] and Boggs et al. [3] for specific details.)
An outline of the trust-region algorithm is as follows.
Given an initial approximation θc and an estimate of the trust-region, τ : 1. Solve (2.2) for sc 2. Set θ+ := θc+ sc 3. Test: If g(θ + ) < kg ck then • set θc := θ+ • adjust τ if necessary • check convergence else • reduce τ 4. Go to Step 1
Algorithm and Software 7
In the construction of portable software that is meant to be used for a variety of problems, it is common to solve (2.2) by first doing a QR factorization of the Jacobian matrix J . (See, e.g., Dennis and Schnabel [10] or Stewart [20].) For a dense n × p matrix the QR factorization requires O(np2) operations. Since J is
2n × (n + p), it therefore requires O(n(n + p)2) = O(n3) operations to compute
its QR factorization. The matrix J has a special structure, however, that can be exploited to solve (2.2) more efficiently. In particular, let g1 denote the first n
components of g and g2 the last n components. Then
J = ∂g1 ∂β˜ ∂g1 ∂δ˜ ∂g2 ∂β˜ ∂g2 ∂δ˜ = ∂g1 ∂β˜ H 0 D where H = diag ( wi ∂f (Xi+ ˜δi; ˜β) ∂˜δi , i = 1, . . . , n ) and D= diag {widi, i = 1, . . . , n} . (2.3)
By exploiting the structure provided by the existence of the zero block and the two diagonal blocks, H and D, the amount of work per iteration required to solve an ODR problem can be reduced from O(n3) to O(np2) operations plus, on average,
less than two evaluations of the model function f . Since there are typically many more observations than parameters, this difference can have a profound influence on the size of problem that can be practicably solved. We point out that if the errors in Xi are ignored and the resulting OLS problem is solved, the amount of
work per iteration required is also O(np2) operations plus an average of less than
two evaluations of f . Using our algorithm, therefore, an ODR problem can be solved as efficiently as an OLS problem. See Boggs et al. [3] for the details.
We conclude this section with a brief description of the Fortran subroutine library, ODRPACK, that we have written to implement the ODR algorithm de-scribed above. A complete description is contained in Boggs et al. ([3] and [4]). The package is portable and has been successfully run on many machines from PCs to supercomputers.
Simulation Studies 8 ODRPACK is designed to solve both OLS and ODR problems, handling many levels of user sophistication and problem difficulty.
• It is easy to use, providing two levels of user control of the computations, extensive error handling facilities, optional printed reports summarizing the iterations and/or the final results, and no size restrictions other than effective machine size.
• The necessary Jacobian matrices are approximated numerically if they are not supplied by the user. The correctness of user supplied derivatives can also be verified by the derivative checking procedure provided.
• Both weighted and unweighted analysis can be performed. Unequal weights wi, i = 1, . . . , n, are allowed, and a different value of dik, i = 1, . . . , n and
k = 1, . . . , m, can be specified for each component of X. A feature has also been incorporated that allows the same weight to apply to many components without having to explicitly enter it more than once.
• Subsets of the unknowns can be treated as constants with their values held fixed at their input values, allowing the user to examine the results obtained by estimating subsets of the unknowns of a general model without rewriting the code that computes f .
• The covariance matrix and the standard errors for the model parameter estimators are optionally provided. (See §4.)
• ODRPACK automatically compensates for problems in which the model parameters and/or unknown errors in the independent variables vary widely in magnitude.
A copy of ODRPACK may be obtained free of charge from the authors.
3. Simulation Studies
Researchers including Fuller [14] and Reilman et al. [17] provide results that give theoretical conditions for choosing ODR over OLS and vice versa. These results, however, are only for straight line models, and, to our knowledge, there are no similar results for either general linear models or for nonlinear models. This state of affairs provides the motivation for a baseline simulation study that compares the
Simulation Studies 9 performance of ODR and OLS on nonlinear models. These results are contained in Boggs et al. [6].
The central assumption in Boggs et al. [6] is that the ratios di = σǫi/σδi
are known exactly. We first review this study, and then present results that extend this work to the case when the di are not known exactly.
The existence of ODRPACK as described in §2 not only makes these studies possible, but also makes them of practical interest. Since this code is publically available, other researchers and practitioners can solve their own ODR problems and conduct similar simulation studies on other models.
The study in Boggs et al. [6] is the first to consider models that are nonlinear in both β and x, although studies involving linear and quadratic models have been reported in Amemiya and Fuller [1], Schnell [18], and Wolter and Fuller [21], for example. Our procedure is to compare the bias, variance, and mean square error of parameter estimates and function estimates obtained using both ODR and OLS. We use four forms of the model function f with two parameter sets per function. We generate errors for each using di = d∗ for seven values of d∗ ranging
from 0.1 to ∞, replicating each problem configuration 500 times. Each of the resulting 91,000 problems is then solved using ODR and OLS. (Recall that the correct values of di are used to obtain the ODR solutions.) Both the ODR and
OLS computations are done by ODRPACK.
To be more specific, we consider a straight line model, a quadratic model, an exponential model, and a sine model. The two true parameter sets for each of these models are chosen so that the maximum slope of f over the range of the X data is 1 and 10, respectively. The xi are taken as 51 equally spaced points on
[-1,1]; thus this study is concerned with a functional model. The errors δ and ǫ are generated as normally distributed and adjusted so that the expected sum of the squared errors is constant over the seven values of d∗
. The values of d∗
that we consider are 0.1, 0.5, 1.0, 2.0, 10.0, 100.0, and ∞, where d∗
= ∞ indicates that there are no errors in the values Xi.
The main conclusion of this study is that ODR is better than OLS for all of the criteria that we consider.
• For parameter bias, ODR is no worse than OLS in 98% of the cases studied and is a clear winner 50% of the time.
• For parameter variance and mean square error, ODR is no worse in 98% of the cases and is appreciably better in 23%.
Simulation Studies 10 • Similar results are obtained for the function estimates.
• For the straight line model, we do not see a preference for OLS over ODR, as is predicted by Reilman et al. [17]. ODR produces a significantly lower bias than OLS, but the variance and mean square error values are only slightly smaller for OLS. We thus conclude that ODR is still preferable in this case. • In accordance with the observations of Wolter and Fuller [21] and Anderson [2] our computational results do not reflect the infinite moments of ODR. In practice, the ratios di are seldom known exactly. Thus it is of interest to
extend the above results to the case where an approximate value of each di is used
in the ODR procedure. The results of such a study are presented next.
To make this second Monte Carlo study tractable, we reduce its scope some-what. We consider the same seven functions as described above, but with only one parameter set per function. For this study, the ratios used to compute the generated errors are d∗
equal to 0.1, 1.0, and 10.0, and the values of the error ratios that are used by ODRPACK in estimating the parameters are di = γd∗,
where γ has the values 0.1, 0.5, 1.0, 2.0 and 10. That is, we investigate cases where the error ratio used in the estimation procedure is correct to within a factor of 10. We again compare the ODR results to an OLS fit using the same criteria as in the baseline study reported above.
We give a complete description of the study and the results in Boggs, Rogers and Schnabel [8]. In summary, when the di are correct to within a factor of 10.0,
then:
• The bias of the parameter estimates using ODR is no worse than the bias using OLS in 88% of the cases studied, and is significantly smaller 59% of the time.
• The variance of the parameter estimates using ODR is no worse in 90% of the cases and is smaller in 40%.
• The mean square error of the parameter estimates using ODR is no worse in 87% of the cases and is smaller in 50%.
Similar results are observed for the function estimates. In addition, there appears to be some support for overestimating rather than underestimating the di.
We conclude from these two studies that if the di are known to within a factor
The Asymptotic Covariance Matrix 11
how estimated confidence regions may break down if the di are not known to
within a factor of 2.
4. The Asymptotic Covariance Matrix
In many parameter estimation problems, confidence regions and/or confidence intervals for the estimated parameters are needed. In the linear OLS case, these can be obtained exactly. In the nonlinear OLS case and in ODR, the computation of exact confidence regions and intervals is computationally intractable, and so approximate methods must be used.
Fuller [14] derives the asymptotic form of the covariance matrix for the esti-mated parameters of measurement error models. This can be used to construct approximate confidence regions and intervals. In Boggs and Rogers [7] we discuss the efficient computation of this covariance matrix in the context of the algorithm and software of §2. Using a Monte Carlo simulation study, we then assess the quality of confidence regions and intervals computed from this covariance matrix. In this section we review that work.
The asymptotic covariance matrix can be defined using the extended OLS problem of §2, namely, min ˜ θ 2n X i=1 gi(˜θ)2 (4.1)
and the corresponding Jacobian matrix J ∈ ℜ2n×(n+p) with (i, j) element defined
by
Jij =
∂gi
∂ ˜θj
. Let ˆθ be the estimate of θ from (4.1) and let
ˆ σ2 = g(ˆθ) ′ Ωg(ˆθ) n + p where Ω = W 2 0 0 D2 ! , W = diag {wi, i = 1, . . . , n} ,
The Asymptotic Covariance Matrix 12
and D is given by (2.3). Also, assume that Ω correctly reflects the distribution of the errors ǫ and δ as specified in §1. Then the covariance matrix is
ˆ
V = ˆσ2[J′
ΩJ]−1,
which Fuller [14] proves is asymptotically correct as the error variance tends to zero.
The matrix ˆV can be used to construct approximate confidence regions and intervals for the estimated parameters ˆθ. We call this method the linearization method since it is based on the assumption that g is adequately approximated by a linear function of the form (2.1) in the neighborhood of ˆθ. The goodness of the resulting linearized regions and linearized intervals depends on the nonlinearity of f and the size of ˆσ2. Donaldson and Schnabel [11] show that the linearized
con-fidence intervals for nonlinear OLS problems appear to be reasonable in practice, but that the linearized confidence regions can be inadequate. (More accurate, but comensurately more expensive approximations can be obtained, as described by, e.g., Donaldson and Schnabel [11] and Efron [13].)
To assess linearized confidence regions and intervals in the context of ODR problems, we conduct a Monte Carlo simulation similar to that in Donaldson and Schnabel [11]. Our study consists of four example problems. Three of these problems exist in the literature. They are Fuller [14, Example 3.2.2], Ratkowsky [16, Example 6.11], and Draper and Smith [12, Chapter 10, Problem E]. The fourth example is taken from a consulting session involving a “psychophysical” experiment evaluating the ability of human subjects to perceive a visual signal as a function of the intensity of the signal.
For each example we generate 500 sets of “observed” data with normal random error and true error ratios d∗
i. We then solve each of the resulting problems using
ODRPACK. As in the second study reported in §3, this study also solves each problem using a range of assumed error ratios di = γd
∗
i, where γ = 0.1, 0.5, 1.0,
2.0, and 10.0. For each of the 500 data sets, we also construct the 95% confidence regions and intervals for the estimate ˆθ using the linearization method. Finally, we calculate the observed coverage, which is the percentage of cases in which the true value is actually contained in the computed region or interval. Our results, presented in Boggs and Rogers [7], are summarized as follows:
• When the di are correct, we obtain surprisingly good estimates of the
con-fidence regions and intervals as compared to the results of Donaldson and Schnabel [11], who often found the observed coverage for a nominal 95%
References 13 confidence region to be less than 80%. We assume that this is due to our choice of examples and is not a general property of ODR.
• The confidence region estimates significantly degrade, even when the esti-mated values of di are off by no more than a factor of 2.
• The confidence intervals when the di are correct are also quite good and
remain good when the di are only off by a factor of 2.
• The confidence intervals significantly degrade when the di are off by a factor
of 10.
• Confidence regions and intervals are better for ˆβ than for ˆδ.
• In the case of confidence intervals for ˆβ, overestimation of the di is preferable
to underestimation. For ˆδ the reverse is true. This occurs because, as the di
are increased, ˆδ becomes more restricted, whereas as the di are decreased,
an artifically small value of ˆσ is produced.
Note that ˆV is the covariance matrix for the entire parameter vector ˆθ. In practice, however, one is typically only interested in the covariance matrix for the model parameters ˆβ. This is the upper left p × p part of ˆV, which we denote ˆVβ.
Just as the special structure of J allows us to create a fast algorithm for solving (2.2), that same structure permits us to equip ODRPACK with an efficient and stable means of calculating ˆVβ. The formulas are contained in Boggs and Rogers
[7] and are based on the work in Boggs, Byrd and Schnabel [5].
The overall conclusion from this third study is that the linearized confidence regions and especially confidence intervals have some validity for ODR problems when the di are known to within a factor of 2. Furthermore, the computations
needed to obtain these linearized regions and intervals for ˆβ are cheap, and are automatically produced by ODRPACK. Thus we advocate the use of linearized confidence regions and intervals for measurement error models in the same spirit, and with the same caveats, that accompany their use in nonlinear OLS problems.
Acknowledgement. The authors gratefully acknowledge our colleagues R. H. Byrd, R. B. Schnabel, and C. H. Spiegelman for their part in the work that is reported here. We also thank K. R. Eberhardt, H. K. Iyer and G. W. Stewart for many useful discussions on this topic.
References 14
References
[1] Y. Amemiya and W. A. Fuller, Estimation for the nonlinear functional rela-tionship: errors-in-variables model, Report to the U. S. Bureau of the Census on J.S.A. 85-1, Iowa State University, Ames, Iowa, 1985.
[2] T. W. Anderson, Estimation of linear functional relationships: approximate dis-tributions and connections with simultaneous equations in econometrics, Journal of the Royal Statistical Society (Series B), 38 (1976), pp. 1–36.
[3] P. T. Boggs, R. H. Byrd, J. R. Donaldson, and R. B. Schnabel, ODR-PACK — software for weighted orthogonal distance regression, ACM Transactions on Mathematical Software, 15 (1989), pp. 348–364.
[4] , User’s reference guide for ODRPACK — software for weighted orthogonal distance regression, Internal Report 89-4103, National Institute of Standards and Technology, Gaithersburg, Maryland, 1989.
[5] P. T. Boggs, R. H. Byrd, and R. B. Schnabel, A stable and efficient algo-rithm for nonlinear orthogonal distance regression, SIAM Journal of Scientific and Statistical Computing, 8 (1987), pp. 1052–1078.
[6] P. T. Boggs, J. R. Donaldson, R. B. Schnabel, and C. H. Spiegelman, A computational examination of orthogonal distance regression, Journal of Econo-metrics, 38 (1988), pp. 169–201.
[7] P. T. Boggs and J. E. Rogers, The computation and use of the asymptotic covariance matrix for measurement error models, Internal Report 89-4102 (Revised 1990), National Institute of Standards and Technology, Gaithersburg, Maryland, 1989.
[8] P. T. Boggs, J. E. Rogers, and R. B. Schnabel, A second Monte Carlo study of weighted orthogonal distance regression, Internal Report (in progress), National Institute of Standards and Technology, Gaithersburg, Maryland, 1991.
[9] H. L. Britt and R. H. Luecke, The estimation of parameters in nonlinear implicit models, Technometrics, 15 (1973), pp. 233–247.
[10] J. E. Dennis and R. B. Schnabel, Numerical Methods for Unconstrained Op-timization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, New Jersey, 1983.
References 15
[11] J. R. Donaldson and R. B. Schnabel, Computational experience with confi-dence regions and conficonfi-dence intervals for nonlinear least squares, Technometrics, 29 (1987), pp. 67–82.
[12] N. R. Draper and H. Smith, Applied Regression Analysis, Second Edition, John Wiley and Sons, New York, New York, 1981.
[13] B. Efron, The Jackknife, the Bootstrap and Other Resampling Plans, Society for Industrial and Applied Mathematics, Philadelphia, Pennsylvania, 1985.
[14] W. A. Fuller, Measurement Error Models, John Wiley and Sons, New York, New York, 1987.
[15] D. R. Powell and J. R. MacDonald, A rapidly convergent iterative method for the solution of the generalized nonlinear least squares problem, Computer Journal, 15 (1972), pp. 148–155.
[16] D. A. Ratkowsky, Nonlinear Regression Modeling, Marcel Dekker, New York, New York, 1983.
[17] M. A. Reilman, R. F. Gunst, and M. Y. Lakshminarayanan, Stochastic regression with errors in both variables, Journal of Quality Technology, 18 (1986), pp. 162–169.
[18] D. Schnell, Maximum likelihood estimation of parameters in the implicit nonlin-ear errors-in-variables model, Master’s thesis, Iowa State University Department of Statistics, Ames, Iowa, 1983.
[19] H. Schwetlick and V. Tiller, Numerical methods for estimating parameters in nonlinear models with errors in the variables, Technometrics, 27 (1985), pp. 17–24. [20] G. W. Stewart, Introduction to Matrix Computations, Academic Press, New
York, New York, 1973.
[21] K. M. Wolter and W. A. Fuller, Estimation of nonlinear errors-in-variables models, The Annals of Statistics, 10 (1982), pp. 539–548.
