rectly extendable to ELISAs (25). The most commonly used models include linearizing functions, which attempt to transform

(1)

0095-1137/91/071439-08$02.00/0

Comparisons of Standard

Curve-Fitting

Methods To

Quantitate

Neisseria

meningitidis

Group A

Polysaccharide Antibody

Levels

by

Enzyme-Linked Immunosorbent Assay

BRIAN D.

PLIKAYTIS,l

SUSAN H. TURNER,2* LINDA L. GHEESLING,2 ANDGEORGE M. CARLONE2

Biostatistics and Information ManagementBranch' and Molecular Biology Laboratory, MeningitisandSpecial Pathogens Branch,2 Division of Bacterial andMycoticDiseases, CenterforInfectiousDiseases,

Centersfor Disease Control, Atlanta, Georgia 30333

Received 11 February 1991/Accepted 17 April 1991

Weexamined several ofthemorecommonly used models (log-log,twoformsof thelogit-log, andthe four-parameter logistic-log transformations) for forming standard or calibrationcurves by usinga standardized enzyme-linked immunosorbent assay (ELISA). Assay range, accuracy, and error for each function were measured and compared. AntibodylevelstoNeisseria meningitidisgroupApolysaccharidewereestimated by calculating antibodyconcentrations ofa serially diluted standard reference serumof known concentration. Each functionachievedahighsquaredcorrelation coefficient (r2> 0.97), indicatingahigh degree ofaccuracy informing the standard curves. However, when predicted antibody concentrations werecomparedwith the known values, thelog-logfunction exhibitedthe leastprecision, withextremepercentagesoferroroccurring atseveral dilutions. Apartially specifiedlogit-logtransformationperformedbetter than thelog-logmodelover a reduced rangeof standarddilutions. Thisindicated thata highr2 alonewasnotareliablemeasureofthe

accuracyof thestandardcurve.Ofthemethods surveyed,thelogistic-log and fullyspecified logit-log functions werethemostaccuratemodelsfor formingstandardcurvesand forinterpolating antibody concentrations from the standard curve. Theaccuracy of the fully specified logit-log function ishighlydependent on theprecise specificationof twounknownquantities, the optical densitiesatzeroand infiniteconcentrations, priortofitting themodel toatypicalsetofcalibration data. Thefour-parameter logistic-logfunctionwasthepreferredchoice forquantitatingN. meningitidisgroup Atotalpolysaccharide antibody by using astandardized ELISA. The function does notrequireprespecificationofanyparametersbeforeestimatingthe standardcurve,and thefour parameters are readily interpretable in terms of identifiable physical quantities. This model also has the advantage that it is easiest to visualize since it does not incorporatecomplex transformations of the optical densityscale.

Thedevelopment ofbacterialpolysaccharidevaccineshas created a needfor the accurate assessment of human anti-body levels. The enzyme-linked immunosorbent assay

(ELISA) is being used with increased frequency for the quantitation of human antibody response to a variety of polysaccharide-containing vaccines (2, 3, 5, 9).TheELISA has proven to be useful in the determination ofantibody levels inserafromadultsimmunizedwith bivalent meningo-coccalgroupA and Cpolysaccharide vaccine (3).

Several mathematical models have been developed to describe standard orcalibration curvesfor usein

quantita-tiveradioimmunoassays(24, 25)andELISAs(6, 14, 15, 19). The method usedtoformthe calibrationcurve dictates the working range of the assay and the overall accuracy of calculating concentrations of antibody in patients (24-26). There isno consensuson whichcurve-fitting methodtouse routinelyforthese immunoassays. However, somemethods describe the data with greaterprecision than others (6, 14, 15). Inmany reports, themethod usedfor determination of antibody levels is not given, sojudgments on its accuracy cannotbemade.

Standardcurvesinbothradioimmunoassays andELISAs exhibitasigmoidalorSshapewhen theoptical density (OD) is plotted as a function oflog dilution. Consequently, the techniques employed in radioimmunoassay analysis are

di-*Correspondingauthor.

rectlyextendabletoELISAs(25).Themostcommonlyused modelsincludelinearizing functions,which attemptto trans-form the sigmoidal curve into a straight line (log-log and logit-log transformations) (7, 19), and nonlinear functions, which fitthe sigmoidalcurve directly withoutfurther trans-formations of the OD scale (logistic-log functions) (10, 22, 24).

The squared Pearson correlation coefficient (r2 value) is routinelyusedasone measuretogaugetheadequacyof fit of themodeltothedata (1). However, examiningthe r2alone

mayleadtodeceptivelyoptimisticconclusionsregardingthe

accuracyof the calibrationcurve inthe absence of

compar-ing calculated and true values ofantibody concentrations. This is especially true for the linearizing functions. While highcorrelations may indicate that thetransformations are

working well, comparisons of known antibody concentra-tions with their predicted values from the standard curve

mayrevealseveralunacceptably highpercentagesoferrorat

givenserumdilutions(18). Theinconsistencies between the highr2 and theinflated percentages oferror aredue tothe fact that the r2 is calculated in the transformed OD-log dilution scale anddoesnotoffer reliable information regard-ingtheadequacyof the fit of the model whenconsideringthe

accuracyofcalculated antibodyconcentrations.

Onemethod formeasuringbothassayaccuracyanderror is to serially dilute a sample of known concentration and interpolate concentrations from the calibration curve. The concentrations should remainfairly constant for each

dilu-1439

on March 10, 2021 by guest

http://jcm.asm.org/

(2)

tion and closely approximate the known concentration.

Alternatively, an unknown patient sample may be serially diluted and antibodyconcentrationsmay becalculatedfrom the standard curve at each dilution. In this instance, the

absolute concentrationis unknown, but the calculated con-centrations should be approximately equal.

In this study, we compared the Neisseria meningitidis

group A antibody level in a serially diluted standard refer-ence serum by use offour commonly used mathematical

models (log-log, partially specified logit-log, fully specified

logit-log, andfour-parameter logistic-log). We examined the adequacy of each of the functions in forming standard curves. The assay accuracy, range, and error with which

sera of known concentrations are interpolated from the curves were measured for each of the models. We present

evidencethat thelogistic-logfunction gives the mostreliable

measure of antibody levels over the widest range of serum dilutions.

MATERIALS ANDMETHODS

ELISA. The ELISA wasdone bythemethod of C.Frasch,

Center for Biologics Evaluation and Research, Food and DrugAdministration (11). The wells were coated at a final polysaccharide concentration of0.5 ,ug/ml in coating buffer

(10mM phosphate-buffered saline, pH 7.4) at 28°C for 6 h. The PB-2 standard (Center for Biologics Evaluation and Research, Food and Drug Administration) was serially di-luted, in triplicate, in the ELISA plate wells starting at a 1:75

dilution and was incubated

overnight

at 4°C. The phos-phatase-labeled affinity-purified goatantibodyto human

im-munoglobulin (Ig) G, IgA, and IgM conjugate (Kirkegaard

and Perry Laboratories, Inc., Gaithersburg, Md.) was di-luted 1:2,000 and incubatedat 28°C for 2 h.

Statistical methods. Several mathematical models were used toformcalibrationcurvesin thisstudy, including (i)the

four-parameter logistic-log model, (ii) the fully specified

logit-log model, (iii) the partially specified logit-log model, and(iv)the log-log model.

(i) Four-parameter logistic-log model. This function has beendetailed extensively(10,20, 22, 24) andis employedin a wide variety of scientific disciplines. Standard curves

commonly display a pronounced sigmoidal shape when

plotted on an OD versus log dilution scale. The logistic function adopts the same generalshape and is a reasonable relationship to use in modeling standard curves. It is

de-scribedasOD =d+{(a-

d)I[1

+

(dilution/c)b]}.

Parameters a and d represent theupper and lower asymptotes,

respec-tively, ofthecurve andcorrespondto thetheoretical ODof

the assay atinfinite andzeroconcentrations, respectively. c is the dilution associated withthe pointofsymmetry ofthe

sigmoidandis located at themidpoint ofthe assayfound at theinflection pointof the curve. b is a curvature parameter and is related to the slopeofthe curve.

(ii) Fully specified logit-log model. Rodbard and Hutt (22) described an algebraically equivalent expression for the

logisticfunction whicheffectivelylinearizesthe relationship

when plotted on a logit OD-log dilution scale, i.e., logit

(OD)fS=a +blog(dilution),where a and b are the yintercept

andslope, respectively, for the line.

Inthisexpression,thefully specified(fs)logit of the OD is equal to

logit(OD)f,

= log[(OD -

ODmin)/(ODmax

- OD)].

ODmin

andODmaxareunknown quantities and correspond to the ODs at zero and infinite concentrations, respectively.

ODmin

and

ODmax

are theoretically equal to the lower and upper asymptotes, respectively, ofthe logisticfunction and

areequivalenttod anda,respectively, in that model.These twoparameters must beestimated

prior

to

performing

the

logit transformation and generating the standard curve

by

using this function.

(iii) Partially specified logit-log model. A reduced formof the fully specified logit-log model may be derived by ex-pressingtheODat agivendilution intermsoftheproportion of the OD at infinite concentration (19), i.e., y =

(OD/

ODmax), where

ODmax

is the OD atinfinite concentration.

Using the traditional definition for thelogit ofa

proportion,

p,results in theequation logit(p) = log[p/(1.0-p)],and,by substituting the proportion ofthe given OD to the OD at

infinite concentration (y) forp, this equation is rewritten as

logit(y)=

log{(OD/GDm.)/[1.0

-

(OD/ODmax)I}.

Fundamen-tal algebra reduces this expression to a definition for the

partially specified (ps) logit, i.e.,

logit(OD)P,

=

log[OD/

(ODmax

-

OD)].

This maybethought ofas areducedform

of the fully specified logit-log model derived by assuming

that theODcorrespondingtoazeroconcentration should be 0, resulting in the

ODmin

parameter being set equal to 0. Plotted on a logit OD-log dilution scale, this leads to the

linearexpression

logit(OD)ps

= a + blog(dilution), wherea and b are the y interceptand slope, respectively, for the line. (iv) Log-log model. The log-log model is a simple linear function relating the log of the ODtothelogof the

dilution,

i.e., log(OD) = a + blog(dilution), where a and b arethe y

intercept and slope, respectively, for theline.

The standard reference serum was serially diluted, in

triplicate, eight times. With all models examined, the repli-cates associated with the most dilute sera were discarded because the ODs associated with these samples rarely

ex-ceeded background levels. This resulted in all assays

con-sisting of a seven-point dilution series. ODsfrom the three

replicates for each dilution were averaged to generate the points used to form the standard curves. Since the magni-tudes of

ODmax

and ODmin were unknown for a particular assay, the parameters a and d from the four-parameter

logistic-log fit were used for the fully specified logit-log function. An alternative method wasused toestimate

ODmax

forthepartially specified logit. The averageOD

correspond-ing to the highest concentration of an assay was used to approximate

ODmax

for this function, eliminating that dilu-tion and resulting in a six-point assay for this model. The parametersforthe logistic-log function were determined by using the Gauss-Newton method of estimation (8a). The parameters for the logit-log and log-log functions were calculated by using the method of least squares (8). All regression parameters were estimated by using procedures NLIN and REG in the SAS system of statistical software (27). All computations were made by use of a Compaq

Deskpro 386s/20 personal computer equipped with a math coprocessor (Compaq Computer Corp., Houston, Tex.).

Serum of known concentration (4,800 antibody units (U) per ml) was serially diluted eight times in duplicate. The

duplicateODswere averaged,and calculated concentrations were interpolated from each of the standard curves. The duplicates associated with the lowest concentration were discarded since they rarely exceeded background levels. Each ELISA platecontained fourreplicates of these sequen-tial dilutions.

RESULTS

Calculation ofantibody concentrations forstandard curve. A typical set of calibration data (standards data) plotted on an OD-log dilution scale is shown in Fig. 1A. The data

on March 10, 2021 by guest

http://jcm.asm.org/

(3)

A

0D 2.0 _.' 1.0 CT) o 0.0

-aJ

() -1.0. 0 a) -2.0

CL)

c -3.0

43

J.L -4.0 -3.7 -3.4 -3.1 -2.8 -2.4 -2.2 -1.9 Log Dilution C 0 0) 0) -J -3.7 -3.4 -3.1 -2.8 -2.4 -2.2 Log Dilution 1.0 0.0 -1.0. -2.0. -3.0

-B

-3.7 -3.4 -3.1 -2.8 -2.4 -2.2 -1.9 Log

Dilution

D

-3.7 -3.4 -3.1 -2.8 -2.4 -2.2 -1.9 Log Dilution

FIG. 1. Graphs displaying fourstandardcurvesfittoatypicalsetof calibration data fromaserially diluted standard referenceserum(PB-2; 4,800 U/mi). Mathematical models correspondingtoeachcurve arethe four-parameter logistic-log (A), the fully specified logit-log (B), the partially specified logit-log (C), and the log-log (D) models.

display the characteristic sigmoidal shape and are overlaid with thestandard curveestimated by using the four-param-eterlogistic-logfunction. This standardcurvedescribed the data withahigh degree ofaccuracy,asindicated by the low percentages oferror at each dilution (Table 1). This error wasless than 5.0% forany dilutionacrossthe entireassay, andther2 was0.999. Themeancalculated concentration for the seven dilutions (4,862 U/ml) closely approximated the known concentration of 4,800 U/ml (1.3% error), and this estimatewasstableacrossalldilutionsasevidenced byalow

standard errorof the mean(SEM) of 56U/ml and a coeffi-cient of variation(CV) of3.0.

Figure 1B shows the same calibration data with the standard curve estimated by using the fully specified logit-log function. Since theODminandODmaxneededtoformthe

logit transformation were unknown, d and a, as estimated from the four-parameter logistic-log function, were substi-tuted fortheseparameters. The results of this fitaredetailed in Table 1. As with the logistic-log function, the fully specified logit-logfunction fit the standards data withahigh

TABLE 1. Calculated antibodyconcentrations forstandardsdatapoints byusingfour models toform standard curvesa

Logistic-log Fully specified logit-log Partiallyspecified logit-log Log-log

Logdilution Calculated concn %Error" Calculated concn % Error Calculated concn % Error Calculated concn % Error

(U/mi) % ro' (U/mi) %Err (U/mi) %Err (U/mI) Ero

-3.7 4,962 3.4 4,777 0.5 5,426 13.0 3,845 19.9 -3.4 5,036 4.9 4,891 1.9 4,746 1.1 4,476 6.8 -3.1 4,915 2.4 4,813 0.3 4,396 8.4 5,344 11.3 -2.8 4,769 0.7 4,708 1.9 4,284 10.8 6,078 26.6 -2.4 4,675 2.6 4,655 3.0 4,448 7.4 6,069 26.4 -2.2 4,988 3.9 5,012 4.4 5,671 18.2 5,171 7.7 -1.9 4,692 2.3 4,752 1.0 NAC NA 3,347 30.3 Mean + SEM 4,862 ± 56 1.3 4,801 ± 45 0.0 4,828± 238 0.6 4,904 ±400 2.2

aFor the calculatedantibody concentrations forthelogistic-log, fully specified logit-log, partially specifiedlogit-log,andlog-log models,theCVswere3.0,2.5,

12.1,and21.6,respectively,and ther2values were0.999, 0.999, 0.991, and 0.977,respectively.CV is(standarddeviation/mean) x100.

b% Error,absolute value{[(calculatedconcentration -4,800U/ml)4,800U/ml] x100}.

cNA,notapplicable. Mean ODs at thisdilutionwereused toestimateODmax.

2.0 ._ a) 0 0 40~ 0 1.5 1.0 0.5 0.0 1.5. 0.5. -0.5. -1.5. -2.5 -3.5. 0) 0 -J -0 a1) . CD E. 0

on March 10, 2021 by guest

http://jcm.asm.org/

Downloaded from

(4)

TABLE 2. Calculatedantibody concentrationsfrominterpolatingdataforseriallydiluted sera from fourstandard curvesa

Logistic-log Fully specifiedlogit-log Partially specified logit-log Log-log

Logdilution Meancalculated % Mean calculated concn % Meancalculated % Meancalculated %

concn(U/mI± SEM) Errorb (U/ml± SEM) Error concn(U/ml± SEM) Error concn

(U/mI

+ Error

SEM) -3.7 3,950 ± 114 17.7 4,175 ± 107 13.0 4,440 + 427 7.5 2,996 ± 390 37.6 -3.4 4,853 ± 76 1.1 5,055 ± 106 5.3 6,322 ± 521 31.7 6,827 ± 787 42.2 -3.1 4,943 +78 3.0 5,082 ± 117 5.9 5,655 ± 393 17.8 7,514 ± 675 56.6 -2.8 4,681 ±74 2.5 4,754 ±96 1.0 4,765 + 198 0.7 7,004± 343 45.9 -2.4 4,601+ 66 4.2 4,612 ±71 3.9 4,283 ± 50 10.8 6,078 ±63 26.6 -2.2 5,208 ± 58 8.5 5,148± 56 7.3 4,798 ± 225 0.0 4,777 ± 135 0.5 -1.9 5,510 ± 211 14.8 5,350 ± 180 11.5 NAC NA 2,925 ± 111 39.1 Mean ± SEM 4,821 ± 187 0.4 4,882 ± 150 1.7 5,044 ± 321 5.1 5,446± 721 13.5

a Forthe calculatedantibodyconcentrations forthelogistic-log, fullyspecified logit-log, partially specified logit-log, and log-logmodels, theCVswere10.3,

8.215.6,and 35.0,respectively. CV is (standard deviation/mean) x 100.

b%Error,absolute value{[(calculatedconcentration -4,800U/ml)/4,800U/mll x 100}.

c NA, notapplicable. MeanODsatthis dilutionwere used toestimateODmax.

degree ofaccuracy.The r2 washigh,0.999, and the percent-ages of error remained consistently less than5.0% overall

dilutions in the assay. The overall meanof 4,801 U/ml was

almost identical to the known concentration of 4,800 U/ml (0.0%error),andthelowSEMof45U/ml, coupled withthe low CV of 2.5, indicated that this estimate was also stable overtheentire rangeofthe assay.

The curve for the partially specified logit-log model is presented in Fig. 1C, and the results are shown in Table 1.

Optimally, theOD associatedwith infinite concentration, a

quantity which is usually unknown, should be used for this parameter. Since most investigators that employ this model would not also estimate the four parameters of the

logistic-log function, the estimate for this quantity, a, from the

logistic-log model would be unavailable for use in the

par-tially specified logit transformation. The mean of the three

ODs measuredatthe highest concentration was usedasan

estimate for this parameter. This effectively eliminated the

highest concentrationfrom the standard curve and resulted in a reduced, six-dilution range for the assay. The high

r2

(0.991)maylead onetoconclude that the modelproducedan

extremely accurate standard curve. However, three of the

dilutions had percentages of error exceeding 10.0%. The overall mean concentration (4,828 U/ml) closely

approxi-mated the knownconcentration of4,800U/ml,but the SEM

of238 U/mlwas approximately4.5 times the magnitude of

the standard errors for the logistic-log and fully specified logit-log fits. This, coupled with the large CV of 12.1,

indicatesthatthis model is less stable across the entire range

ofthe assay than the logistic-logorfully specified logit-log models.

Ofthemodelssurveyed,thelog-log functionwasthe least accurate (Fig. 1D and Table 1). The high r2, 0.977, would indicate that the standard curve was performing well; how-ever, several of the dilutions had extreme percentages of error,oneashigh as30.3%.While the meanconcentration of

4,904 U/mladequatelyestimatedtheknown concentration of 4,800 U/ml, the SEM of 400 U/ml was extreme and the overall CV, 21.6, wasunacceptably high.

Calculation of antibody concentrations for serially diluted sera. Aserum sample of known concentration (4,800 U/ml)

was seriallydiluted, and concentrations were determined at each dilutiontoassessthe accuracy withwhich the standard curve,generatedwith each mathematical model, could esti-mate antibody concentration. Table 2 details the mean calculated antibody concentrations for each dilution

deter-mined from four plates containing four separate replicate

serialdilutions.

Of themodels tested,thefour-parameter logistic-log

func-tion provided the most precise estimates of antibody

con-centrationsasjudged bythe percentage erroroftheoverall

meancalculated concentration(Table2). With theexception

of the lowest andhighest dilutions,the percentages oferror wereall less than 10.0%.Theoverall meanconcentrationof 4,821U/mlestimated the knownconcentration of4,800U/ml

with a0.4%error.The mean was estimated with an SEMof

187 U/ml, which resulted in aCV of 10.3%.

Thefully specified logit-logmodelinwhich d anda from the logistic-log function were substituted for

ODmin

and

ODmax,

respectively, achieved the samedegree ofaccuracy asthefour-parameterlogistic-log function (Table 2). Again,

with the exception of the lowest and highest dilutions, the calculated concentrations estimated the known

concentra-tionswith percentages oferrorless than10.0%. Theoverall

meanof4,882U/mlestimated the knownconcentration with an errorof

1.7%,

and the SEMand CV werecomparable to those ofthelogistic-log fit.

Thepartially specified logit-log model,where ODmax was estimated by using the mean of the ODs at the highest concentration, did not result in calculated concentrations

with the samedegree of accuracyasthelogistic-logorfully specified logit-log models (Table 2). By using this method,

theuseful dilutionrangeof the assaywasreducedandsome

concentrationswerecalculatedwith

unacceptable

errors(up

to 31.7%). Theoverall mean wasestimated with littleerror

(5.1%), but the SEMof321 U/ml was approximately twice thatofthelogistic-logandfully specifiedlogit-log functions.

TheCV, 15.6,wasalso greater in

magnitude

thanthatofthe logistic-log and fully specified logit-log models.

The log-log model was the least adequate of the models tested in this study. Concentrations were calculated with extremeerrors,oneexceeding 55%.Themeanconcentration

over a seven-dilution range, 5,446

U/ml, approximated

the knownconcentrationwithanerrorof

13.5%,

and the SEM of 721 U/ml and the CV of 35.0 were more than three times greater than those values of the four-parameter logistic-log

model.

DISCUSSION

There is a wide variety of curve-fitting methods in use

which generate standardorcalibrationcurves.Rodgers (25)

on March 10, 2021 by guest

http://jcm.asm.org/

(5)

listed many of these methods in a report which included a comprehensive bibliography. Sandel and Vogt (26) com-pared the accuracies of nine methods by using an evaluation index calculated from the deviations of the calculated con-centrations of the standards from their known values. Rod-bard et al. (24) classified several methods by using various criteria, including ease of graphic analysis, simplicity, and resulting statistical information. It is clear from these discus-sions that no one model will be optimal for all experimental situations or all assays or there would not be the prolifera-tion of methods that exist for these analyses. Each technique has advantages and disadvantages and must be judged on its applicability to the experiment.

Rodgers (25) classified the models investigated in this study as semiempirical in that there are theoretical justifica-tions for why they should fit the standards data under some simplifying assumptions. Rodbard et al. argued (21, 22, 24) that, for suitably optimized radioimmunoassays, it can be demonstrated that the calibration curve should be smooth and symmetrical. These arguments may be extended to other continuous-response assays, including ELISAs.

It would not be possible to comprehensively review all existing methods for generating calibration curves from ELISA data. In this study, we explored the utility ofseveral of the most commonly applied mathematical models by using calculated antibody concentrations from a serially diluted serum of known concentration that was independent of the standards used to generate the working curves.

The standard curves examined for thisinvestigation were generated by using an unweighted least-squares analysis. Heteroscedasticity or nonuniform variance of OD measure-ments across standards dilutions was not observed across the range of standards used in our assays, obviating the need to perform weighted regressions. In the discussion section of Sandel and Vogt (26), Rodbard reports that, for such situa-tions, weighting would have only a minimal effect on the calculated regression line or curve. He goes on to state that weighting becomes more important when dealing with "noisy" data or when the precision varies tremendously along the curve. When these situations occurred, we per-formed a robust iteratively reweighted least-squares regres-sion analysis. This method weights each point individually and inversely proportionally to the residual difference be-tween the observed and predicted values for that point. Whereas ordinary least-squares analysis may be adversely affected by these outlying points, the robust fit is resistant to them and minimizes their effects on the overall fit of the model to the data. Examination of the weights for each point offers a mechanism for identifying outliers, giving the inves-tigator the opportunity to monitor the assay for potential deterioration. The application of these robust fitting tech-niques to the four-parameter logistic-log function was de-scribed by Tiede and Pagano (28). A BASIC program de-tailed previously by Plikaytis et al. (18) was used to implement these techniques for the purposes of calculating antibody concentrations and immunoglobulin levels and is described more thoroughly elsewhere (4, 18).

Of the methods surveyed in the present study, the log-log model was the least accurate at formingstandard curves and interpolating antibody levels for samples with known con-centrations. This functional form usually yields high r2

values, resulting in the impression that the standard curve is accurate at both describing the standards and interpolating antibody concentrations for patient sera. However, in this study, the log-log model consistently described the stan-dards data with the highest degree of error. The curves also

produced the greatest error when the concentrations for separate, independent sera were calculated. This result underscores the hazardsofusing the r2 statistic asthe sole criterion forjudging the goodness-of-fitof the model. Since the r2 is calculated in the transformed log

OD-log

dilution scale, it does not offer reliable information

regarding

the adequacy of the fit of the model when considering the accuracyof calculated concentrations.

The only situation in which the log-logmodel may work adequately is when patient sera are serially diluted over a

wide rangeofdilutionsand theconcentrationsare

averaged

overall points. However, if

single-point

determinationsare

done,theresults may leadtoserious

miscalculation,

depend-ing on what portion of the standard curve is used for the interpolation. Thisconditionisdifficulttocontrol since it is dependent on knowledge of the particular

region

of the standard curve that delivers the most accurate results and theability toadjust the concentrationofthe

patient

serum so

that thecalculatedconcentrationwould be

interpolated

from thisrestrictedportion of the standardcurve.Since estimates made byusing this modelare associated with inflated

stan-dard errors, there is little to recommend the

log-log

model for generaluse. Alternatively, onemayrefit this modelover

areduced rangeofstandards,either

eliminating

thelowestor

highest concentration of standard serum or both.

Deleting

thehighest concentration resulted ina

six-point

assay, and the accuracyofthecalculatedconcentrations forthe

serially

diluted independent samples continued tobe

unacceptably

low (data not shown). Only when the lowest standard

concentration was also eliminated,

resulting

in a

five-point

assay, did this modelyield adequate results. In this

circum-stance, three of the five dilutions continued to estimate calculated antibody concentrations with greater than 10% error (data not shown).

Thepartiallyspecifiedlogit-log model,in which the

ODmax

parameter was estimated by using the average OD at the highest concentration of standard serum,

represented

a pronounced improvement over the

log-log

method. This averaging technique for

estimating

ODmax

eliminates these points from the standard curve and reduces the effective

range ofthe assayby onedilution. The percentages oferror

for both the standard curve

points

and the calculated

con-centrations forthe independent

sample

decreased

compared

with thoseof thelog-log model. However, aswith the

log-log

model, the partially specified logit-log may still result in misleading calculations ofpatient serum concentrations ifa

single dilution determination is made. This model also re-quires sera to be serially diluted and thecalculated

concen-trations at each dilution to be averaged to

yield

a final

determination of antibody concentration. The results de-tailed in Table 2 indicate that the mean calculations of

patient antibody concentrations would be closerto the true

valuesfor thepartially specified logit-log modelthan for the log-log model. The reduction of both the standard error of

the overall meanofcalculated

antibody

concentrations and theCVusing thepartially specified

logit-log

model indicates

that the mean calculations of patient

antibody

concentra-tionsaremorestable than thoseof the

simple

log-log

model. Given the similarity in

computational

burden,

the

partially

specifiedlogit-log modelwould be

preferred

overthe

log-log

method.

Of the linearizing techniques

investigated,

the

fully

spec-ified logit-log model was the most accurate. The calibration curve described the standards with little error, and the concentrations of the

independent

samples

were also calcu-lated with reduced error. In this

study,

the

four-parameter

on March 10, 2021 by guest

http://jcm.asm.org/

(6)

logistic-log model was fit to the standards data first and the parameter estimates for d and a were usedtoestimate ODmin and

ODmax,

respectively, in the fully specified logit transfor-mation. This was done to investigate the behavior of this transformation under theoretically optimal situations. In actual practice, it is unlikely that one would estimate a standard curve by using more than one technique. Conse-quently, investigators using the fully specified logit-log model alone would not routinely haveaccess to the

logistic-log parameter estimates. We havefound theaccuracyof the

fully specified logit-log model to be highly sensitive to the

accuracy with which these two parameters are specified, prior to calculating the logit, with the estimate for

ODmin

being more critical than the estimate for

ODmax.

Typically,

these quantities would not be knownprecisely, and

estimat-ing them in the absence of results from the logistic-log fit would degrade theperformance of thismodel. Ifone was to reason that the OD at the lowest concentration should be zero, the ODmin parameter could be arbitrarily set equal to 0.0. This would reduce the fully specified logit-log model to

the partially specified logit-log model. The performance of this reduced model would suffer when compared with the fully specified logit-log function in which both

ODmin

and

ODmax

were used in the logit transformation. Inany event, the results for the fully specified logit-logmodel reported in this study are ideal and, in the absence of fitting the logistic-log function, would not reflect the true behavior of this model.

The logistic-log model described our N. meningitidis

group A ELISA data with the most accuracy. The calibra-tion curve described thestandards datawith little error, and the concentrations of the independent samples were inter-polated from the standard curve with the highest degree of accuracy of the modelstested in thisstudy, asevidencedby the mean antibody concentrations (Table 2). The low stan-dard error of the overall mean and CV associated with computations ofindependent patient sera indicate that this

functionforms standard curves that may be used to interpo-late patient antibody concentrations with a high degree of reliability overawide range ofdilutions.

Ourinvestigations indicate that thelogistic-log functionis the most accurate and informative model to use when

formingELISAcalibration curvesfor N. meningitidis group

A. There is a high degree ofaccuracy over a wide assay

range when computing antibody concentrations of indepen-dent patient sera. It is important to note that the four-parameter logistic-log function best fits standards data that conform to a sigmoidal or S shape when plotted on an

OD-log dilution scale. This method is general enough that other assays exhibiting this same relationship will benefit from the use of thisfunctionto formstandard curvesand the

interpolation ofconcentrations for patient sera.

The logistic-log model is symmetrical through a central point ofinflection. If thestandardsdata are notsymmetrical,

this asymmetry may be addressed through the use of addi-tional parameters in the logistic-log model (24). The param-eters of this model are estimated in an iterative fashion in which a set ofinitial estimates is continually improved and

reestimated, in a stepwise fashion, until there is no notice-able change in the revised estimates. If the assay does not display a complete sigmoidal

relationship,

the estimation process may not iterate to a final solution or, if a solutionis

reached, the parameter estimates may be associated with large standard errors. However, in our experience, assays which do not include a clearly defined upper bound or

asymptote still lend themselves to analysis by use of this

model (data not shown). In this situation, graphs of the

calibration curves must be examined to see how wellthey

describe the standards data. If the curves follow the

stan-dards data points well, they may be used to interpolate

antibody concentrations from unknown patient sera. If the curves are notrepresentativeof the standards data and if this condition occursfrequently,it may be necessary to adoptan alternativemodel for forming standard curves or to optimize the assay parameters so that the resulting standard curve

conforms more closely tothelogistic-log model.

One alternative method which many investigators use to form standard curves is the spline fit. This technique is commonly thought of as a mathematical version of the French curve. Theprocedureis implemented by dividing the dataintogroups and fitting a low-order polynomial (a poly-nomial of degree three for cubic splines) to each of the separate groups with the constraint that the polynomial of each group bejoined to generate a smooth curve. Splines may be fit exactly, passing through each data point. Addi-tionally, splines may be smoothed to form a more uniform curve with feweroscillations, and, depending on the degree of smoothing specified, will approach a straight line. If such afunction was fit exactly and included in Table 1, a perfect fit could be achieved, yielding 0% error at each dilution. Alternatively, the spline may be smoothed so that the line passes close to each of the points, with the percent errors controlled by the degree of smoothing. One major disadvan-tage of this technique is that, depending on the degree of smoothing, it is strongly influenced by outlying or aberrant observations, in effect, including experimental and instru-ment errorin the final fit. Since the calibration curve derived

from the spline technique may be fit through the standards data points exactly, this method must be evaluated and compared with other methods by using independent serum specimens with dilutions that fall between thestandards data points. Whenthis procedure is followed, splinefits often do notperform as well as the more successful parameter-based fits described in this report (25, 26). An additional concern is that spline fits can fit data from bad assays as well as from good assays, whereas the parametric models described here will yield progressively poorer results as the assay degener-ates. Thus, a model which has successfully fit data from a certain assay several times may serve as a tool toidentify a bad assay when the fit is lessaccurate. Thisability to detect a bad assay is absent from the spline-fitting technique (24). Another alternative for forming standard curves is the power curve model of the formOD= a x (dilution)b, where a and b are the two parameters to be estimated. This function may be linearized to yield log(OD) = log(a) + blog(dilution) and as such is functionally equivalent to the log-log model described in this paper. As with the log-log method, the success withwhich the powerfunctionwill form a standard curve will depend on how well the log-log

transformation linearizes the standards data.

While it is likely that an investigator will choose one method exclusively for standard curve estimation, there is something to be gained by fitting both the four-parameter logistic-log and the fully specified logit-log functions. A curve may be generated to describe data from serially diluting patient sera. Fitting the logistic-log model and then using the a and d parameters from that fit to linearize the data as precisely as possible by using the fully specified

logit-log model will facilitate testing forparallelism between the patient sera and the standard curve. Although a tech-nique for testing forparallelism between logistic curves has been described (20), this procedure is more involved than

on March 10, 2021 by guest

http://jcm.asm.org/

(7)

testing for parallelism between two linear curves, and the use of the fully specified logit-log function simplifies this evaluation.

In the past, the selection of the calibration curve model was largely dictated by the computational power at hand (e.g., hand-held calculators, desktop computers, and main-framecomputers, etc.) aswell as the knowledge and

exper-tise of the investigator. Although knowledge ofthe applied technique is still required, the readyavailability of inexpen-sive and powerful personal computers and the existence of several software packages that can perform bothlinear and nonlinear parameter estimation makethechoiceofmethods virtually independent of their computational requirements. Calculations for this report were made on aCompaq 386s/20 computer with a math coprocessor using the SAS system of statistical software (27). The time ofexecutionforparameter estimation was minimal and did not differmarkedly among the four methods examined. In additionto theSAS system, there are several other software packages available to per-form these types of analyses. Robert Maciel (16, 17)markets a comprehensive series of programs tailored to ELISA applications. The program described by Canellas and Karu (6) fits the logistic-log model, with an optional weighting function which adjusts for systematic nonuniform variance. Rodbard and Lewald (23)introducedaprogram thatused the

logit-log and log-log models. This program was expanded (ALLFIT) to incorporate other models, including the four-parameter logistic-log method (7), and has evolved to its

presentimplementation, FLEXIFIT(12, 13),whichincludes a constrained smoothing spline fit to estimate curve shapeas

well as the four-parameter logistic-log function to test for

parallelism and similarityofshapefor any set of two ormore curves. In addition, a series of public domain programs which fit the four-parameter logistic-log modelto standards data and calculate antibodyconcentrationsforpatient serum samples are available. These programs include botha

least-squares fit and the robust iteratively reweighted

least-squares fit (4, 18).

In conclusion, the logistic-log function is superior to alternative methods in modeling standard curves, consider-ing the accuracy ofdescribing the standards and interpolat-ing patient antibody concentrations. In addition, no param-eters need to bespecifiedprior to fitting the model, and the final estimates give valuable details about the assay regard-ing the OD at zero and infinite concentrations and the midrange of theassay,giving thedilutionassociatedwith the point ofinflection. Thismodelalso hastheadvantagethat it is the easiest to visualize since it does not incorporate complex transformations of the OD scale.

REFERENCES

1. Armitage,P., and G. Berry. 1987. Statistical methodsinmedical research, p. 141-159. Blackwell Scientific Publications, Ltd., Oxford.

2. Beuvery, E. C., M. H. Kayhty, A. B. Leussink, and V.Kanhai. 1984. Comparison of radioimmunoassay and enzyme-linked immunosorbent assay in measurement ofantibodies to Neis-seria meningitidis group A capsular polysaccharide. J. Clin. Microbiol. 20:672-676.

3. Beuvery,E. C., A. B.Leussink, R.W. VanDelft,R. H.Tiesjma, and J. Nagel. 1982. Immunoglobulin M and G antibody re-sponses and persistence of these antibodies in adults after vaccinationwith acombinedmeningococcalgroup A and group Cpolysaccharide vaccine. Infect. Immun. 37:579-585. 4. Black, C. M., B. D. Plikaytis, T. W. Wells, R. M. Ramirez,

G. M. Carlone, B. A. Chilmonczyk, and C. B. Reiner. 1988. Two-site immunoenzymemetric assaysfor serumIgG subclass infant/mother ratiosatfullterm.J.Immunol. Methods106:71-81. 5. Boctor,F.N.,N. E. Barka,and M.S.Agopian. 1989. Quantita-tionofIgG antibodytoStreptococcus pneumoniae vaccineby ELISA andFAST-ELISAusing tyraminated antigen.J. Immu-nol. Methods 120:167-171.

6. Canellas, P. F., and A. E. Karu. 1981. Statistical package for analysis ofcompetition ELISA results. J. Immunol. Methods 47:375-385.

7. DeLean, A.,P.J. Munson,and D.Rodbard. 1978.Simultaneous analysis of families of sigmoidalcurves:applicationtobioassay, radioligandassay, andphysiological dose-responsecurves.Am. J. Physiol. 235:E97-E102.

8. Draper,N.R., and H.Smith. 1981.Appliedregressionanalysis, p. 1-69. JohnWiley & Sons, Inc., New York.

8a.Draper,N.R.,andH.Smith. 1981.Applied regressionanalysis, p. 458-529. JohnWiley &Sons, Inc., New York.

9. Fattom, A., C. Lue, S. C. Szu, J. Mestecky, G. Schiffman, D. Bryla,W. F.Vann,D.Watson,L.M.Kimzey, J.B.Robbins,and R. Schneerson. 1990. Serum antibody response in adult volun-teers elicited by injection of Streptococcus pneumoniae type 12F polysaccharide alone or conjugated to diphtheria toxoid. Infect. Immun. 58:2309-2312.

10. Finney, D.J. 1976. Radioligand assay. Biometrics 32:721-740. 11. Frasch, C. (Center forBiologics Evaluation andResearch, Food

andDrug Administration). 1990. Personal communication. 12. Guardabasso, V., P. J. Munson, and D. Rodbard. 1988. A

versatile methodfor simultaneous analysis of families ofcurves. FASEBJ.2:209-215.

13. Guardabasso, V., D. Rodbard, and P. J. Munson. 1987. A model-free approach toestimation of relative potency in dose-response curve analysis. Am.J. Physiol. 252:E357-E364. 14. Karpinski, K.F.,S. Hayward, and H. Tryphonas. 1987.

Statis-tical considerations in the quantitation of serum immunoglobu-lin levels using enzyme-linked immunosorbent assay (ELISA).

J. Immunol. Methods 103:189-194.

15. Kortland, W., H. J. Endeman, and J. 0. 0. Hoeke. 1987. A three-parameter Langmuir-type model for fitting standard curves of sandwich enzyme immunoassays with special

atten-tionto thea-fetoprotein assay. Anal. Biochem. 162:5-10. 16. Maciel, R. 1990. ELISA AID. Robert Maciel Associates, Inc.,

Arlington, Mass.

17. Maciel,R.J. 1985.Standardcurvefittinginimmunodiagnostics:

Aprimer. J. Clin. Immunoassay 8:98-106.

18. Plikaytis, B. D., G. M. Carlone, P. Edmonds,and L. W. Mayer. 1986. Robust estimation of standard curves for protein molecu-lar weight and linear-duplex DNA base-pair number after gel electrophoresis. Anal. Biochem. 152:346-364.

19. Ritchie, D. G., J. M. Nickerson, and G. M. Fuller. 1981. Two simple programs for the analysis of data from enzyme-linked immunosorbent (ELISA) assays on a programmable desk-top calculator. Anal. Biochem. 110:281-290.

20. Rodbard, D. 1974. Statistical quality control and routine data processing for radioimmunoassays and immunoradiometric as-says. Clin. Chem. 20:1255-1270.

21. Rodbard, D., W. Bridson, and P. L. Rayford. 1969. Rapid calculation ofradioimmunoassay results. J. Lab. Clin. Med. 74:770-781.

22. Rodbard, D., and D. M. Hutt. 1974. Statistical analysis of radioimmunoassays andimmunoradiometric (labelled antibody) assays: ageneralized weighted, iterative, least-squaresmethod forlogistic curvefitting, p. 165-192. In Radioimmunoassay and relatedprocedures in medicine. Proceedingsof theSymposium, Istanbul, 1973. International Atomic Energy Agency, Vienna. 23. Rodbard, D., and J. E. Lewald. 1970. Computer analysis of

radioligand assay and radioimmunoassay data. Acta Endo-crinol. Suppl. 147:79-103.

24. Rodbard, D., P. J. Munson, and A. De Lean. 1978. Improved curve-fitting, parallelism testing, characterizationofsensitivity, validation, andoptimizationforradioligand assays, p. 469-514. In Radioimmunoassay and related procedures in medicine. Proceedings of theSymposium,WestBerlin,1977.International

on March 10, 2021 by guest

http://jcm.asm.org/

(8)

Atomic Energy Agency, Vienna.

25. Rodgers, R. P. C. 1984. Dataanalysis and quality control of

assays: a practical primer, p. 253-308. In W. R. Butt (ed.), Practical immunoassay, the state of the art. Marcel Dekker, Inc., New York.

26. Sandel,P., and W.Vogt. 1978. Performance of various mathe-matical methodsfor calculation of radioimmunoassay results,p.

373-381.InRadioimmunoassay and related procedures in

med-icine.Proceedings of the Symposium, West Berlin, 1977. Inter-national AtomicEnergy Agency, Vienna.

27. SAS Institute, Inc. 1990. SAS/STAT user'sguide version 6. SAS Institute, Inc., Cary, N.C.

28. Tiede, J. J., and M. Pagano. 1979. The application of robust calibrationtoradioimmunoassay. Biometrics 35:567-574.