0095-1137/78/0008-0689$02.00/0
Copyright©1978 AmericanSocietyforMicrobiology PrintedinU.S.A.
Effect of Atypical
Antibiotic Resistance
on
Microorganism
Identification by Pattern
Recognition
JAMES C. BOYDt, JOHN W. LEWIS,' J. JOSEPH MARR,2 ALICE M. HARPER,3 ANDBRUCE R. KOWALSKI3
Division of Laboratory Medicine, Departments of PathologyandMedicine, Washington UniversitySchool
ofMedicine, St. Louis,Missouri63110'; DepartmentofMedicine,St. LouisUniversitySchoolof Medicine, St.Louis,Missouri63104';andLaboratoryof Chemometrics, University of Washington,Seattle,
Washington981953
Received for publication 23 August 1978
Weclassified microorganisms from the clinical laboratory by usinginformation
provided by the Gram stain and antibiotic sensitivity profiles obtained withthe
Bauer-Kirby technique. Approximately 4,000 microorganisms, routinely identified
andtested for antibioticsensitivities inalargehospital microbiology laboratory,
were used as a data set forseveral pattern recognition classification methods:
K-nearest-neighbor analysis, statistical isolinear multicomponent analysis,
Baye-sian inference, and linear discriminant analysis. K-nearest-neighbor analysis
yielded the highest prospective classification accuracyfor gram-negative
orga-nisms, 90%. When those organisms displaying an atypical antibiotic resistance
patternwere excluded from thedata, thegram-negative classification accuracy
improvedto 95%. These results are inferior to currently accepted biochemical
identification methods. Microorganisms with atypical antibiotic resistance
pat-ternsarelikelytobemisidentified andare commonenough (17% ofourisolates)
to limit the feasibility of routine identification of microorganisms from their
antibioticsensitivities.
Prior attempts to classify micororganisms on
the basis of their antibiotic sensitivities have
metwithvaryingsuccess(6, 10, 18,20).None of
these studies has shown aprospective
identifi-cation accuracy greater than 86%, not high
enoughto allow routine use of antibiotic
sensi-tivitiesfororganismidentification. Thequestion
of what effect multiply resistant organisms
would have on the classification accuracies of these methods has remained unanswered.
Computer-basedpatternrecognitionmethods
have beenapplied successfully toclassification
problems in a variety ofdisciplines, including analytical chemistry (14), image processing(19), wave form analysis (11), and optical character recognition(21). These methodsappeared
prom-isingwhen applied to the problem of microor-ganism identification from antibiotic sensitivity data(18);however,thisstudyhad thelimitation
of a small data base which did notinclude
mul-tiply resistant bacterial strains. Other methods
used in the identification of microorganisms
from theirantibioticsensitivities have required
assumptionsto be made about the distribution ofsensitivitydata(10, 20) or about the statistical
independence of sensitivity measurements for
tPresentaddress: University of Virginia MedicalCenter, Charlottesville,VA22901.
different antibiotics(6). Since these assumptions
are unlikely to hold for antibiotic sensitivity
data, we believed that a study using pattern
recognitionmethods which did not require these
assumptionswasindicated.
Ourstudywasdesignedto assessthe accuracy ofmicroorganismidentification from antibiotic sensitivity data and to evaluate the effect of
multiple antibiotic resistance on these identifi-cations.
MATERIALS AND METHODS
Routine antibiotic sensitivities were measured in the BarnesHospital MicrobiologyLaboratory by the technique of Bauer et al. (1). Microorganismswere identified in the laboratory by using biochemical methodology. Enterobacteriaceae were identified with theAnalytab Products Co.API20E system; when necessary, the identification wassupplemented with conventional biochemical tests, as recommended by Edwards and Ewing (9). Pseudomonadaceae were identifiedwithconventionalbiochemical tests, as de-scribedby HughandGilardi(12).Staphylococcal spe-cies were identified with Gram stain, catalase, and coagulase tests.Streptococci were identified by using catalase, growth in bile, hydrolysis of esculin, and growth in 6.5% sodium chloride. Sensitivity zone di-ametersandorganismidentificationsfor 4,213 consec-utive, unselectedspecimens werecollected over a 3.5-monthperiod from the on-line computer system serv-689
on February 7, 2020 by guest
http://jcm.asm.org/
ing the Barnes Hospital Microbiology Laboratory (17). Included in this data set were 3,448 gram-negative isolates. Organism classes having fewer than 45 iso-lates were excluded. This yielded a data base having eight classes ofgram-negative organisms and three classes of gram-positive organisms (Table 1). These classesencompassed 94% of the isolates in our clinical laboratory. The sensitivity data for infrequently tested antibiotics were not considered in this study. The antibiotics usedin identification were as follows: for gram-negative organisms-ampicillin, carbenicillin, cefamandole, cephalothin,colistin, gentamicin, kana-mycin,tetracycline,andtobramycin; forgram-positive organisms-ampicillin, cephalothin,chloramphenicol, clindamycin, erythromycin, gentamicin, methicillin, penicillin, tetracycline, tobramycin, and vancomycin. Pattern recognition methods (13) were used to pre-dictorganism identities from their antibiotic sensitiv-itymeasurements. The basic function of the pattern recognition techniques was to reduce multidimen-sional data (antibiotic sensitivity measurements for multiple antibiotics) to meaningful information (or-ganism identifications). Oncethe data werecollected, pattern recognition wascarried out in several steps: initial data examination, feature selection, classifier training, and classifier testing.
Initial dataexamination.Inthisphase, theraw
sensitivity measurements were plotted to allow ex-aminationof the distributionof antibiotic sensitivity
zonediametersandtogatherpreliminaryimpressions of howclosely the antibiotic sensitivitymeasurements clusteredwithin each group oforganisms.
Frequency histograms of the zone size measure-ments were the first method used to plot the data. These plots were relied upon to detect non-normal distribution of themeasurementsfor each antibiotic. If themeasurementsforindividualantibiotics didnot
follow the normal bell-shaped Gaussian distribution, the necessity for using pattern recognition methods which did not rely on the assumption ofnormally distributed data wouldbe apparent.
To assess how closely the antibiotic sensitivity
measurements clustered within each group of orga-nisms,a means wasneeded forconsideringthe
multi-TABLE 1. Organismclasses studied
Class Organism isolatesNo.of
Gramnegative
1 Enterobacter sp. 233
2 Klebsiellapneumoniae 442
3 E.coli 1,452
4 Pseudomonasaeruginosa 525
5 Proteusmirabilis 396
6 S.marcescens 98
7 C. diversus 46
8 P.morganii 52
Grampositive
1 Staphylococcusaureus 433
2 Enterococcus 149
3 Staphylococcus epidermi- 141 dis
ple-antibiotic zone size measurements of each isolate simultaneously. One way of accomplishing this was to represent each organism as a labeled point in an N-dimensional plot(N was equal to the number of anti-biotics).Each of the N axes would then correspond to anantibiotic; e.g., the ampicillin zone diameter mea-surementfor each organism would serve to position thelabeled point for that organism along the ampicil-lin axis and the measurements for the other antibiotics correspondingly for the other axes. If only two anti-bioticswere considered, this would reduce to simple two-dimensional plotting. If three antibiotics were plotted, a perspective display could be used to visualize the three-dimensional plot. However, we were con-cerned with more than three antibiotics and thus with spacesofhigher dimensionality than three. These, of course, cannot bevisualizeddirectly, but can be ma-nipulatedwithmathematicalmethodsthat follow di-rectanalogies to analyzing two- or three-dimensional space.Theprincipal-components technique (3), which we used to visualize our data points in multidimen-sionalspace, is amathematical method which allows one toproject or map the points in higher-dimensional space onto a two-dimensional space (i.e., creating a two-dimensional plot),with thecriterionthat the data structurein thehigher-dimensionalspacebe preserved tothe greatestpossibleextent.In this way (see Fig. 1), we wereable to examine directly the clustering rela-tionships which existed among the antibiotic sensitiv-itydata of ourorganismswhenthey wereconsidered
aspointsin amultidimensional space.
Featureselection.Aftersomeappreciationfor the underlying data structure had been gained through initial data examination, the nextproblem to be solved wastodetermine which antibiotics supplied the most useful information for identifying organisms. In the pattern recognition literature, thiswould be called a problemof featureselection.Inapproaching this prob-lem,weapplied severalfeatureselectionmethods (15, 16) toreduce, ifpossible,the number of measurements
usedinsubsequentanalysesandtoenhance the
infor-mation representation of the data by selectingonly those measurementswhichhad value in the identifi-cationoforganisms.Theoverallpurposeof ourfeature
selectionwas to extract thejnost usefulinformation fororganism identification from the fewest measure-ments.
Classifiertrainingandtesting.Thenextstep in the patternrecognitionprocesswasthedevelopment of optimalschemes for organism identification from antibiotic sensitivity measurements. Subsets of 446
and563organismsselectedatrandom from the
gram-positive andgram-negativegroups,respectively,were
used indeveloping organismclassification rules.These
organisms were called the "training set." By using
those antibiotic sensitivity measurements ranked highest by the feature selection methods, organism classification rulesweredevelopedfrom theexamples in the training set by application of four different learning methods: K-nearest-neighbor analysis (KNN) (5), statistical isolinearmulticomponent anal-ysis(SIMCA) (22), linear discriminantanalysis (LDA) (2), and a classifierutilizingthe classification rule of Bayes (BAYES) (8). These methodswere represent-ative of the major pattern recognition approaches
on February 7, 2020 by guest
http://jcm.asm.org/
whichhave beenappliedtosimilar problems. Oneof
ourchief objectives in usingseveral different methods
wastominimize the likelihood thatourresultswould
be biased bytheparticularmethod chosen.
Compari-sonof the results obtainedby different methodswas
also expectedtobe valuable.
Those organisms whichwerenot partof the training
setformeda"testset," whichwasused for prospective
evaluation of theperformance of the various classifi-cation rulesdeveloped abovein classifiertraining.This
was animportant evaluation stepsince it tested the
accuracyof theclassifierswhich had beendeveloped
in the preceding steps, using data other than those usedtotrain theclassifiers.
Atypicallyresistantorganism.To evaluate the
effect ofmultiply and/or atypically resistant species, each gram-negative organism group was split into
subgroups by noting the antibiotics to which each isolate was highly resistant. An individual organism was defined to be highly resistant to a particular
antibioticif itgrewtotheedgeof the discimpregnated with that antibiotic. Thus, Escherichia coli highly resistanttoampicillin formedonesubgroup, whereas
E.coli highlyresistanttotetracyclineandcephalothin formedanother subgroup.
Thereisample evidenceinthepatternrecognition literature (4, 7) thattoevolveareliableclassification
rule, one needsat least three tofive times asmany
members in eachgroupof the trainingsetasthereare
dimensions(antibiotics). Thus,sincewehad nine
an-tibiotic measurementsfor each organism, each
sub-grouphadtocontainatleast27members (three times nine antibiotics) to develop a reliable classification
rule for thatsubgroup. Sparsely populated subgroups (N<27)wereexcluded from studyonthis basis. This
resulted in 14subgroups, encompassing sevenof the
original eight organismclasses and83%of theoriginal gram-negative isolates. Serratia marcescens hadno
subgroupswithgreaterthan21members and thuswas
totallyexcluded from thisphaseof thestudy.Except for S.marcescens,the excludedsubgroupsrepresented onlyasmall fraction of theisolatesintheir respective
parent groups and thus were considered to be the organismswithatypicalantibiotic resistancepatterns.
Atrainingprocesssimilartothat described abovewas
carriedout ontheremaining subgroups for three of
the four classificationmethods. Since theperformance of LDA had been theworst of the four algorithms employed for the previous phases (Citrobacter
diver-suswascompletely misclassified bythis methodasE. coli),itwaseliminatedfrom further consideration.
RESULTS
Zone size distributions. Histograms of the
distribution ofzonediameters for individual
an-tibiotics showeda characteristic double-peaked
bimodall) shape formanyantibiotics. One of the
peaks occurredata zonesize of6mm,indicating
bacterialspecies which hadgrowntothe edge of
theantibiotic-impregnated disc. The otherpeak
occurred atvarious zone diameters, depending
ontheantibioticsensitivity of the organism. The
fact thatthe bimodality ofzonesize distributions
for individualantibiotics persisted evenwithin
a singleclass oforganismsshowed that the
an-tibioticsensitivitydata were notdistributed
nor-mally.Thissupportedourchoice of pattern
rec-ognition classification techniqueswhich did not
require normally distributed data.
Gram-positive organisms. Only8 of the 11
available antibioticsensitivitymeasurements for
each gram-positive isolate were necessary to
achieve optimal identification accuracy for the
gram-positive organisms; the addition of
chor-amphenicol, erythromycin, or tobramycin did
not improve the identification accuracy. The
results of SIMCA classification of a randomly selected subset ofgram-positive organisms, us-ing these eight antibiotic sensitivity
measure-mentsfor eachorganism,areshown in Table 2.
Although the overall classification accuracy of
96% indicated that thesespecies couldbe
relia-blyidentifiedfromtheirantibiotic sensitivities,
the routine identification oftheseorganisms by
inspection of colonies and use of the catalase and coagulase tests is both reliable andrapid. Furtheranalysiswasconfmed, therefore,tothe more important problem of identifying
gram-negative organisms.
Gram-negàtive organisms. Feature selec-tion did not reduce the number ofantibiotics needed for optimal gram-negative identification. C. diversus andProteusmorganii had toofew isolates to form both a
(retrospective)
trainingsetanda (prospective) testset;hence, the
pro-spective identification study included only six
organism classes.The accuracy ofretrospective
and prospective identification, using the data
from the nine antibiotics, is shown in Table 3.
The bestresultswere obtained withKNN,90%
in the training and test sets. BAYES analysis did wellonthetrainingset,91%overall, butwas
less accurate (87%) in the prospective
identifi-cation of the test set organisms. LDA and SIMCA performed more poorly in identifying gram-negative organisms.
Effect of atypical resistancepatterns. In
an attempt to understand why some isolates
weremisidentified,thosemisclassifiedby KNN
were examined for high resistance (as defined
above) to each of the antibiotics. The majority
of the isolates in the E. coli group, forexample,
were nothighlyresistant to any of the
antibiot-TABLE 2. Percentages of gram-positiveorganisms correctlyidentified
Organism N %identifiedCorrectly
S.aureus 273 99
Enterococcus 94 96
S.epidermidis 79 89
aClassificationperformed by SIMCAalgorithm, us-ing fourprincipal components.
8,
on February 7, 2020 by guest
http://jcm.asm.org/
TABLE 3. Percentagesofgram-negativeorganisms correctlyidentified
%Correctly identified by: Data set
KNN SIMCA BAYES LDA
Training, all 90 82 91 77
classes
Training, six 90 82 91 78
classes'
Test, six classes 90 85 87 82
a C. diversus and P. morganiiwere excludedfrom
the six-class datasets.
ics. However, most misclassified E. coli were
highly resistantto one or moreantibiotics.
In-spection of principal-component projections of
the E. coli sensitivity data indicated that there
were several distinct clusters within this class
(Fig. 1). Each cluster was foundto correspond
toasubgroupof E.coltwhichwashighly
resist-ant to the same antibiotic(s). The other orga-nismclassesexhibited asimilarphenomenon of
subgrouping according to the pattern of high
resistancetoanantibioticoragroupof
antibiot-ics. This observation formed the basis for our
subgroupingscheme (described above) and
pro-videdamechanism for identifying theatypically
resistant organisms. The results of KNN,
SIMCA, and BAYES classification after
exclu-sion of the atypicallyresistantsubgroupsof
or-ganisms (Table 4)indicate that the classification
accuracy for all three methods was improved.
KNN now correctly classified 95% of the
orga-nisms. The remaining misclassifications were
due largely to organisms of different species
which were highly resistant to the same anti-bioticorgroupofantibiotics.
DISCUSSION
We evaluated several pattern recognition
techniquesfororganismidentification basedon
antibiotic sensitivity patterns. Our data base
encompassedtheorganismsmostcommonly
iso-lated inalarge hospital microbiology laboratory
and included all multiply resistant organisms
encountered.
Our overall prospective identification
accu-racyforgram-negative organisms (90% correct)
was comparable to the 90%accuracy obtained
byMacDonald (18),who also usedpattern
rec-ognition methods and Bauer-Kirby data. The
prospectiveidentification accuraciesobtainedby
othermethodologieshavegenerallybeenlower,
82%for LDA (6) and 86% for BAYES (10). An
accuracyof 97%wasreportedforquadratic
dis-criminantanalysis (20),but thismethodwasnot testedprospectively toensure that it identified unknown isolates as accurately as it identified
cI
E
v
B.
-g
b-Second Principal Component
FIG. 1. Clustering of E. coli sensitivity data.
Shownis acomputerplot of E. coli sensitivity zone
sizemeasurementswhich have been projected by
us-ingprincipal-component analysis from
nine-dimen-sional (i.e., nine antibiotics) space onto the two-di-mensionalplane generated by the second and third principal components. This projection offers the viewer anoptimal glimpseatthe spatial relationships which exist among sensitivity data ofthe E. coli
isolates.Only the fivemostpopulous subgroupsofE.
coli(definedbypatternsofhigh antibiotic resistance)
areplotted here. Each subgroup is depicted by a
differentplotting symbol. Three distinct clusters of
organisms canbe viewed from this projection
repre-senting: (lower left) nonresistant organisms
(dia-mond) or organisms resistant to tetracycline
(octa-gon); (middle)organismsresistant toampicillin and carbenicillin(star)ortokanamycin and tetracycline (square); (upper right) organisms resistantto ampi-cillin, carbeniampi-cillin, and tetracycline (triangle).
TABLE 4. Percentagesof gram-negativeorganisms
correctlyidentified afterexclusionof atypically resistantorganisms
%Correctlyidentified Classification method
Trainingset Test set
KNN 96 95
SIMCA 95 93
BAYES 98 94
those in thetraining set.The totalnumber and typesofantibiotics towhichbacterial sensitivi-ties were measured and the methods for
deter-mining sensitivities were not the same in all
studies. Our investigation ofgram-negative
or-ganisms employed 9 antibiotics, whereas
Dar-land(6) andMacDonald (18)used 12. Friedman
and MacLowry (10) used eight dilutions of 11
antibiotics in a tube dilution system, whereas
o w
o o; w
Q wQ
v
CD
E GD
Q O Q
on February 7, 2020 by guest
http://jcm.asm.org/
IDENTIFICATION ACCURACY AND ATYPICAL RESISTANCE 693 Sielaff et al. (20) utilized the light-scattering
indexes of 18 antibiotics measured byan
Auto-bacI. Furtherdifferences existedin the number
andthe typesof bacterial species used in each of these studies. The effects of these other
vari-ables are not known; therefore, comparison
amongthese studies is difficult.
Many investigators have suspected that the
occurrence of multiple- or atypical antibiotic
résistance patterns would adversely affect the
accuracyoforganismidentification. Some have
excludedmultiply resistant organisms from their
studies (6, 18), whereas others have reportedno
apparent effect ofmultiple resistance on their
identification accuracy (20). Since KNN
classi-fication hadcorrectly identified thegreatest
per-centageof unknownorganisms inourstudy,we
reasoned thatanexamination of the
KNN-mis-classified organisms would be helpful in
deter-mining thefactors responsible forourlowoverall
organism identification rate. Our investigation
of the antibiotic resistance patterns of
KNN-misclassifiedorganisms established that atypical
antibiotic resistancewas amajor factor reducing
the accuracy of organism identifications
ob-tained from antibiotic sensitivity profiles,
re-gardless of what computer classification
tech-niquewasused.Subgroupingourorganisms
ac-cordingtopatternsofhigh antibiotic resistance
showed that organisms having common
anti-biotic resistancepatternswerereadily identified,
whereas thosehaving atypicalpatterns
(includ-ing multiply resistant species) were quite often
misidentified. Organisms falling into the
sub-groupsthatwecharacterizedashaving atypical
antibiotic resistance patterns comprised about
17% ofourisolates and accounted forover
one-half of the incorrectly identified organisms.
Ex-cluding theatypically resistant groupsof
orga-nisms from ourdata baseimproved the overall
KNNorganism identification accuracy from 90
to95% (Table 4).
The high prevalence of atypically resistant
organisms and the fact that these organisms
accountforthemajority of organism
misidenti-fications help explain the disappointing
orga-nism identificationrates noted in moststudies
(including this one). Several possible approaches
to obviate this problem merit further
explora-tion.First,a meansmight be devisedtoseparate
organisms with atypical resistance patterns.Ail
otherorganisms could then be given provisional
identifications with an acceptableerrorrate. A
secondpossibilitywouldbetouseantimicrobial
agentsother thanclinically employed antibiotics
for organism sensitivity testing and
identifica-tion. In thisway,theatypical resistancepatterns
resulting from resistance transfer plasmidscould
be minimized. However, with this method the
advantage of organism identification from
rou-tinely obtained antibiotic sensitivites would be lost. A third possibility would be to identify organisms by using data from routine antibiotic sensitivities combined with a subset of the more traditionally employed biochemical tests. In this manner, the additional information available
from antibiotic sensitivity data might allow a
reduction in the number of biochemical tests needed for organism identification.
Given the 90% correct overall identification ratenotedin ourstudy,it does not appear
prac-ticaltoidentify microorganisms
exclusively
fromtheir antibiotic sensitivity profiles as aroutine
practice. Anadditionalconcernisthat, although
sensitivity patterns areapproximatelythe same
across thecountry, there isenough interhospital
variation to cast doubt on the general
applica-tion of thismethod. This variation and the
in-trahospital variationwith time wouldprobably
require that each hospital developitsowndata
base and periodically update its classifiers. If
means canbedevelopedto overcomethe
detri-mental effectsofatypically resistant organisms
ontheaccuracyofidentification,then
microor-ganism identificationusingantibiotic
sensitivity
measurements might be auseful additiontothe
armamentarium of theclinicalmicrobiology
lab-oratory.
ACKNOWLEDGMENTS
We acknowledge the support of National Institutes of HealthNational Research Service Award fellowship 5F32 6M05009-02 (J.C.B.), National Science Foundation grant BMS-71-01597A01 (J.J.M.), and Public Health Servicegrant RO1-MB-00184-02 (A.M.H. and B.R.K.).
LITERATURE CITED
1. Bauer, A. W., W.M.M. Kirby,J.C.Sherris, and M. Turck. 1966. Antibiotic susceptibility testing by a standardizedsingle discmethod. Am.J. Clin. Pathol. 45:493-496.
2. Bender, C. F., and B. R. Kowalski. 1974. Multiclass linearclassifier for spectralinterpretation(pattern rec-ognition).Anal.Chem.46:294-296.
3. Chien, Y. T.,and K. S. Fu. 1967. On the generalized Karhunen-Loeve expansion. IEEETrans. Inf. Theory 13:518-520.
4. Cover,T.M. 1965.Geometrical andstatistical properties ofsystemsoflinear inequalities with applications to pattern recognition. IEEE Trans. Electron. Comput. 14:326-334.
5. Cover, T. M.,and P. E.Hart.1967. Nearestneighbor pattern classification. IEEE Trans. Inf. Theory 13: 21-27.
6. Darland, G. 1975. Discriminant analysis of antibiotic susceptibilityas ameans of bacterial identification. J. Clin.Microbiol. 2:391-396.
7. Demartini, J., and A. Vincent-Carrefour.1977.Topics inpattern recognition, p. 107-126.InA. Remond (ed.), EEG informantics. Adidactic review ofmethodsand applicationsofEEGdata processing. Elsevier Scientific Publishing Co.,New York.
8. Duda,R.O.,and P. E.Hart.1973. Bayes decision theory, p. 10-43. In Pattern classification and scene analysis.
VOL. 8,1978
on February 7, 2020 by guest
http://jcm.asm.org/
694 BOYD ET AL.
John Wiley& Sons, Inc., NewYork.
9. Edwards,P.R., and W. H. Ewing. 1972.Identification ofenterobacteriaceae.BurgessPublishing Co., Minne-apolis.
10.Friedman,R.,and J.MacLowry.1973.Computer iden-tification ofbacteria on the basis oftheirantibiotic susceptibiitypatterns.Apple. Microbiol.26:314-317. 11. Fu,K. S. 1976. Patternrecognitioninremotesensingof
the earth's resources. IEEE Trans.Geosci. Electron. 14:10-18.
12. Hugh, R., andG. L. Gilardi. 1974. Pseudomonas, p.
250-269.In E. H. Lennette, E. H.Spaulding,and J. P. Truant(ed.),Manualofclinicalmicrobiology.American Society forMicrobiology,Washington, D.C.
13. Koskinen,J. R.,andB.R.Kowalski.1975.Interactive
patternrecognitioninthechemicallaboratory. J.Chem.
Inf. Comput. Sci.15:119-123.
14. Kowalski, B. R. 1975. Measurements analysis. Anal. Chem.47:1152A-1162A.
15. Kowalski,B. R., and C. F. Bender. 1975. Pattern
rec-ognition.Apowerful approachtointerpretingchemical
data. J.Am.Chem.Soc.94:5632-5639.
16. Kowalski,B. R.,andC. F.Bender. 1976.Anorthogonal
feature selection method. Pattern Recognition 8:1-4. 17. Lewis,J.W., andJ. J.Marr. 1975.Alow-cost clinical microbiology computer system. p.97-100. In J. Zim-merman (ed.), Proceedingsof the 1975 Mumps User's Group Meeting.Mumps User'sGroup,St.Louis. 18. MacDonald, J. C.1978.Patternrecognitionin
microbi-ology.Am.Lab.10:78-85.
19. Rosenfeld, A. 1976. Digital picture analysis. Springer PublishingCo., New York.
20. Sielaff,B.H.,E. A.Johnson,and J. M.Matsen. 1976. Computer-assisted bacterial identification utilizing an-timicrobialsusceptibilityprofilesgeneratedbyAutobac 1. J.Clin.Microbiol.3:105-109.
21. Ullmann, J. R. 1973. Pattern recognition techniques. Crane, Russak and Co.,NewYork.
22. Wold,S. 1976. Patternrecognitionby means ofdisjoint principal componentsmodels.Pattern Recognition8: 127-139.
