Feedback calibration training improves
whisky sensory profiling
AB.09
Dr Chris Findlay, Compusense, Canada
Sensory Descriptive Analysis
•
Sensory Descriptive Analysis
•
Whisky Panel Y2K
•
Feedback Calibration 2006
•
Whisky Panel 2012
•
Comparison of progress over 12 years
1. Screening and Selection of the Panel 2. Training and Ballot Creation
– Panelists are trained using reference standards All descriptive attributes are rated on unstructured 100-point line scales.
– Training sessions are 2 hours long and 20 sessions are typically used to train a new group of screened panelists
3. Data Collection and Analysis
– Ten to twelve trained panelists performed three replicate evaluations of around 10 products using a balanced William’s design, for a total of at least 30 evaluations per product.
– Means are calculated and analysis of variance is used to determine significance
– Results are presented in various plots
Sensory Descriptive Analysis
Y2K
Whisky
Product Abbreviated Product Name
Johnnie Walker – Black Label – Blended Scotch Whisky JWB Maker’s Mark – Kentucky Straight Bourbon Whisky MM
Jack Daniel’s – Tennessee Sour Mash Whiskey JD Labrot & Graham – Woodford Reserve - Kentucky Straight Bourbon
Whiskey WR
Four Roses – Kentucky Straight Bourbon Whiskey 4R Jameson – Triple Distilled Irish Whiskey JAM
Descriptor Concentration Stock solution Recipe (into 300ml of BAS)
Smokey-wood
Real Hickory Smoke Flavour
140μl of flavour Smokey-charcoal 80/20 Smoke Flavour 15μl of flavour Phenolic o-Cresol 0.025ppm
2.3μl of o-Cresol into 100ml ethanol (40%)
0.324ml of stock solution Penolic Guicacol 0.5ppm
69μl Guiacol into 100ml of ethanol/ vodka (40%)
0.216ml of stock solution Fruity Isoamyl acetate 10.9ppm
100μl of Isoamyl acetate into 100ml of water
3.273ml of stock solution Floral Geraniol 31.2ppm
780μl of Geraniol into 100ml of water (total)
1.2ml of stock solution
Sweet-caramel
Caramel Natural Flavour
1.4ml of flavour (Metarom)
Sweet-vanilla
Vanilla Extract (Club House)
0.9ml of extract Sourness Acetic Acid 1000ppm 6ml of white vinegar (5%) Solvent Ethyl Acetate 557ppm
0.120ml of the Ethyl Acetate
Woody Cedar Extract 0.6ml of extract Musty 2,4,6-Trichloroanisole 4.6ppm
115mg Trichloroanisole into 100ml ethanol
1.2ml of stock solution Spicy Eugenol 2.8ppm
78μl of Eugenol into 100ml of ethanol (40%)
1ml of stock solution Green/ Grassy Cis-3-Hexen-1-ol 100ppm 30μl of Cis-3-Hexenol Malty
Massive Irish beer
Nutty
Peanut Extract (Metarom) 9667ppm 2.9ml of Peanut Extract Buttery Diacetyl 2.24ppm
20μl of Diacetyl into 100ml (25ml vodka + 75ml water)
3.2ml of stock solution
Sulfur
Canned cooked corn
Table 1. Aroma Spikes for the Whisky Panel, July 2000
Based upon personal communications with John Piggott, 2000. Lee, Paterson, Piggott and Richardson, 2001
•
All samples are evaluated at 20% ABV. Blends are
prepared each day using room temperature distilled
water.
•
Samples are poured approximately 1 hour before
testing; 25 ml of each sample is poured into a 6 oz
blue glass copita and covered immediately with a lid.
•
Samples are labeled with three-digit blinding codes
and served at room temperature.
• At the start of testing, panelists receive a warm-up sample for calibration purposes.
• For each evaluation, the panelists receive one lidded 6 oz blue glass copita containing 25 ml of product.
• Panelists evaluate each sample monadically for aroma and flavor attributes.
• Panelists expectorate the samples for all attributes.
• Panelists cleanse their palates between samples with room temperature distilled water and unsalted soda crackers. Panelists are also provided covered mugs of steaming water with which to clear their nasal passages.
• Panelists get a five minute break between samples.
• The test is run in individual computerized booths under white lights.
Means are based upon 3 replications per product.
All responses were collected on 100-point unstructured line scales.
p-value Value LSD JW MM JD WR FR JAM
Medicinal Aroma 0.00 2.0 14.0 9.7 9.6 10.7 9.4 8.7
Phenolic Aroma 0.00 2.1 15.7 12.0 12.8 11.7 11.7 10.7 Tobacco Aroma 0.69 2.0 8.5 7.2 7.8 7.3 7.9 7.3 Cooked Cereal Aroma 0.75 1.2 5.9 5.5 5.3 6.1 5.2 5.3 Malty Aroma 0.05 1.8 7.7 7.5 8.0 9.0 7.2 7.2 Grassy Aroma 0.02 1.4 9.0 9.2 8.8 9.2 9.0 9.9 Floral Aroma 0.02 1.9 6.4 8.4 8.5 9.2 9.8 9.3 Fruity Aroma 0.01 2.2 11.8 14.6 15.5 16.3 13.5 15.2 Solvent Aroma 0.20 1.5 7.6 7.1 7.5 7.3 7.1 6.8 Vanilla Aroma 0.01 2.1 7.2 10.3 9.4 11.1 10.2 8.9 Oak Aroma 0.87 2.3 8.7 8.1 8.1 7.4 8.0 7.6 Cedar Aroma 0.27 1.8 6.4 7.5 6.7 5.8 6.4 5.5 Buttery Aroma 0.15 1.5 2.9 3.7 4.0 4.7 3.5 3.8 Nutty Aroma 0.35 1.6 8.1 9.3 9.3 10.5 9.4 9.0 Medicinal Flavor 0.00 2.2 15.3 10.4 9.2 10.7 9.9 9.4 Phenolic Flavor 0.00 2.1 17.5 12.5 13.7 13.1 13.0 11.9 Tobacco Flavor 0.19 2.0 9.8 7.9 8.0 7.5 7.9 8.0 Cooked Cereal Flavor 0.81 1.1 5.6 5.6 6.0 6.3 5.7 5.6 Malty Flavor 0.64 1.8 7.7 8.9 8.9 9.3 8.3 8.9 Grassy Flavor 0.00 1.3 8.2 9.4 9.4 10.0 9.3 9.0 Floral Flavor 0.00 1.9 5.9 9.0 7.9 9.1 10.2 8.0 Fruity Flavor 0.01 2.0 11.1 13.6 14.4 14.9 13.5 14.8 Solvent Flavor 0.07 1.3 8.0 7.3 6.7 7.8 6.5 6.4 Vanilla Flavor 0.01 1.7 7.5 9.9 9.8 10.7 9.8 10.5 Oak Flavor 0.65 2.1 10.3 9.1 8.6 9.2 8.3 8.9 Cedar Flavor 0.75 1.9 6.5 7.7 7.2 7.3 7.1 5.8 Buttery Flavor 0.53 1.4 2.6 3.7 3.8 4.1 3.8 3.3 Nutty Flavor 0.94 1.8 8.4 9.2 9.1 9.6 9.3 9.5 Sweetness 0.03 1.6 17.0 18.4 18.0 19.1 19.2 19.4 Sourness 0.12 1.1 9.6 9.7 9.0 9.9 9.2 8.3
Phenolic Flavour Medicinal Flavour Floral_Flavour Vanilla_Flavour Fruity_Flavour Grassy Sweetness Vanilla Aroma Malty Aroma -2.02 2.02 -2.02 2.02 Medicinal_Aroma Malty_Aroma Grassy_Aroma Fruity_Aroma Floral_Flavor JWB MM JD WR 4R JAM
Improving
•
Descriptive Analysis is an
analytical
method
•
The quality of the results of DA depends on both
accuracy & precision.
–
Are the results correct and repeatable?
•
Results depend on training assessors
–
“Experts” still require panel screening and
training
Two basic questions about DA
•
Can we get the panel right from the start?
•
What is the best possible panel?
What is an “order of operations”?
–
In mathematics an equation is calculated using…
–
BEDMAS (Brackets, Exponents, Divide, Multiply,
Add & Subtract)
The Sensory Order of Operations
1. Identify the attribute (attribute standard)
2. Rank its intensity
3. Scale the intensity (calibration standard)
Specific Standard Group of Attributes Verbal or Evocative
A primary reference exists that defines the attribute
completely
A few good examples provide the definition for several related attributes
No specific reference can be used, but the concept
can be communicated Sugar or Salt Fruit or floral Barnyard or Diesel
Attribute Identity
Full Scaling Rankable Off/On The attribute can be
measured across the full range with precision of <10% of the scale range
Across the range for the product, may be
detected at 2,3 or 4 levels
At the level in the product it is either absent or
present, but does not lend itself to scaling
Sucrose in juices Bitterness in black coffee Metallic in beverages
Attribute Classification System
Providing Immediate Feedback
Assists Learning
•
Feedback administration significantly influences
category learning
•
Observational feedback severely impairs learning vs.
feedback learning
•
Giving selectively positive or negative feedback will
impair learning vs. “full” feedback
•
Delaying feedback more than a few seconds severely
impairs learning
Ashby, G.F. and O’Brien, J.B. (2007)
Implicit Information –
The Feedback Calibration Method (FCM
®) is based upon providing
panelists with “true” information at the time they evaluate the
attribute.
If feedback is either untrue or trivial, the panelist will become
confused and the desired learning will not take place.
Meaningful targets for feedback depend upon;
• an understanding of the role of the stimulus-response curve of
any attribute
• the effect of context
• the intensity of the attribute for the product category
•
Immediate feedback provides panelists with strong
individualized method of learning attributes and scaling.
•
It also facilitates integration of new panelists.
•
Panels can develop and refine their own targets.
•
Calibration can be achieved through specific lexicons with
reproducible attribute standards.
•
Training times can be cut in half with no penalty in panel
performance.
Castura, Findlay and Lesschaeve; 2005, Findlay, Castura and Lesschaeve, 2007; Findlay, Castura, Schlich and Lesschaeve, 2006
2012
Whisky
Sensory Descriptive Analysis
Using Feedback Calibration
FCM® Sensory Descriptive Analysis
1. Recruit and screen panelists
2. Identify the key sensory attributes of the product range
3. Apply a sensory order of operations approach to attribute development and classification
4. Develop meaningful feedback targets for individualized training 5. Use Feedback Calibration sessions to train the panel
6. Set proficiency targets for panelists
7. Assess the proficiency of the panelists and panel 8. Finalize the ballot
9. Measure the attribute responses for the products 10. Analyze and interpret product results
•
Panelists were trained using reference standards and were
calibrated for the descriptive attributes using the
Compusense Feedback Calibration Method (FCM
®). All
descriptive attributes were rated on unstructured 100-point
line scales.
•
Testing was conducted at the Compusense Research Facility
in Guelph, July 2012.
•
Eleven trained panelists performed three replicate
evaluations of the 10 products using a balanced William’s
design, for a total of 33 evaluations per product.
•
All samples were evaluated at 20% ABV. Blends were prepared
each day using room temperature distilled water.
•
Samples were poured approximately 1 hour before testing; 25
ml of each sample was poured into a 6 oz blue glass copita
and covered immediately with a lid.
•
Samples were labeled with three-digit blinding codes and
served at room temperature.
Method of Evaluation: Lift the sample to the nose and remove the lid. Sniff rapidly and deeply 3 times. Repeat as needed.
Attribute Definition Left Anchor Right Anchor Reference Standard Medicinal Band-aid, antiseptic None Very Strong Aroxa capsule: 4-ethyl phenol Phenolic Phenolic (peaty) None Very Strong Aroxa capsule: Guaiacol Tobacco Tobacco, hay, dry grass None Very Strong Aroxa capsule: Beta-cyclocitral
Aroxa capsule: 3-ethyl pyridine Cooked Cereal Cooked cereal, cooked grains None Very Strong
Aroxa capsule: 2-acetyl pyridine Aroxa capsule: Methional Aroxa capsule: Isobutyraldehyde Malty Malt, malted barley None Very Strong Happy Home Malt Syrup Grassy Fresh cut grass, green leaves, cuttings, green beans,
green banana peel None Very Strong Aroxa capsule: Cis-3-hexenol Floral Roses, violets, lilacs None Very Strong Aroxa capsule: Beta-damascenone
Aroxa capsule: Beta-ionone Fruity Banana, apple, peach, pear, cherry, black currant,
prunes, plums, pineapple, orange, lemon, lime None Very Strong
Aroxa capsule: Isoamyl acetate Aroxa capsule: Ethyl hexanoate Solvent Nail polish remover, paint thinner None Very Strong Aroxa capsule: Ethyl acetate Vanilla Vanilla, vanillin None Very Strong Aroxa capsule: Vanillin
Oak Oak, sawdust, papery None Very Strong Aroxa capsule: Trans-2-nonenal Oak shavings
Cedar Cedar None Very Strong Cedar shavings Buttery Butter, diacetyl None Very Strong Aroxa capsule: Diacetyl
Nutty Hazelnut None Very Strong Aroxa capsule: 5-methyl-2-hept-4-one
Table 2. Aroma spikes for Whisky Panel, July 2012
Sensory Descriptive Attributes and Definitions
• At the start of testing, panelists received a warm-up sample for calibration purposes.
• For each evaluation, the panelists received one lidded 6 oz blue glass copita containing 25 ml of product.
• Panelists evaluated each sample monadically for aroma and flavor attributes.
• Panelists expectorated the samples for all attributes.
• Panelists cleansed their palates between samples with room temperature distilled water and unsalted crackers. Panelists were also provided covered mugs of steaming water with which to clear their nasal passages.
• Panelists were given a five minute break between samples.
• The test was completed in individual computerized booths under white lights.
•
The data was collected using Compusense
®at-hand
software.
All raw data had been reviewed.
•
Mean scores and Fisher’s LSD analysis (at 5%) were calculated
in Senstools Version 3.3.1 and used to identify significant
differences between the products.
•
Generalized Procrustes Analysis (GPA) was performed in
Senstools Version 3.3.1.
Analysis
95% Confidence Intervals for All Whisky
-0.50 1.50 -0.70 0.70 JAM FR WR JD MM JWBWhisky Sensory Map
-0.50 0.60 -0.70 0.70 JAM WR JD MM JWB
Whisky Sensory Map
Dimensions 3 and 4
Comparing Y2K to 2012
ATTRIBUTE
Significant at p<0.05 Least Significant Difference Year 2000 Year 2012 Fruity aroma 3.2 1.7 Floral aroma 3.3 1.7 Phenolic aroma 5.4 1.8 Smoky aroma 5.1 1.5 Sweet aroma 3.1 1.7 Phenolic flavour 3.8 1.6 Smoky flavour 3.8 1.5
On unstructured line scale
anchored at 0 and 100
3.96
1.64
Whisky Training Time (h)
12
6
Table 3. Least significant difference and training time of panels after introduction of FCM training. Attributes selected are matched between both panels
FCM Introduced in 2006
|
2000’s
|
|
1990’s
2010’s
40 hours
20 hours
6 hours
The Effectiveness of FCM Training
Calibrated Descriptive Analysis
When using FCM training…
•
Analytical sensory profiles of products are both more
accurate and precise.
•
A library of the sensory properties of products can be
created
•
Competitor profiles are meaningful
•
Reliable multi-attribute measures of sensory shelf life
can be obtained.
References
• Ashby, G.F. and O’Brien, J.B. (2007). The effects of positive versus negative
feedback on information-integration category learning. Journal of Perception & Psychophysics 69(6): 865-878
• Castura, J.C., Findlay, C.J., Lesschaeve, I. (2005) Monitoring calibration of
descriptive sensory panels using distance from target measurements. Food Quality and Preference 16(8): 682-690.
• Findlay, C.J., Castura, J.C., Lesschaeve, I. (2007). Feedback Calibration: a training method for descriptive panels. Food Quality and Preference 18(2): 321-328
• Findlay, C.J., Castura, J.C. Schlich, P., Lesschaeve, I. (2006). Use of feedback calibration to reduce the training time for wine panels. Food Quality and Preference 17(3-4): 266-276.
• Lawless, H. T., & Heymann, H. Sensory evaluation of food, 1998. Chapman Hall, New York.
• Lee, K.-Y. M., Paterson, A., Piggott, J. R. and Richardson, G. D. (2001), Origins of Flavour in Whiskies and a Revised Flavour Wheel: a Review. Jnl Institute Brewing, 107: 287–313.
Thank You
compusense.com
+1 519 836 9993 – Worldwide
1 800 367 6666 – Toll free in North America
