CHAPTER 6: FACTORS OF DIFFICULTY IN CONDUCTING RESEARCH IN APPLIED SETTINGS WITH SPECIFIC REFERENCE TO THE PRESENT STUDY
The f a c i a l s c r e e n
D e sp i t e Mudfo rd 's (1987) r e s e a r c h d e m o n s t r a t i n g the a c c e p t a b i l i t y of f a c i a l s c r e e n i n g t o n u r s i n g s t a f f , t h e i n t r o d u c t i o n of t h i s p ro c e d u r e g e n e r a t e d much o p p o s i t i o n from n u r s i n g s t a f f who i n i t i a l l y were not i n v o l v e d d i r e c t l y in t h e program p r o c e d u r e s . I t i s t hought t h a t t h e a pp ea r an ce of t he f a c i a l s c r e e n , which when a p p l i e d r e se m b le d a b l a c k hood, was r e s p o n s i b l e f o r t h e n e g a t i v e r e a c t i o n . I t i s u n f o r t u n a t e t h a t the time of use of t h e f a c i a l s c r e e n immediately f o l l o w e d a mass a d v e r t i s i n g campaign in t h e media f o r t h e p r e v e n t i o n of AIDS which r e l i e d h e a v i l y on v i s u a l imagery of d e a t h in t h e form of t h e hooded "grim r e a p e r " . However as t h e e f f e c t i v e n e s s of t he p ro ce du re became e v i d e n t t h e n e g a t i v e r e a c t i o n s d i s a p p e a r e d and one of t he most outspoken "members of t he o p p o s i t i o n " became one of the most a r d e n t s u p p o r t e r s of t h e program.
The f a c i a l s c r e e n proved d i f f i c u l t t o use when f i r s t i n t r o d u c e d in t he s t r u c t u r e d a c t i v i t i e s s e t t i n g in phase I , t r i a l 1. The l evel of c o n c e n t r a t i o n r e q u i r e d t o p r o v i d e c ue s f o r t he t a s k s t o be completed by t h e c l i e n t , i n t e r f e r e d g r e a t l y w it h o b s e r v i n g h i s problem b e h a v i o r s and p r o v i d i n g t h e a p p r o p r i a t e v erbal i n s t r u c t i o n r e q u i r e d by t h e formal program, and then c o n s e q u a t i n g h i s r e s p o n s e t o t h a t i n s t r u c t i o n w i t h a p p l i c a t i o n of the s c r e e n f o r non-compliance wi th t h e v e r b a l i n s t r u c t i o n . I t took some p r a c t i c e t o devel op s k i l l a t c o n c e n t r a t i n g on both p r o c e d u r e s and t h i s d i f f i c u l t y i s t h ou gh t t o be r e s p o n s i b l e f o r the low r a t e (15%) of r e d u c t i o n of problem b e h a v i o r s in t h e f i r s t
w e e k of i n t e r v e n t i o n in t h i s s e t t i n g . I d e a l l y t h e s e t a s k s s h o u l d h a v e b e e n d o n e b y t w o d i f f e r e n t t e a m m e m b e r s b u t it w a s i m p o s s i b l e to p r o v i d e t h i s level of s t a f f i n g . A d m i n i s t r a t i v e c r i t i c i s m s of c o s t T h e p r o g r a m w a s c r i t i c i z e d at a d m i n i s t r a t i v e level w i t h i n the i n s t i t u t i o n b e c a u s e of t h e d e m a n d m a d e on s t a f f i n g n u m b e r s o v e r the d u r a t i o n of th e p r o g r a m . T h e p r o g r a m a c t i v e l y o c c u p i e d s t a f f t i m e o v e r a p e r i o d of 2 6 w e e k s of b a s e l i n i n g a n d i n t e r v e n t i o n e x c l u d i n g the r e v e r s a l p h a s e . W h e n o n e c o n s i d e r s t h a t t h i s is 6 m o n t h s o r h a l f a y e a r it d o e s s e e m a l o n g t i m e a n d s u c h c r i t i c i s m p r i m a f a c i e w o u l d s e e m to b e j u s t i f i e d . In o r d e r to o f f s e t s u c h c r i t i c i s m it m u s t b e r e c o g n i z e d th a t a p p l i e d b e h a v i o r a n a l y s i s p r o c e d u r e s a r e la b o r i n t e n s i v e a n d it m a y b e th at fo r m a l c o s t b e n e f i t a n a l y s e s s h o u l d b e c o m e p a r t of t h e o v e r a l l e v a l u a t i o n of p r o g r a m s . In t h i s p a r t i c u l a r c a s e t h e c o s t of e v e n s u c h a la r g e n u m b e r of s t a f f o v e r a 6 m o n t h p e r i o d w o u l d p a l e in t o i n s i g n i f i c a n c e w h e n c o m p a r e d to th e c o s t of c a r i n g fo r s u c h a y o u n g p e r s o n f o r th e r e s t of h i s life in an i n s t i t u t i o n a l s e t t i n g . H o w e v e r t h i s a u t h o r b e l i e v e s th a t un t i l s u c h a n a l y s e s a p p e a r in the l i t e r a t u r e it is e a s y fo r a d m i n i s t r a t o r s t o i g n o r e t h i s a s p e c t a n d to c r i t i c i z e the r e q u i r e d la b o r i n t e n s i v e n e s s i n s t e a d . S u c h a n a l y s e s m a y a l s o p r o m p t t o l e r a n c e of p r o c e d u r e s a i m e d at i n c r e a s i n g the p o s s i b i l i t y of a c h i e v i n g g e n e r a l i s e d b e h a v i o r c h a n g e . H a n d l i n g the r a w d a t a T h e la rg e n u m b e r of b e h a v i o r s m o n i t o r e d , (7), o v e r the n u m b e r of s e t t i n g s , (5), fo r the 2 6 w e e k s , g e n e r a t e d a n e n o r m o u s a m o u n t of r a w da t a . In o r d e r to p r e s e n t the d a t a in s o m e m a n a g e a b l e f o r m it w a s n e c e s s a r y to c o l l a p s e th e d a t a , a n d a f i v e d a y w e e k b l o c k
average was chosen for convenience of presentation although such averaging obviously loses the richness of the day to day variations in behavior. While this problem could have been avoided by targetting fewer behaviors, it was theoretically sound to work on all behaviors which were observed to culminate in tantrums given that there was no way of knowing whether these behaviors were seizure prodromata or the residual stereotypical behaviors of a severely to profoundly intellectually disabled person.
Interobserver agreement
A further major difficulty evolved around data collection to ensure an acceptable level of interobserver agreement. Page and Iwata (1986) recommend that applied researchers consider four interrelated decisions in assessing and reporting interobserver agreement. These involve deciding upon: (1) Which agreement index to report with shorter time intervals allowing exact block-by-block agreement to be calculated being the method recommended; (2) The unit score to be presented, for example, a composite score or scores for specific behaviors, with precedents having been set for both; (3) The time span, for example, session or trial, with the recommendation that agreement be assessed over the period for which the dependant variable is calculated and reported but with the proviso that the molecular level of trial reliability is preferable; (4) Whether to co-plot the data collected by the second observer. Various arguments are offered by the authors on the advantages and disadvantages of this procedure and they conclude that the strategy has seen only very