Rural Behavioral Health
Programs and Promising
Practices
June 2011
U. S. Department of Health and Human Services
Health Resources and Services Administration
2
This document was prepared for the U.S. Department of Health and Human Services (HHS), Health
Resources and Services Administration (HRSA), Office of Rural Health Policy (ORHP) under HRSA
Contract # HHSH250200866010C.
This publication lists non-Federal resources in order to provide additional information to consumers. The
views and content in these resources have not been formally approved by HHS. Listing these resources is
not an endorsement by HHS or its components.
3
Table of Contents
Overview ... 4
Introduction ... 6
Methodology ... 8
Overview of Nominated Programs ... 9
Developing a Promising Practice ... 11
Focused Technical Assistance ... 16
Conclusions ... 21
References ... 23
Appendices ... 24
Appendix A: Questions included in the Nomination Form
... 24
Appendix B: Phone Interview Follow-Up Questions
... 29
Appendix C: Description of Rural Behavioral Health Programs
... 30
4
Rural Behavioral Health Programs and Promising Practices
Overview
The U.S. Department of Health and Human Services (HHS), Health Resources and Services Administration (HRSA), Office ofRuralHealthPolicy(ORHP)coordinatesactivitiesrelatedtoruralhealthcarewithintheU.S.DepartmentofHealthand HumanServices.ORHPhasdepartment-wideresponsibilityforanalyzingthepossibleeffectsofpolicydecisionson62 millionresidentsofruralcommunities.ORHPwascreatedbySection711oftheSocialSecurityActtoadvisethe
SecretaryofHealthandHumanServicesonhealthissueswithinruralcommunities,includingtheeffectsofMedicareand Medicaidonruralcitizens’accesstocare,theviabilityofruralhospitals,andtheavailabilityofphysiciansandother healthprofessionals.ORHPadministersgrantprogramsdesignedtobuildhealthcarecapacityatboththelocalandState levels.
Inparticular,thecommunity-basedgrantprogramswithinORHPthatdrawauthorityfromSection330AofthePublic HealthServiceAct,suchastheRuralHealthCareServicesOutreachgrantprogram,aremandatedto“expandaccessto, coordinate,restrainthecostof,andimprovethequalityofessentialhealthcareservices,includingpreventativeand emergencyservices,throughthedevelopmentofintegratedhealthcaredeliverysystemsornetworksintheruralareas andregions”.Asaresultoftheprogram’slegislativefocus,theOutreachgrantprogramprovidesfundingtoconsortiafor thedirectprovisionofhealthcareservicesaswellasforcommunityhealthservicecollaboration.
Currentlytherearemanychallengestoprovidingmentalhealthandsubstanceabuse(behavioralhealth)careinrural America.Assuch,ORHPsupportedastudyin2008toexaminethebarrierstoevaluatingprogramsinruralandfrontier behavioralhealthandtodisseminatethisinformationamongcurrentgranteesandfutureapplicantsfortheRuralHealth CareServicesOutreach,RuralHealthNetworkDevelopment,andRuralHealthNetworkDevelopmentPlanningprograms thathaveafocusonbehavioralhealthdeliveryinruralcommunities.Theintentwastocloselyexaminetheseprograms inadescriptivemannerandoutlinerequirementstomovetheseprograms/practicestomorerigorousscientific
validationthatwouldbevaluableindemonstratingtheireffectiveness.Thisdocumentwillallowruralbehavioralhealth programstolearnfromtheinformationgatheredinthisstudyandthevalueofcollectingandusingdatatomake programimprovements,demonstrateeffectivenessandimportanceofservices,andfindfundingforsustainability. Evidence-basedPractices(EBPs)arepracticesthatintegratebestresearchevidencewithclinicalexpertiseandpatient values(InstituteofMedicine,2001).EBPsareincreasinginthefieldofbehavioralhealth:consumerswanttreatments withproveneffectiveness;providerswanttoincreasetheirknowledgetoenableprovisionofthosetreatments;research andfundingentitieslookforopportunitiestoidentifynewevidence-basedtreatments;and,insomecases,policymakers arelegislatingtheprovisionofevidence-basedpractices(e.g.,OregonStatute182.525,whichmandatesexpendituresfor evidence-basedpractices).
OverallthisshifttowardtheuseanddevelopmentofEBPsinbehavioralhealthservicesismovinginapositivedirection, withtheendresultbeingtheprovisionofmoreeffectiveandstandardizedbehavioralhealthtreatments.However, manyoftheseEBPsaretestedinurbanareas,andarenoteasilyadaptedtoruralareaswithoutlosingkeycomponents ofthetestedpractice.
DuetothedifficultyofdevelopingandevaluatingEBPs,theterm,“promisingpractice,”hasbeenusedtoreferto behavioralhealthpracticesthatdonotyethavetheevidence-basetobeconsideredEBPs,butwhichappeartobe effectivebasedonalessstringentdefinitionofresearchevidence(e.g.,usingacomparisongroupthatisnot well-matchedtothetreatmentgroup)orpreliminaryand/orsimpledataevaluation.Theterm,“bestpractice,”isusedto refertoapracticewhichisgenerallythoughttobeeffectivebaseduponanecdotalevidence,butforwhichobjective dataislacking.Withoutsomedegreeofdataandevaluationtosupporttheeffectivenessofapractice,itisdifficultto developanevidence-basedpractice.
5
A practiceisdefinedasaspecificprocedurethatcanbegeneralizedtotargetedsituations.Forexample,inusinga
specificdepressionscreeningquestionnaire,therearestandardprocedurestofollow(e.g.,aspecificorderofquestions, aspecificmethodofscoringtheresponses).Thisscreeningquestionnairemaybeusedinavarietyofsettings,witha varietyofpatients,buttherelianceonaspecificproceduremakesitapractice.Incontrast,a programencompasses multiplepractices,activities,andstrategies. Forexample,a“DepressionAwarenessProgram”wouldnotonly
incorporatethepracticeofdepressionscreenings,butwouldusetargetedbehavioralhealtheducation,outreach,health fairs,andmarketing,whichareallpartinparcelofaprogramandspecifictothetargetpopulation.Aprogrammaybean EBP,promising,orbestpracticeifprogramvariablescanbegeneralizedtodifferentpopulationsand/orsettings.
Althoughthereareanumberofinterestingandnovelbehavioralhealthpracticescurrentlybeingimplementedinrural America,therearefewerpracticesthatareevaluatedandtestedfortheireffectivenessthaninurbanareas.Muchofthe evidencefortheeffectivenessofthesepracticescomesfromsubjectiveexperienceandobservation,ratherthandata trackingandanalysis.
Thisreportdocumentsastudytoidentifyinnovativeruralprograms(i.e.,setsofpractices,activities,andstrategies),and usestheresultstosuggestbasictoolsforruralorganizationstoembarkuponthestepsnecessarytomoveaprogram thatisperceivedtobeeffectiveintoapromisingpractice.Achievingthestatusofpromisingpracticeisbeneficialto programsbecauseitprovidesevidenceoftheprogram’seffectivenesstoatargetpopulationandfunderswhomay investintheprogram,andalsoenablesotherruralprogramstoadaptandusetheprogramintheirowncommunity.This documentwillprovideanoverviewoftoolssuchascommunitybuilding,grant-writing,datacollection,andprogram definition,whichmayproveusefultoorganizationsthatareinterestedintransitioningtheiruniqueandinnovative behavioralhealthprogramstopromisingpractices.
Intheinitialstageofthisstudy,theWesternInterstateCommissionforHigherEducation(WICHE)MentalHealth
Programidentifiedprogramsinruralbehavioralhealth.WICHEstaffspokewith62ruralbehavioralhealthandsubstance abuseprovidersaroundthecountrywhohaddevelopednovelandinnovativeprogramstoimprovebehavioralhealth servicesinruralareas.Althoughmostprogramscollectedsomeformofdata,fewprogramsutilizedthosedatato determinetheirprograms’effectiveness.Overthecourseoftheseinterviews,mostorganizationsreportedneeding additionalresourcesandknowledgeinordertoevaluatetheeffectivenessoftheir programs.
• Manyprogramsreportedalackofstafftimeandfundingtosupportprogramevaluationand/orresearch activities.Overworkedstaffmembersareunabletosetasidetimefordata-relatedactivities,suchasthe developmentofdatacollectionprocedures,datacollectionitself,anddataanalysis.Inaddition,staffmembers donothavetimetodocumentprogram-relatedactivities,orapplyforfundingtosupportprogramand
evaluationactivities.
• Findingassistancefromoutsideresearcherswithexpertiseinresearchdesign,datacollection,andprogram evaluationisalsoachallenge,asmostprogramsdonothavethefinancialresourcestosupportexternal consultants.Thus,ruralorganizationswishingtoevaluatetheeffectivenessofaprogramareinadoublebind– theyoftendonothavethetimetodevelopadatacollectionandevaluationprocedure,andalsodonothave financialresourcestohireanyonetoassistwiththisprocess.
• Evenifprogramsdohavethestafftimetodevotetoevaluationactivities,theyoftendonothavetheexpertise tocarryoutthoseactivities.Providersingeneral,andruralprovidersspecifically,areoftennottrainedin researchdesign,datacollectionandanalysis,andprogramevaluation.Inaddition,providersoftendonothave knowledgeorexperiencewithgrant-writing(e.g.,wheretoidentifygrantopportunities,howtoapplyforthem, andhowtowritegrantapplications).Withoutsuccessfulgrant-writingopportunities,itbecomesnearly
6
• Afurtherhurdleinaprogram’sabilitytobecomeapromisingpracticeisthelackofunderstandingastowhat constitutesapractice.Tenpercentoforganizations(7)respondedtoWICHE’sinitialcallfornominationsby sendinginformationabouttheorganizationasawhole;nominatingagroupofprovidersorafacilityratherthan amethodofprovidingtreatmentorinsomewayamelioratingtheproblemsfacingindividualswhoneed
behavioralhealthservicesinunderservedruralareas.Someprogramscannotreachthestatusofpromising practicenotonlybecausetheydonothavedata,butalsobecauseofalackofunderstandingastowhat constitutesapractice.
Thisdocumentisprimarilytargetedtowardproviders,butismeanttoapplytoarangeofindividuals:
• Providers canlearnaboutdatacollectionandanalysis,communityengagement,andgrant-writing.
• Policy makerscanlearnwhatruralorganizationsneedtoimplementanddeveloppromisingpractices,andhow
thisinformationisinstructiveforlegislativechangestosupporttheseefforts.
• Researchers canlearnwhatprovidersneedinthewayofresearchtoolstoenableeffectiveevaluationofrural
practices(e.g.,surveydesignassistance).
• Consumers canlearnwhattolookforinordertoevaluatewhetheragivenpracticeiseffectiveintheirtreatment
andwhetheritwouldbeagoodfitfortheirbehavioralhealthconcerns.
Theintentofthisdocumentistoprovideastartingpointforruralstakeholderstocollaborateandensureinnovative practicesdemonstrateeffectivenessbasedonsoundscienceandhavethefundingnecessarytosustain their activities.
Introduction
Mentalillnessesaffectupto1in5individualsintheUnitedStates(U.S.DepartmentofHealthandHumanServices, 1999).Whencomparedtomajorphysicalillnesses,suchascardiovasculardiseases,cancer,respiratoryconditions,and infectiousdiseases,mentalillnessrankssecondinthecalculatedburdenofdisease(i.e.,thenumberofyearsoflifelost toprematuredeathandyearslivedwiththedisability;Murray&Lopez,1996).Whileoftenperceivedtohaveminimal influenceonanindividual’slife,mentalillnessesareseverelydisabling.Thedisabilityassociatedwithmajordepressionis equivalenttothedisabilityassociatedwithblindnessorparaplegia,andthedisabilityassociatedwithactivepsychoses seeninschizophreniaisequalinburdentoquadriplegia(U.S.DepartmentofHealthandHumanServices,1999).Arecent reportbytheWorldHealthOrganization(2008)indicatesmentalillnessesarethebiggesthealthburdeninNorth
America,largelyduetodisabilitywhichresultsinalossofproductiveyearsoflife.
Behavioral Health in Rural America
Approximately20percentoftheUnitedStatespopulationisaffectedbybehavioralhealthissueseachyear(Kessleret al.,2004),andalthoughtherateofbehavioralhealthproblemsdoesnotdiffersubstantiallybetweenruralandurban areas, theexperienceofmentalillnessdiffersdramatically(Mohatt,Adams,Bradley,Morris,2005).Theprevalenceof andentryintocareforbehavioralhealthproblemsisgenerallycomparableinruralandurbanpopulations,butthe qualityofcarethatruralpatientsreceiveforbehavioralhealthproblemsmaybepoorer,particularlyforresidentsin outlyingruralareas(Fortney,Rost,&Zhang,1999;Kessleretal.,2004).RuralAmericanswithbehavioralhealthdisorders aresignificantlylesslikelytoreceiveanytypeoftreatmentfortheirbehavioralhealthproblemsthanurbanand
suburbanAmericans(Wang,Lane,Olfson,Pincus,Wells,&Kessler,2005).Individualslivinginruralareasaresignificantly lesslikelythantheirurbancounterpartstoreceivespecialtybehavioralhealthcare(Wang,et.al.2005)andmorelikely toreceivegeneralmedicalcareonlyorhumanservicesonly(e.g.,pastoralcounseling)(Wang,Demler,Olfson,Pincus, Wells,&Kessler,2006).Unfortunately,thelikelihoodofreceivingminimallyadequatebehavioralhealthcareinthe generalmedicalsectorandhumanservicessectorissubstantiallylowerthaninthespecialtybehavioralhealthsector (Wang,et.al. 2005).
7
Forthepast40years,60percentofruralAmericahasbeenunderservedforbehavioralhealthneeds(NewFreedom CommissiononMentalHealth,2003),andmorethan85percentofthenation’sbehavioralhealthprofessionalshortage areasarelocatedinruralAmerica(Bird,Dempsey,&Hartley,2001).Individualslivinginruralcommunitiesarefacedwith threedistinctburdenstoadequatebehavioralhealthcare:accessibility,availability,andacceptability(Mohattetal., 2005;NewFreedomCommissiononMentalHealth,2003).Intermsofaccessibility,ruralresidentstypicallydonotknow whentheyneedbehavioralhealthcare,wheretheycanfindthatcare,andwhatcareoptionsareavailabletomeettheir behavioralhealthneeds(NewFreedomCommissiononMentalHealth,2003).Whentheydofindbehavioralhealthcare, itisnotuncommonforindividualsinruralareastotravelhundredsofmilestoaccessthoseservices.Theavailabilityof behavioralhealthprovidersinruralareasistoolimitedtosupporturbanmodelsofservicedelivery,inwhichindividuals needingbehavioralhealthserviceshaveavarietyofbehavioralhealthprovidersfromwhichtochoose.Inadditionto physicalbarrierstobehavioralhealthcare,ruralresidentsalsofacepsychologicalbarriers.ManyAmericansattach stigmatohavingorseekinghelpforbehavioralhealthproblems.Thisstigmaisparticularlydetrimentalinruralareas wherethereislittletonoanonymityinseekingbehavioralhealthservices(Mohattetal.,2005).Thesepsychological barriersseverelylimittheacceptabilityofbehavioralhealthservicesinruralareas.
Challenges Confronting Rural Evidence-Based Practices
ThebehavioralhealthcarebarriersfacedbyruralAmericansmakebehavioralhealthcaredeliveryinruralareas substantiallydifferentthaninurbanareas,whichisaparticularchallengeforcreatingevidence-basedpracticesforand deliveringevidence-basedpracticestoruralresidents.Inrecentyears,evidence-basedpractices(EBPs)haveemergedas ameanstoensurequalitybehavioralhealthcareamongindividualssufferingfrommentalillnesses.EBPshavebeen extremelysuccessfulintreatingmentalillnesses.However,mostEBPsaredevelopedinurbanareasandtestedonurban residents,withlittle,ifany,thoughtgiventohowtheymightbeimplementedinruralareaswhereresourcesaremuch scarcer.Thedrasticdifferencesinbehavioralhealthcareavailability,accessibility,andacceptabilitybetweenruraland urbanresidentsmakeitchallengingtoconductmanyEBPsinruralareas,leadingtoastrongneedtodevelop rural-specificEBPs,and/ordeterminewhetherandhowexistingEBPscanbemodifiedtoproducesimilartreatmenteffectsin ruralareasasareobservedinurbanareas.
Unfortunately,amajorityofruralareaslacktheresourcesneededtodevelopandsustainrural-specificEBPs.Ruralareas arealreadyunderservedandunderfunded,andmostruralbehavioralhealthprofessionalsdonothavethetimeor trainingtoplanandconductclinicaltrials,ortoanalyzeanddocumentresultsfromscientificstudies.Evenwithtimeand expertiseavailabletocarryoutanddocumentscientificstudiesofrural-specificpractices,ruralbehavioralhealth
servicesoftendonothavethefundstocarryoutpilotstudiesnecessaryforobtaininglargergrants.Inmanyruralareasit isthereforeachallengetocarryoutlarge-scaleclinicaltrialsnecessarytotesttheeffectivenessofruralprograms.In addition,fewagenciesprovidefundingopportunitiestoindividualsoutsideofacademiccenters,whichonceagainputs ruralareasatadisadvantagefordevelopingnewEBPsspecificforruralareas,orensuringEBPsareeffectivelyadapted forruralareas.
ThefactthatruralbehavioralhealthprovidersdonothavemanyresourcestoimplementEBPs,ortodevelopnewEBPs leavesruralprovidersfacingarealchallengetoprovidingEBPsforruralresidents.However,manyruralprovidershave developedprogramsspecifictotheirareas,whichmaybequitesuccessfulinimprovingtheavailability,accessibility,and acceptabilityofbehavioralhealthservicesinruralareas.Ruralprovidersareinabetterpositiontounderstandthe uniqueneedsoftheircommunities,andmanyhaveadaptedtheirprogramsandtreatmentstomeetthoseneeds. Unfortunatelythereisnomediumforsharingthisknowledge,anditoftenremainsisolatedinspecificcommunities,with fewresourcesavailablefordocumentingandpublicizingtheeffectivenessoftheseuniqueruralprograms.
8
Goals
Inresponsetotheneedfortechnicalassistanceindeterminingaprogram’seffectiveness,WICHEconductedastudyto a)identifyruralprogramsthatwereattemptingtomeetabehavioralhealthneedinaruralarea,andb)determinethe generalresourcestheseprogramsneededtomovetheirprogramstowardpromisingpractices.Thisprocessincludeda solicitationfornominationsviaaweb-basedsurvey,andfollow-upinterviewswithkeyindividualsfromeachprogram. Theendresultofthisprocessisasnapshotofapproximately60ruralbehavioralhealthprogramsthroughoutthe country,andamoredetailedunderstandingofwhatruralprogramsneedtodotobecomepromisingpractices.
Methodology
Theidentificationofruralbehavioralhealthprogramsinvolvedamulti-stepprocessthatfirstsolicitednominationsfor novelandinnovativeprograms,andthencollectedadditionalinformationabouttheprogramsthroughasurveyand follow-upinterviews.Below,isadiscussionoftheprocedureforsolicitingnominations,surveying,andinterviewingeach oftheseprograms.
Reaching Programs/Practices
Criteriawerenotgivenforthetypeofprogramstonominateforinclusioninthisstudy(e.g.,hotline,outreach),other thanthecriteriathattheprogramhaveapositiveimpactonthebehavioralhealthofindividualslivinginruralareas. Nominationsweresolicitedviae-mailannouncementsdistributedtonationalruralhealthorganizations(theNational AssociationforRuralMentalHealth,theNationalAssociationforRuralHealth,theNationalAssociationofStateMental HealthProgramDirectors),nationalruralbehavioralhealthfundingagencies(SubstanceAbuseandMentalHealth ServicesAdministration,HealthResourcesandServicesAdministration),aswellasorganization-specificcontactssuchas theRuralAssistanceCenterandtheWICHEMentalHealthProgram.Thepurposeofthesee-mailswastoensurebroad distributionoftheannouncementtoruralconsumers,providers,andpolicymakers.
Theannouncementstatedthegoalofthesurveywastoidentifypracticesthatweresuccessfulinmeetingabehavioral healthneedinaruralarea.Thedefinitionof"meetingabehavioralhealthneedinaruralarea"waspurposefullyvague. Potentialrespondentswereinformed“meetinganeed"couldincludeincreasingaccesstoand/oravailabilityof
behavioralhealthservices,increasingawarenessaboutbehavioralhealthissues,recruitmentand/orretentionof behavioralhealthproviders,treatmentofabehavioralhealthproblem,andpreventionservices,amongothers.The diversityofprogramsrespondingtothenominationindicatesthereareverydiversebehavioralhealthneedsinrural areas,andalargenumberofwaystomeetthosevariousneeds.
The Survey
Individualswhonominatedprogramsbyrespondingtothesurveyansweredaseriesofquestionsdesignedtocreatea comprehensivesummaryoftheprogram,anddeterminewhethertheprogramshadanydataontheireffectiveness.The surveyisincludedasAppendixAandincludedcategoriesofquestionssuchas:contactinformation,characteristicsofthe nominatedprogram,characteristicsofthetargetedpopulation,trainingofstaff,documentationofprogram-related information,anddatacollection/researchcapacity.
9
Phone Interviews
Phoneinterviewswereconductedwith62organizationsthatcompletedthesurvey,tocollectmoreinformationonthe nominatedprograms.Organizationsthatstartedbutdidnotcompletethesurvey,ordidnotrespondtorequestsfor phoneinterviews,werenotinterviewed.Theinterviewsfocusedonfurtherunderstandingtheprograms,andincluded additionalquestions,suchasthosedirectedatthesustainabilityoftheprogram,anditsuniqueness.Thelistofquestions usedforthesephoneinterviewsappearsinAppendixB.Inadditiontothesestandardquestions,phoneinterviewsalso includedprogram-specificquestionswhosegoalwastoclarifyresponsesgivenonthesurvey.
Site Visits
ElevenprogramswereselectedforsitevisitsbytheprojectteamandprojectofficerattheOfficeofRuralHealthPolicy. Incaseswheretheprogramwasconfinedtoonelocation,thesesitevisitsincludedanin-persontouroftheprogramand meetingswithessentialprogramstaff.Incaseswheretheprogramwasmoredistributed,aphonecallwasconducted, includingasmanystaffmembersaspossible.Thegoalwassimplytogetabroaderpictureoftheprogramsbyspeaking withpeoplenotontheinitialphoneinterviews(providers,directors,consumers)andtovisittheprogramsites.
Programswereselectedforsitevisitsbasedongeographicaldistribution,aswellasthetypeofprogramitrepresented (e.g.,hotline,integratedcareclinic,etc.).Thegoalofthesitevisitselectionprocesswastogainanin-depthviewofa varietyofprogramsthroughoutallregionsofthecountry.Therefore,ageographicallyandtopicallydiversesetof programswasselectedforthesitevisits.
Overview of Nominated Programs
Sixty-nineprogramsfromacrossthecountrycompletedtheinitialsurvey.Theprograms,representing32States,were spreadacrossallfourCensusregions,andallnineCensusdivisions.Figure1showsthelocationsoftheseprograms(note thatsomelocationsnominatedmorethanoneprogram,sotherearefewerthan69locationsinthefigure).Although therearelikelysubstantiallymorebehavioralhealthprogramsintheU.S.specificallydesignedtoserveruralareas,the factthatrespondentsrepresentedallareasofthecountrysuggeststheprogramsdescribedinthisdocumentare somewhatrepresentativeofthetypesofruralbehavioralhealthprogramsimplementedinruralareas.
10
Categorization of Nominations
Afterlearningaboutthenominatedprograms,itbecamecleartheyfellintooneofelevencategories,basedonthetype ofprogramand/orpopulationbeingserved.Someprogramsfellundermorethanonecategory,butallwereabletobe encompassedbythecategoriesbelow.
• Programs adapting EBPs for rural areasfoundwaystocarryoutexistingEBPsinruralareas,despitebeing unabletoreplicateprogramswithfullfidelity.
• Community education and outreachprogramseducatedindividualsinthecommunityaboutbehavioralhealth issues.
• Court teamsaimedtokeepindividualswithbehavioralhealthissuesfromreturningtothejusticesystem.
• Crisis servicesprogramsprovidedcareinsomewayforacutebehavioralhealthproblems.
• Hotlineprogramsusedtelephoneservicestoprovidesupporttoruralresidentswithbehavioralhealthconcerns or issues.
• Integrated careprogramsblendedphysicalandbehavioralhealthservicesinsomeway.
• Peer supportprogramsusedpeerstoprovidebasictreatmentorsupporttoindividualswithmentalillnesses.
• Programsservingspecial populations targetedinterventionstoaparticularpopulation,suchaschildren or the
elderly.
• Telemental health programsusedphoneorvideoconferencingtoprovidebehavioralhealthtreatment.
• Training programseducatedfutureruralbehavioralhealthproviders.
Themes of Robust Programs
Afterreviewingthesurveyresponses,talkingwithprograms,andconductingthesitevisits,afewthemesemergedas beingindicativeofaprogram’ssuccessandabilitytoserveitsruralcommunity.Eachofthesethemesisdescribed, below.ThesethemesareusedintheprogramdescriptionsinAppendixCtohighlighteachprogram’sparticular
strengths,andarealsoexpandeduponinthenextsection,todescribehowruralorganizationsingeneralcanstrengthen theirprogramsineachoftheseareasinanefforttodeveloptheiruniquepracticesintopromisingpractices.The
interviewandselectionprocessledtoinsightsintohowprogramscanexcelineachoftheseareas,andbegintodevelop thedatacollectionandevaluationtoolsneededtobecomeapromisingpractice.
Relevance to Rural
Aprogram’srelevancereflectsitsspecificitytoaruralcommunity.Someprogramsbringmuch-neededservicestorural areas,butitisunclearwhethertheprogramsweredesignedfororadaptedtotheruralarea.Aprogramsuccessfulinits relevancetoruralcommunitiesdevelopsaprogramspecificforaruralcommunity,orimplementsanestablished programinawaythatisspecificallyadaptedforaruralcommunity.
Impact on Rural
Aprogramthatincreasestheavailability,accessibility,and/oracceptabilityofbehavioralhealthservicesinruralareas hasasuccessfulimpactonruralcommunities.Aprogramwithhighimpactonruralisaddressingoneormoreofthe barrierstobehavioralhealthservicesinruralcommunities.
Sustainability and Expansion Capability
Sustainabilityandexpansioncapabilityreflectstheawarenessofandabilitytoacquirelong-termfundingforthe program.Ruralprogramsaredependentonasustainableandexpandablesourceoffundingfortheirprograms.Some highlypromisingruralprogramscannotcontinueorexpandduetolackoffunding.Onefeatureofprogramsthatare successfulinmeetingaruralneedisarelianceonmultiplefundingsources,andcreativityinseekingandobtainingnew funding.
11 Capacity
Aprogram’scapacityreflectstheextenttowhichaprogram’sstaffhas,orisabletoobtain,thequalificationsnecessary toaccomplishthegoalsoftheprogram.Programsthatdemonstratesuccessinprogramcapacityhaveahighlyqualified staff,andprovideopportunitiesforstafftogainadditionaltrainingintopicsspecifictotheprogram.Forexample, providingorsupportingadditionaltraininginelderlybehavioralhealthissuescansubstantiallyimprovestaff qualificationstoimplementabehavioralhealthprogramforelderlyindividualsinruralareas.
Documentation of Program Information
Programdocumentationreferstotheinternalandexternalmaterialsprogramsusetodescribeand/oradvertisetheir services.Althoughmostorganizationshavepolicyandproceduremanualsfortheorganization,amajorityofprograms lackformal,internalprogramdocumentation,suchasaproceduremanualforthenominatedprogram,toallowfor replicatingtheprograminothercommunities.However,programsthatsuccessfullydocumentprograminformation generallyhaveexternalmarketingmaterials,andhavedevelopedeffectiveeffortstocommunicatewithkeyaudiences andpotentialfundersintheirrespectivecommunities.
Effectiveness
Aprogram’seffectivenessisdependentontheextenttheprogramtracksandusesdatafordecision-making.Few programscollectdataontheireffectiveness,knowhowtobegincollectingdata,andknowhowtousedatatoevaluate theprogram’seffectiveness.Lackofdataisalimitationincreatingapromisingpractice.
Community Engagement
Communityengagementreflectsthedegreetowhichaprograminvolvesmultiplestakeholdersfromthecommunityin itsdevelopment,execution,andexpansion.Programsincludedinthisdocumenttendtobestrongontheotherthemes whentheyhaveahighdegreeofcommunityengagement.
Developing a Promising Practice
Thesevencharacteristicsofsuccessfulprograms,asdefinedbythisstudy,provideafoundationbywhichprogramscan takeadditionalstepstomovetowardpromisingpractices.Amajorityofprogramsalreadyhavestrengthsinsomeof theseareas.Forexample,aprogrammayhaveengagedcommunitymemberssinceitsinception,andmayhaveexcellent staffcapacityfortheprogram.Programsshouldusethesesevencharacteristicstoassesswheretheirprogramsexcel andwheretheymighttargetfurtherdevelopment.Theinitialstudyofruralpracticeshighlightsthefactthatrural programsalreadyhavestrengthsinsomeoftheseareas,andthissectiondescribes,inbrief,someofthesteps organizationsmaytaketoimprovetheirprogramineachofthesesevenareas.Thetopicsofeffectivenessand
sustainabilitycameuprepeatedlyasareaswhereprogramswantedmoresupport. The“FocusedTechnicalAssistance” sectionisasupplementarysectionattheendofthisdocumentdevotedtothesetopics.
Relevance/Impact on rural behavioral health
Oneofthefirstthingsanorganizationneedstodotomoveitsruralpracticesforwardistodeterminewhattherelevance istotheruralcommunityitserves.Mostorganizationsinterviewedhavemissionstatementsfortheorganizationasa whole,butveryfewhaveformalmissionstatementsfortheprogramstheynominated.Identifyingtheimpactapractice willhaveonaruralcommunityisanessentialfirststepincreatingandtestingapromisingpractice.Therearethreesteps toimprovingapractice’srelevancetoandimpactonruralbehavioralhealth:Definingwhatisrural,definingthepractice, andidentifyingthepractice’sgoals.
12 What is rural?
Thedefinitionofwhatconstitutesaruralpracticeisnotstatic.“Rural”canandoftendoesmeandifferentthingsbased onwhereanindividualislocatedandwhoisusingtheterm.Forexample,ruralPennsylvaniaandruralAlaskaaretwo verydifferentplaceswithhighlyuniqueobstaclestoprovidingbehavioralhealthservicesforindividuals living in those communities.
ThemostcommondefinitionsofruralcomefromtheCensusBureauandtheOfficeofManagementandBudget(OMB), buttherearemorethanadozendifferentdefinitionsofrural.BoththeCensusBureauandtheOMBhaveslightly differentdefinitionsofrural,andthedefinitionusedmayhaveanimpactonthefundingofthepractice,orthefunding opportunitiesavailabletothepractice.Forexample,theCensusBureaudefinesanurbanizedareaorurbanclusterasa coreregionwithapopulationdensityofatleast1,000persquaremilesurroundingregionswithapopulationdensityof atleast500peoplepersquaremile. Ruralareasarethoseoutsideofurbanizedareasorurbanclusters.TheOMBdefines anurbanarea,orMetropolitanStatisticalArea,asacentralcountywithatleastoneurbanizedarea,andnearbycounties where25percentofindividualscommuteintooroutofthecorecounty.Ruralareas,ornon-Metropolitancounties,are thoseoutsideoftheMetropolitanStatisticalAreas.Thus,thedifferencebetweenthedefinitionsislargelyanoutcomeof thedifferentwaysofdefiningurbanareas. Foramorethoroughdiscussionofthistopic,refertotheRuralAssistance Center(RAC)onlineresourcepageregardingthedefinitionof“rural”andtherelatedtoolfordeterminingeligibility basedonthesedefinitionsof “rural”: Toviewtheruralgeographic eligibilityfortheOfficeofRuralHealthPolicyprograms,pleasereferto:
Defining “program”
Thedefinitionofaprogramisanessentialstartingpointtodevelopingapromisingpractice.Apromisingpracticeisnot simplyagroupofprovidersofferingservicesinanunderservedarea,norisitsimplybringingaservicetoaruralareanot alreadypresent(e.g.,addingasubstanceabusetreatmentprovidertotheorganization).Whendefiningaprogram,for evaluationorfunding,itisimportanttohaveaprograminmindthatismeanttoimproveaspecific,measureable outcomeonbehavioralhealthinaruralcommunityandhasyettobeidentifiedashavingapositiveeffectonthe community.Forexample,increasingthenumberofindividualsscreenedfordepression,increasingthenumberof graduatesfromaruraltrainingprogram,anddecreasingthewaittimeforaninitialvisitareallspecific,measurable outcomesthatmightresultfromimplementingnewprograms.Developingnewproceduresorprogramstomeeta specificneedinaruralareaandimprovingtheseaforementionedtypesofoutcomes,constitutesatestable,rural programthatcouldmovetowardapromisingpracticewithawell-designed study.
Identifying program goals
Indefiningaprogram,itisimportanttohavespecificgoalsinmindastowhattheprogramwillaccomplishinorderto meetaruralneed.Inruralareas,themostcommonlydiscussedbarrierstotreatmentinclude:accessibility,availability, andacceptability(Mohatt,2005).Ruralresidentsoftenhavetroubleaccessingcare–providersmaybehundredsof milesaway. Theavailabilityofbehavioralhealthprovidersinruralareasisoftentoolimitedtosupporturbanmodelsof servicedelivery(e.g.,evidence-basedpracticestestedinurbanareas).Inaddition,thestigmamanyAmericansattachto havingorseekinghelpforbehavioralhealthproblemsisparticularlydetrimentalinruralareaswherethereislittletono anonymityinseekingbehavioralhealthservices(Mohattetal.,2005).
Foraprogramtomeetaneedinaruralarea,itwillgenerallyaddressoneofthesethreebarrierstotreatment.A programthatcannotdefinehowitisspecificallyaddressingaruralneedinanovelandinnovativewaywillhavea difficulttimeconvincingfundingagenciestocontributetothedevelopmentandtestingoftheprogram.
13
Community Engagement
Onekeyfeatureofprogramssuccessfulinotherareasistheengagementofmanydifferentstakeholderswithinarural community.Programsthateffectivelyaddressruralbehavioralhealthneedsincorporateanumberofdifferentagencies, providers,andoftenconsumersintotheirprograms,whethertheyarejuststartingout,orhavebeenservingrural communitiesfordecades.Thisengagementofcommunitiesdemonstratesthecommunityunderstandsandissupportive oftheneedfortheprogram. Furthermore,activeengagementdemonstratestheprogramisabletosuccessfullymarket whatitdoestothecommunity,therebyincreasingawarenessoftheservicesprovidedandhowtoaccessthem.Thekey inbuildingmorecommunityengagementistorealizeothersinthecommunityarealsointerestedinbuilding
connections.
Ensuring the need will be met
Bringingarangeoforganizationsandindividualstodiscussionsaboutaprogram’sgoalsandimpactonthebehavioral healthofaruralcommunityensurestheprogramwilleffectivelymeetthecommunity’sneeds.Inaddition,havinga numberofdifferentorganizationsinvestedintheprogramwillhelpensureitslongevityandbringadditionalareasof expertisetothetable.Someoftheprogramsidentifiedhavemonthlyorquarterlymeetingsinwhichvarious
organizationsinthecommunitycometogethertodiscusstheirworkandtofindwaystobettermeettheuniqueneeds oftheircommunity.Often,thesecommunitydiscussionscanhelpaprogramidentifysourcesoffundingitwouldnot havefoundotherwise,makeconnectionstoevaluators,andhelpaprogramgainthetrustandrespectitneedstothrive intheruralcommunity.
Getting stakeholders involved
Manyprogramswithagooddealofcommunityengagementstartedbysimplypickingupthephoneand/ormakingan in-personvisittolocalproviders,consumers,policymakers,and/orresearchers.Mostprogramsweremetwith enthusiasmwhentheytookthisapproach,andsetupgroupsofcommunitymemberswhomeetregularlytodiscuss communitybehavioralhealthissues.Programssuccessfulinengagingthecommunityworkhardtomaintainthese connectionsdespitebusyschedules,staffturnover,andlackofresources.
Programssuccessfullyengagingtheircommunitiesalsoprovidecommunitieswithinformationaboutbehavioralhealth ingeneral,andtheirprogramspecifically.Byengagingcommunitiesinthisway,theseprogramsareabletogetvaluable knowledgeintothecommunityanddecreasestigmaintheprocessofeducatingcommunitiesaboutbehavioralhealth issues.Forexample,programsmayprovidetheirstaffmembersascommunityeducators,whogotoareaprimarycare providers(e.g.,doctorsandnurses),housingcommunities,churchgroups,schools,etc.toteachpeopleaboutbehavioral healthissues.Theprogramsmayofferfreebehavioralhealthscreeningsathospitalhealthfairsandsetupboothsat communityevents.Eachprogramseestheseconnectionsasvitaltoitssuccess,andasobservedduringthenomination andinterviewprocess,thesecommunityconnectionsareessentialtoaprogram’ssustainabilityandimpactinthe community.
Connecting with researchers and evaluators
Afewoftheidentifiedprogramshaveconnectionswithresearchersand/orevaluatorsatlocaluniversitiesorcollegesto assistwithdatacollectionandanalysis,and/orgrant-writing.Theseconnectionsarefewerthantheconnectionswith otherstakeholdersinthecommunities;someprogramsnotethecostofusingaresearcherand/orevaluatortohelpwith evaluationasamajorhurdleinconnectingwiththeseindividuals.
Someorganizationsalreadyhavetiestolocaluniversitiesandcolleges.Forthoseorganizationsthatdonothaveexisting tiestoahighereducationinstitution,aperusaloffacultywithinterestsinPsychology,Medicine,SocialWork,orsimilar areaonacampus’websitewillgenerallyturnupanemailaddressorphonenumberwithwhichtoinitiatecontact. Theseconnectionscanbelocal,withintheprogram’scommunity,oroutsideofStateorregionalboundaries.Aswith connectingtootherstakeholdersinthecommunity,theprogramneedstotakechargeofbuildingtheseconnections withresearcherstoensureithastheexpertiseneededtoevaluatetheimpactoftheprogramintheruralcommunity.
14
Althoughitcanbeahurdletofindadditionalfundingtosupportanevaluationwhenanorganizationisalready underfunded,thereareafewwaystoaddressthisissue.Thefirstistoincludearesearcherorevaluatorinagrant application.Someindividualsmaybewillingtohelpwiththegrantapplicationiftheywillbeapartoftheevaluation process.AsecondoptionistosendarequesttoaPsychology,Medicine,orSocialWorkdepartment,askingwhetheror notthereareanygraduatestudentswhomaybeinterestedinhelpingwithevaluatingandtestingtheprogramasapart oftheirgraduatework.Mostdepartmentshavee-maildistributionlists,anditisnotuncommonforrequestsforthis typeofhelptobedistributedtofacultyandstudentsviaemail.Athirdwaytoconnectwithresearchersisatnational conferences,suchastheannualmeetingoftheNationalRuralHealthAssociation(NRHA)ortheNationalAssociationfor RuralMentalHealth(NARMH).Giventheexistenceofcomputers,phones,andtheinternet,itisnotnecessaryfor programstoconnectsolelywithresearchersatnearbyuniversities.Someprogramsinterviewedarethousandsofmiles awayfromtheirevaluators.
Capacity and Documentation
Inordertobeapromisingpractice,aprogrammustdocumentitspracticessothatotherorganizationscancarryout thosepracticesinaccordancewiththeoriginalprogramandsothattheprogramisabletoadvertiseitsservicestothe community.Closelyrelatedtothisideaofdocumentationisthedefinitionofstaffroles,andhavingtheappropriatestaff capacitytocarryoutthespecifiedpractices.
Specifying details
Itisimportanttodocumenttheprogramorpracticeinasmuchdetailandasclearlyaspossible,sothatsomeoneelse couldreplicatetheprogrambasedondocumentationalone.Thefollowingquestionscanbehelpfulingettingstarted withthisdocumentationprocess:
• Whoisthetargetpopulation?
• Whatarethemaincomponentsoftheprogram?
• Wheredoestheprogramtakeplace?
• When does the program happen?
• Whyistheprogramneeded?
• Howdoestheprogramhappen(e.g.,whatarestaffmembers’roles)?
Marketing program services to communities
Onceaprogramappearstohavepositiveeffects,itisimportanttostartmarketingtheprogramtoboththelocal community,andthelargerruralbehavioralhealthcommunity.Localmarketingcanbedonebydevelopingabrochure, handout,orsomeothermarketingtool(e.g.,amagnetorpen)todelivertoorganizationsintheareathatmayhavean interestintheprogram.Thisbrochure/handoutshouldbesuccinct,summarizethemaindetailsoftheprogram,and providecontactinformation(suchasaphonenumber,e-mailaddress,orWebsite)forpeoplewhowishtoknowmore abouttheprogram.Oncedeveloped,thisbrochure/handoutcanbedistributedtorelevantorganizationsinthe
community.Forexample,acrisislinemaydistributeahandoutdescribingtheirservicestolocalhospitals,schools,law enforcementoffices,nursinghomes,faithcommunities,andbusinesses.
Inadditiontomarketingtheprogramtoorganizationswithinthecommunity,itisalsoimportanttomarkettheprogram tootherorganizationswithinterestsinruralbehavioralhealth.Sendingoute-mailnoticesoftheprogramto
organizationswithsimilarinterestscanspreadinformationabouttheprogram.TheRuralAssistanceCenter(RAC)Online providesanumberofpotentialavenuesformarketingprograms.Inadditiontospecialtyconferences(e.g.,theannual meetingoftheAmericanPsychologicalAssociation),conferencessuchastheannualmeetingsoftheNationalRural HealthAssociation(NRHA)andtheNationalAssociationforRuralMentalHealth(NARMH)offeradditionalvenuesfor marketingprogramsthatappeartobeeffective.Presentingaposterorsponsoringaboothatoneoftheseconferences canbeeffectivemethodstoensureothersinthelargerruralandbehavioralhealthpracticecommunityareawareofthe program.
15
Defining staff roles
Itisimportanttodemonstratehowtheprogramstaffisqualifiedtoperformtheroleforwhichtheyareassigned. Collectinginformationonthecharacteristicsofprogramstaff(e.g.,theireducationandexperience),howtheywere recruitedandhired,theirjobdescriptions,thetrainingtheyreceivedtoperformtheirroles,andhowperformanceis trackedviasupervisionprotocolswillhelpdefinestaffroles.Bydefiningstaffroles,aprogramdemonstratesthestaff capacitytocarryoutprogramactivitiesastheyweredesigned.
Importance for replication and evaluation
Inordertoensurereplicabilityofaprogram,itisessentialtohavearigorousdocumentationofthedetailsofthe program.Programdocumentationservestwopurposes:ithelpsanorganizationimplementaprogramconsistently,and ithelpsindividualsoutsidetheorganizationreplicatetheprogram.
Programdocumentationisalsonecessarywhenapplyingforfundingtosupporttheprogram’sgoals(oritsevaluation). Carefullydocumentingtheprogram’simplementationfacilitatestheprocessofwritinggrantsandcutsbackonthestaff timeneededtoapplyforfundingopportunities,sincemuchoftheinformationneededforthegrantwillbewritten beforegrant-writingbegins.
Effectiveness
Theeffectivenessofaprogramcanonlybemeasuredthroughdatacollection.Althoughitisagoodsignforaprogramto receivepositivefeedbackfromtheindividualsservedandforprogramstafftonoticeapositiveimpact,these
observationsarenotsufficientforthepurposesofdefiningapromisingpractice.
Developing a data collection/evaluation plan
“Anevaluationplanisalotlikeanarchitect'splansforahouse.Itisawrittendocumentthatspecifiestheevaluation designanddetailsthepracticesandprocedurestousetoconducttheevaluation.”(U.S.DepartmentofHealthand HumanServices,2010). Onewaytobegindevelopinganevaluationplanistoanswerthequestion:Whatisimportantto knowordemonstrateaboutthepractice?Then,afteransweringthisquestion,developobjectivesinmeasurableterms. Forexample,ratherthanaskingwhetheradepressionscreeningquestionnaireiseffective,focusthequestiontoa measurableoutcome,suchas,“Isthedepressionscreeningquestionnaireleadingtomorereferralstobehavioralhealth providers?”Itisimportanttofocusontheoriginalpurposeandrelatedgoalsoftheprogramwhendevelopingan evaluationplantodeterminewhetherornotaprogramiseffective.
Assessing effectiveness
Onceaprogramhasanevaluationplanandhascollecteddata,statisticalanalysesareneededtodeterminewhetheror nottheprogramiseffective.Asaresultoflimitedresourcesandmanpower,ruralprogramswhencomparedtourban programsdonotreadilyhaveexpertiseinstatisticsorthetimetolearnnewskillsinexperimentaldesignandanalysis.As notedintheprecedingsectiontitled,“CommunityEngagement”,connectingwithstudentsandresearchersatlocal universitiesisonewaytosupplementresearchandevaluationknowledgewithinaprogram.Often,astudentina graduateprogrammaybeinterestedinassistingwithanevaluation,inordertogainaccesstodataforathesisor dissertation.Makingconnectionswithlocalresearchersexpandstherangeofavailableknowledgeandmayprovidea meansforassessingaprogram’seffectivenessthatwouldnototherwisebeavailabletoaruralorganization.
16
Sustainability/Expansion
Aprogramneedstohavealong-termplanforfundingtobeasustainableprogram.Someinnovativeprogramsidentified lackedalong-termplanandhadterminatedorwereclosetoterminatingbythetimeoftheinterviews.Themost
successfulprogramswithregardtosustainabilityandexpansiongavethoughttohowtheprogramwouldbesustained beforeitwasimplemented;asustainabilityplanisapartoftheinitialdevelopmentoftheprogram.
Sustainabilityandexpansionsuccessiscloselyrelatedtoaprogram’ssuccessfulengagementofthecommunity.Often,a programissustainablebecauseofitstiestoothercommunityprograms.Insomecases,programsaresuccessfulin fundingtheprogramsolelyfromcontributionsfromothercommunityorganizations.
Developing a sustainability and expansion plan
Havingasustainabilityplaninplacefromthebeginningofaprogram’sdevelopmentisessentialforthelong-termability ofaprogramtoserveitscommunity.OftentheseplansincludeproceduressuchasgettingMedicareorMedicaid reimbursementfornon-traditionalservices;gettingservicesatreducedornocostfromotherorganizationsinthe community;creativelyapplyingforfundingfromlocal,State,andFederalsources;andcontractingwithother organizationsinthecommunitytoprovideessentialservices.
BasedonWICHE’sdiscussionswiththenominatedruralprograms,themostrobustsustainabilityplansformwhen communitystakeholdersareinvolvedinthecreationandexecutionofthesustainabilityplan.Communitystakeholders whoareinvolvedfromthebeginningareabletosharetheirownsustainabilityplans,helpidentifyandconnectprograms withsourcesoffunding,aswellasdonatetheirownresourcestotheprogram.Resourcesfromstakeholderscanappear intheformoffundingandalsointheformofdonatedtime,services,skills.Someprogramsaresustainablebecausethey areabletorelyfullyonsmallcontributionsfromothercommunitystakeholders,whoperformacriticalfunctionforthe program.
Applying for funding
Therearemyriadfundingopportunitiestosupportaprogram’sexpenses.Oftenthesefundingopportunitiescomefrom Federalentities,suchastheHealthResourcesandServicesAdministration,OfficeofRuralHealthPolicy,theCentersfor MedicaidandMedicare,andtheSubstanceAbuseandMentalHealthServicesAdministration.Additionally,funding comesfromStateoffices,suchastheDivisionofBehavioralHealth,ortheStateLegislatureaswellasfromhospitals, providers,privatehealthfoundationsorotherswithinaruralcommunity.
Manyprogramsinexistenceforyearsreportusingamixoffundingresourcestogettheprogramstarted.Overtime,as theprogram’sgoalssolidifiedanditsroleinthecommunitybecamemoreapparent,someprogramswereabletorelyon fundingfromoneormoresources,suchasalocalhospitalorMedicare/Medicaid.
Programsthatsubmittedapplicationsandarehighlysuccessfulinobtainingfundingarequitecreativeinthetypesof fundingagenciesandgrantopportunitiestheytarget.Someorganizationsreportwatchingoutforgrantopportunitiesin manydifferentareas(e.g.,housing-relatedgrants),andoftenjoinwithothercommunityagenciestoapplyforthese diverseopportunities(withgreatsuccess).
Focused Technical Assistance
TheFocusedTechnicalAssistancesectionprovidesmoredetailregardingthedevelopmentofthreekeycharacteristics, whichtheruralprogramsincludedinthestudystruggledwith,andarenecessarytocreateapromisingpractice:
relevancetorural,effectiveness,andsustainabilityandexpansioncapacity.Allprogramshadvaryingdegreesofstrength inareassuchascommunityengagement,orstaffcapacity.However,allprogramsspoketoaneedformoredataontheir effectiveness,andadditionalfundingtoexpandtheprogramandmakeitmoresustainable.Therefore,thissection focusesonthesetwoareas,aswellasrelevancetorural(specifically,definitionofa“program”),whichwasnotedbythe initialnominationprocessasanareaforwhichsomeprogramsneededassistance,duetoconfusionoverthetypeof programthatcanbecomeapromisingpractice.
17
Relevance to Rural Communities
Asnotedabove,aprogram’srelevancetoruralisapparentinitsabilitytoaffecttheavailability,acceptability,and/or accessibilityofbehavioralhealthcareinruralareas.Identifyingtherelevancetoruralcommunitiesisthefirstareato targetinordertodevelopaplanforeffectivenessresearch,aswellasforsecuringprogramfunding.Ifaprogramisnot clearonwhatitsgoalsare,itbecomesverydifficulttoknowwheretheprogramishavingapositiveeffectonrural communities.
Theprogramdefinitionoffersthebestopportunityforexplainingaprogram’srelevancetoruralcommunities.In describingaprogram,focusingonthepracticesitencompasses(asdefinedintheOverview)anditsutilityforrural behavioralhealtharekeyfeaturesoftheprogramdefinition.Theprogramdefinitionshouldbeclear,concise,and describethemaincomponentsoftheprogramanditsimpactonruralbehavioralhealth.AppendixCprovidesprogram descriptionsforavarietyofprogramtypesidentifiedinthisstudy. Thesedescriptionscoverallthecomponentsofa ruralbehavioralhealthprogramdefinitionandserveasanexampleonhowtodescribetheprogramclearlyand concisely.Theprogramdescriptionshouldidentifytheorganizationthathousestheprogram.Itshouldincludea statementaboutthetypeofservicesprovided,andthecommunityserved.Itshouldhighlightthekeypracticesinthe program,therelevancetoruralcommunities,andanydatarelatingtotheprogram’seffectiveness.Itshouldalsonote anyconnectionstootherstakeholdersthatenhancetheprogram’sreachorrelevancetothecommunity.
Additional resources
TheSubstanceAbuseandMentalHealthServicesAdministration’s(SAMHSA’s)NationalRegistryofEvidence-Based ProgramsandPractices(NREPP)isanadditionalresourcefromwhichtogatheradditionalexamplesanddescriptionsof effectivebehavioralhealthprogramdescriptions:
Effectiveness
Toachievethestatusofapromisingpractice,programsmustcollectdataandusethemtodeterminetheimpactofthe program.Thismeasurementofeffectivenesswilldemonstratetoproviderswhetheraprogramisontherighttrackin solvingaparticularbehavioralhealthprobleminaruralarea(i.e.,whetherornottheprogramismeetingitsgoals,as definedbyitsrelevancetorural),demonstratetoconsumerswhethertheyshouldseekoutthatprogramoveranother, demonstratetopolicymakerswhethertheyshouldpromotewidespreaduseoftheprogram,anddemonstrateto fundingagencieswhethertheprogramisworththeirinvestment.Forthesereasons,dataareessentialtoaprogram’s developmentandviability. Programsmustcollectdatatoreachthestatusofapromisingpractice,butperhapsmore importantly,dataareessentialforprogramstafftodeterminewhetherornotaprogramiseffectiveinmeetingitsgoals. TheAdministrationforChildrenandFamilies, OfficeofPlanning,Research,andEvaluationhaspublishedaguideto programevaluation,writtenforprogrammanagers,thatisanexceptionallyvaluableresourceforindividualsinneedof assistanceinprogramevaluation(2010). Thisguidewasdevelopedandmaintainedtoexplainthetopicofprogram evaluationandgiveguidanceonhowtoconductaprogramevaluationanddiscusstheresults.Thefollowingsections highlightsomeofthemainpointsthatarecomplementarytotheaforementionedguideforruralbehavioralhealth programs,andexpoundontheinformationbasedontheresultsofthestudyandtheneedsofruralbehavioralhealth programs. Thefullguideprovidessubstantiallymoredetailonthepropercollectionanduseofdata.
18 Data collection
Thereareanumberofdifferenttypesofdatatocollectduringtheevaluationprocessdependingontheprogramgoals, objectives,outcomes,andtheimpacttheprogramintendstohave.Anyinformationcollectedcanserveasdata,evena simplecountofthenumberofclients/patientswhoreceiveservicesonagivenday,month,oryear.Datamayinclude thenumberofparticipatingsitesinaprogram,thecostoftheprogram,changesinknowledgeand/orbehavior,ora program’sreturnoninvestment.Thefactthatanythingcanserveasdataisimportanttoemphasizebecausemost organizationsarecollectinginformationallthetime(whoisandhowmanypeopleareusingservices,howtheyare payingforservices,thelengthoftreatment,andsoon).Implementingadatacollectionplandoesnotnecessarilymean collectingadditionalinformation.Itmaysimplyinvolvetakingadvantageofdatathatalreadyexist. Assuch,itis importanttounderstandtheprogramgoalswhendeterminingthenecessarytypesofdataforevaluation.
TheProgramManager’sGuidetoEvaluationprovidesfurtheremphasisforthispoint(U.S.DepartmentofHealthand HumanServices,2010):“whenpreparingtocollectinformation(or“data”)onyourprogram,rememberthatany informationcollectedcanbeconsideredevaluationdata.Theimportantthingistoidentifywhatyouwouldliketo demonstrateaboutyourprogramorpractice.”
Oncethegoalsoftheprogramaredetermined,itispossibletodeterminehowtoevaluatethosegoals.Forexample,if thegoalofaprogramistoincreasetheuseofdepressionscreeningquestionnairesinelderlyprimarycarevisits,itis necessarytocollectinformationaboutthecharacteristicsoftheindividualsscreened,thenumbersofindividuals screened,howtheywererecruited,barriersencounteredintherecruitmentprocess,andfactorsthatfacilitated recruitment.Ifthegoalofaprogramistoincreasethenumberofprimarycareofficesthatoffertelementalhealth services,itisimportanttocollectinformationonthetypeofservicesofferedinprimarycareoffices,theeducationaland trainingopportunitiesthatexist,andthetypeoftechnicalequipmentavailableinthoseoffices.Ifthegoalofaprogram istoofferstatewidecrisishotlineservices,itisimportanttocollectinformationonthelocationsofindividualswhouse thehotline,marketingeffortsforthehotline,andthenumberofindividualswhoareusingtheservice.Oncethe program’sgoalsaredetermined,anevaluationplancanbedevelopedtoassesswhetherornottheprogramis effectivelymeetingitsgoals.
Types of Data
Programeffectivenesscanbemeasuredintermsofaprocessoranoutcome.Processobjectivesandmeasuresdescribe whataprogramisdoingandhowitwillbeaccomplished.Processmeasuresmayincludemanyaspectsoftheprogram suchasparticipantinformation,planningproducts,activities,services,andcommunityengagementactions.This informationcanhelpaprogrambeaccountablebymeetingspecificobjectives(e.g.,hostingacertainnumberof trainings,servingacertainnumberofpeople).Forexample,aprocessobjectivemaybe: ByJune2012,50percentof ruralprimarycareprovidersinthecommunitywillincludeadepressionscreeningquestionnaireinprimarycarevisits. Processobjectiveshelpmonitorandmanageaprogram,andtheycanhelpkeepaprogramontrackandalertstaffwhen thingsneedtobechangedregardingactivitiesandstrategies.
Whereasprocessevaluationisconductedduringtheprogramimplementationphase,outcomeevaluationisconducted whenanobjective,project,orprogramiscompleted.Outcomemeasuresareusedtypicallywithinoutcomeevaluation toidentifytheshort-termandlong-termresultsandeffectsofparticularprogramactivities(e.g.,practices).Although outcomemeasurescanbeusedtoevaluateprogressattainedduringtheimplementationphase,thegoalofoutcome evaluationistolookattheevidenceoftheprogram’seffectiveness.Statedanotherway,outcomemeasureshelpanswer thequestion,"Whatdifferencedidtheprogrammake?"Anexampleofanoutcomemeasuremaybe:ByJanuary2012, therewillbea25percentdecreaseintherateofsuicideattemptsinthecommunitythatimplementedtheroutine depressionscreenings.
19
Asanadditionalexampleofthedifferencebetweenprocessandoutcomemeasures,consideracommunitythathas developedandimplementedasuicidepreventionprogram.Processcomponentswouldbemeasuredbyansweringthe followingquestionsinrelationtotheinitialtimelineforthepreventionprogram:
1. Howcloselydoesimplementationmatchtheplan? 2. Whattypesofdeviationfromtheplanoccurred? 3. Whatledtothedeviations?
4. Whateffectdidthedeviationshaveontheinterventionandevaluation? 5. Howcanthedeviation(s)becorrected?Orshouldthedeviationbecorrected? 6. Whoprovidedservices(e.g.,whichorganization,andwhichstaff)?
7. Whatserviceswereprovided(e.g.,treatmentapproach,typeofdisordertreated,locationoftreatment,and lengthoftreatment)?
8. Towhomwereservicesprovided(i.e.,whoisthetargetpopulation)?
9. Inwhatcontextwereservicesprovided(e.g.,inahospitalsystem,orinacommunity)?
10. Atwhatcostwereservicesprovided(e.g.,thecostoffacilities,thecostofpersonnel,andthecosttothe consumer)?
Outcomecomponentswillfurtherclarifydatafromtheprocessevaluationusingthefollowingquestionsandmethods: 1. Whatwastheeffectsuicidepreventionactivitieshadonservicecapacity(e.g.,increasesordecreasesinusage),
andothersystemoutcomes?
2. Whatprogram/contextualfactorswereassociatedwithoutcomes? 3. Whatindividualfactorswereassociatedwithoutcomes?
4. Howlastingweretheeffects?
Additional resources
Belowareadditionalresourcesexplainingthedifferenttypesofdatacollection,aswellastoolsthatmayassistin programevaluation.
Resources
1. TheProgramManager’sGuidetoEvaluation:
2. CenterforDiseaseControlEvaluationWorkingGroup:
3. TheKelloggFoundationEvaluationHandbook:
20 Tools
1. TheOfficeofRuralHealthPolicyiscurrentlyworkingtodevelopageneralizableformulatohelpruralprograms measuretheimpacttheyaremakingintheircommunity.ThisEconomicImpactAnalysisFormulashouldbe finalizedandavailablebytheendof2010,andwillbeposted
2. TheNationalInstituteonDrugAbuseprovidesCostAnalysisToolsforsubstanceabuseprogramsthatmayalso beusefulindeterminingaprogram’scosteffectiveness:
a. TheDrugAbuseTreatmentCostAnalysisProgram(DATCAP)isacostdatacollectioninstrumentand interviewguidedesignedtobeusedinavarietyofhealth-relatedsettings.TheDATCAPhelpscollectand organizedetailedinformationonresourcesusedinservicedeliveryandtheirdollarcost.TheDATCAP instrumentisavailableatthefollowingWebsite:
b. TheServicesCostAnalysisProgram(SASCAP)estimatesthecostsofsubstanceabusetreatmentservicesby collectinginformationontheresourcesneededbytreatmentprogramstoprovidespecificservicesandhow theseresourceneedsmaydifferacrosstreatmentservices:
c. TheTreatmentCostAnalysisTool(TCUTCAT)isaself-administeredworkbookdesignedforFinancialOfficers andDirectorstoallocate,analyze,andestimatetreatmentcosts,aswellastoforecasteffectsoffuture changesinstaffing,clientflow,programdesign,andotherresources:
Sustainability and Expansion Capacity
Basedontheinformationgatheredduringthestudy,programscommonlyhaddifficultyidentifyingconsistentrevenue streamstosupporttheiractivities.Theissueoffundingisofcoursecentraltoaprogram’sabilitytosustainitselfandto expand.However,inrelationtocreatingapromisingpractice,grantfundingcanbeimportantforconductingthe researchneededtodetermineaprogram’seffectiveness.Thissectionthereforefocusesonhowtobeginlookingfor grantsandhowtoapplyforgrantsonceopportunitieshavebeenidentified.Thissectionwillhighlightsomeoftheplaces tosignupforgrantannouncementsandtouchonsomeofthestepsprogramscantaketodevelopeffectiveapplications.
Where to look for grants/signing up for announcements
Grantsaregenerallyclassifiedaccordingtotheentitythatisofferingthegrantopportunity.Withineachentity,there maybeanumberofdifferenttypesofgrants.However,thissectionwillfocusonthegeneralclassesofFederal,State, andprivateFoundationfundingopportunities.
Federal Grants
TheprimaryresourceforsearchingFederalgrantsisGrants.gov,aWebsitelistingallavailablegrantopportunitieswithin FederalagenciesIndividualscansubscribetoemailnotificationstoreceiveinformationabout varioustypesofgrantopportunitieswhentheybecomeavailableandsearchforcurrentopportunitiesbasedon
keywords,fundingorganizations,andtypesofgrantmechanisms. Furthermore,Grants.govofferstipsoneffective searchingofitsdatabaseandanumberofresourcestohelpapplicantsidentifygrantopportunitiesandtechnical assistanceoncompletingFederalgrantapplicationforms.TheRAConlineWebsiteisanotherresourcetosearchfor availableruralspecificFederalgrants:
State Grants
ToidentifyStategrants,checkingtheStateHealthandHumanServicesWebsitewillgenerallyprovideinformationon currentlyactivefundingopportunitiesingiventopicareas.State-basedhealthfoundationsalsovarybyState,but searchingtheWebsitesfortheseorganizationscanoftenleadtograntopportunities. Inparticularforrural communities,theStateOfficesofRuralHealth(SORH)arenotonlyamajorpartnerwiththeORHP,buttheSORHs provideruralcommunitieswithintheirrespectiveStatestechnicalassistanceaswellasinformationonavailableState fundingopportunitiesandotherresources
21 Private Foundations
Thereareanumberoforganizationsthatfundrural-orientedgrantapplications,suchastheRobertWoodJohnson FoundationandtheJohnD.andCatherineT.MacArthurFoundation.Inaddition,therearesomeonlineresourcesfor searchinggrantopportunitiesacrossprivateorganizations.However,theseresourcesmayrequireafeeforusingtheir services.
How to apply
Onceagrantopportunityhasbeenidentified,thefirststeptowardapplyingistounderstandthelogisticsofthe applicationbyreadingthroughtheapplicationinstructionsto:
1. Confirmtheprogramiseligibletoapply
2. Identifyanynecessaryapplicationrequirements(e.g.,havingaDataUniversalNumberingSystem(DUNS) numberregisteredfortheapplicantorganization),and
3. Determinewhetherthepurposeofthegrantfitswiththepurposeandgoalsofthebehavioralhealthprogramor project.
Next,itisimportanttogetintouchwiththepointofcontactorprojectofficerforthegrantprogramlistedonthe application.Thegrantprojectofficerisaninvaluableresourcethroughouttheapplicationprocess,fromadviceonhow toformattheapplicationtexttopointingoutplaceswheretheproposedideasmaybeproblematicfromthestandpoint ofareviewer.
Thethirdstepistotalkwithcommunitymembersandengageotherswhoseexpertiseand/orserviceswouldstrengthen theapplication(e.g.,someonewithprogramevaluationexpertiseatanearbyuniversityororganization). Onceeveryone whowillbeinvolvedinthegrantapplicationhasbeenidentifiedandrelevantquestionshavebeenansweredbythe grantprojectofficer,thefinalstepisforprogramstafftostartdraftingtheapplication.
Tips for effective applications
Themostimportantcharacteristicofeffectiveapplicationsisthattheyhavefollowedalltheinstructionscompletely.All relevantsectionsmustbeincluded,allkeypointsmustbeaddressed,andeventheformatandfontoftheapplication mustmeettheexactcriteriaspecifiedinthegrantapplication.Successfulgrantsarewell-written. Havingotherslook overtheapplicationandprovidefeedback,particularlyiftheyreceivedagrant,canalsohelpstrengthenanapplication. Often,itispossibletoreceivecopiesoffundedandnon-fundedapplications,eitherbygoingdirectlytotheindividual whoseapplicationwas(orwasnot)fundedorbyaskingthefundingorganizationtoprovideanexample. Successful grantsaretypicallyveryspecificanddetaileveryaspectofhowaprogramwillbecarriedoutandevaluated.Talkingwith thegrantprojectofficerduringthewritingoftheapplicationcanhelptheapplicantunderstandthebasicsofthegrant programandwhatisrequestedinthegrantprogramapplication.
Additional resources
Manyfundingagenciesoffergrant-writingworkshopstoassistindividualswiththegrantapplicationprocess.The fundingagencymaydirectorganizationstograntwritingworkshopsorinstructionalopportunities.TheRAConlineWeb sitealsoprovidessomeinformationonfundingopportunitiesforruralorganizations:
TheRACisrunbytheUniversityofNorthDakotaandprovidesresourcesandlistsforruralspecificFederal,State,and Foundationgrantopportunities. Inaddition,RACalsohasahotlinetohelpindividualsandorganizationssearchfor specificgrantsandruralinformationingeneral.
Conclusions
Manyruralprovidershaveidentifiedandimplementednovelandpotentiallyeffectivesolutionstotheaccessibility, availability,andacceptabilitybarrierstoruralbehavioralhealthcare,butthereisverylittledocumentationand
exchangeofinformationastowhathasworkedorfailedtoworkacrossruralareas.ServicessuchasSAMHSA’sNational RegistryofEvidence-BasedProgramsandPractices(NREPP)existtoshareinformationaboutEBPs,buttheseservicesare limitedtoEBPsthatarenottailoredtoortestedinruralpopulationsandmaynotgeneralizewelltoruralcommunities. Ruralbehavioralhealthproviderslackacentralresource,suchasNREPP,documentingpotentiallypromisingprograms
22
thathavebeenimplementedinruralareasacrossthecountry. Inaddition,thereisnocentralresourcedocumentingthe obstaclesassociatedwithguidingprogramsincreatinganevidence-baseandapromisingpractice.
Inthisstudy,WICHEsolicitednominationsfromruralprogramsaroundthecountrytoidentifyprogramshavinga positiveimpactonbehavioralhealthinruralcommunities. Theprogramsthatrespondedtothissolicitationwerequite diverse.Eachprogramhaditsownstrengths,andfromthosestrengthsasetofthemesemergedasindicatorsofa program’sabilityandreadinesstomovetowardapromisingpractice:relevancetorural,impactonrural,sustainability andexpansioncapability,capacity,documentationofprograminformation,effectiveness,andcommunityengagement. Eachofthesethemesishighlyrelatedtotheothers.Forexample,aprogramismuchmoresustainablewithengagement fromthecommunity;aprogramwithaclearconceptofitsrelevancetoaruralareaislikelytohaveahighimpactonthe ruralcommunityitisserving;and,themeasureofaprogram’seffectivenessdependsonhowitisexpectedtoimpacta ruralarea.
Theprimarygoalofthisdocumentistoofferastartingpointtoprogramshopingtobecomeapromisingpracticeby providingadditionalinformationoneachofthesethemesandtechnicalassistanceastothewaysinwhichprogramscan bestrengthenedintheseareas.Allprogramshadstrengthsinmultipleareas,butoneareainparticularmostprograms needtostrengthenisindocumentingtheireffectiveness.Programswerehighlyenthusiasticabouttheprospectof becomingapromisingpractice,butamajorityofprogramshadnodataontheireffectiveness.Datasupportingthe effectivenessofaprogramaretheprimarybasisonwhichaprogramcanbecharacterizedasapromisingpractice. However,strengtheningtheotherareaswillalsoaidprogramsincreatingapromisingpractice.Communityengagement, programcapacity,andaconcretenotionofaprogram’srelevancetoruralareasnotonlystrengthentheprogram’s abilitytocollecteffectivenessdata,butalsoaidinobtainingfundsforcontinuationorexpansion,andallowotherrural programstoadaptandusetheprogramwithintheirownrespectivecommunities.
AppendixChighlightsasecondarygoalofthisdocument:Topromoteinformationexchangebycatalogingnoveland potentiallypromisingbehavioralhealthpracticesinruralcommunitiesandservingasalinkbetweencommunitiesthat areimplementingsimilarprogramsand/orthoseproviderswhowouldliketosetupaprogramsimilartoonedescribed here.Manyruralbehavioralhealthprovidersareinterestedinimplementingnewprogramsintheircommunities. They maybeunsurehowtobeginplanningtheprogram,securingfunding,gettingcommunitybuy-in,andimplementingthe program.Bycategorizingprogramsaccordingtothetypeofservicetheyofferandthepopulationtowhichtheserviceis targeted,thisdocumentwillserveasanetworkingtoolforruralbehavioralhealthproviders.
Althoughitisintendedorganizationswillusethisdocumenttomovetheirprogramstowardpromisingpractices,onekey lessoninthisdocumentistheemphasisonbuildingconnectionstoothersinthecommunity,particularlytoresearchers whocanhelpwithevaluationefforts.Itisclearruralbehavioralhealthprogramswithextensivecommunitysupporttend tosucceedandgrowbecauseimprovingbehavioralhealthinruralareasisacommunity-wideeffort.Thisfocuson developingstrongertiestothecommunityaidsinstrengtheningeachofthethemesofrobustprograms,andappearsto increasethelikelihoodofbecomingapromisingpractice.
References
Bird,D.C.,Dempsey,P.,&Hartley,D.(2001).Addressing mental health workforce needs in underserved rural areas:
Accomplishments and challenges.Portland,ME:MaineRuralHealthResearchCenter,MuskieInstitute,Universityof
SouthernMaine.
Fortney,J.,Rost,K.,&Zhang,M.&(1999).Theimpactofgeographicaccessibilityontheintensityandqualityof depressiontreatment.Medical Care, 37, 884-893.
Institute of Medicine (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, D.C.: InstituteofMedicine,CommitteeonQualityofHealthCareinAmerica.
Kessler,R.C.,BerglundP.A.,GlantzM.D.,Koretz,D.S.,Merikangas,K.R.,Walters,E.E.,&Zaslavsky,A.M.(2004). Estimatingtheprevalenceandcorrelatesofseriousmentalillnessincommunityepidemiologicalsurveys.InR.W. Manderscheid&M.J. Henderson(Eds.),Mental Health, United States, 2002 (pp. 155-164).Rockville,MD:Centerfor MentalHealthServices,SubstanceAbuseandMentalHealthServicesAdministration. DHHS Pub. No. (SMA) 3938. Mohatt,D.F.,Adams,S.J.,Bradley,M.M.,&Morris,C.D.(2005).Mental Health and Rural America: 1994-2005 An
Overview and Annotated Bibliography.Rockville,MD:U.S.DepartmentofHealthandHumanServices,Health
ResourcesandServicesAdministration,OfficeofRuralHealthPolicy.
Murray,C.J.L.,&Lopez,A.D.(Eds.)(1996).The global burden of disease. A comprehensive assessment of mortality and
disability from diseases, injuries, and risk factors in 1990 and projected to 2020. Cambridge, MA: Harvard School of
Public Health.
NewFreedomCommissiononMentalHealth(2003).Subcommittee on Rural Issues: Background Paper. Rockville, MD: DHHS Pub. No. SMA-04-3890.
U.S.DepartmentofHealthandHumanServices.(1999).Mental Health: A Report of the Surgeon General. Rockville,MD: U.S.DepartmentofHealthandHumanServices,SubstanceAbuseandMentalHealthServicesAdministration,Center forMentalHealthServices,NationalInstituteofHealth,NationalInstituteofMentalHealth.
U.S.DepartmentofHealthandHumanServices.(2010).The Program Manager’s Guide to Evaluation Second Edition. Washington,D.C.:U.S.DepartmentofHealthandHumanServices,AdministrationforChildrenandFamilies,Office ofPlanning,Research,andEvaluation.
WangP.S.,LaneM.,OlfsonM.,Pincus,H.A.,Wells,K.B.,&Kessler,R.C.(2005).Twelve-monthuseofmentalhealth servicesintheUnitedStates:ResultsfromtheNationalComorbidityStudyReplication.Archives of General
Psychiatry, 62, 629-640.
WangP.S.,DemlerO.,OlfsonM.,Pincus,H.A.,Wells,K.B.,&Kessler,R.C.(2006).Changingprofilesofservicesectors usedformentalhealthcareintheUnitedStates.American Journal of Psychiatry, 163, 1187-1198.
WorldHealthOrganization.(2008).The Global Burden of Disease: 2004 Update. Switzerland.
Appendix A: Questions included in the Nomination Form
1. Contact Information1.1 Pleasefillinyourcontactinformationbelow. Name: Organization: Address: Address 2: City/Town: State: ZIP/Postal Code: Country: Email Address: Phone Number:
1.2. Whatisyourrelationshiptotheprogram?
1.3. Attheconclusionofthisproject,wewilldevelopadocumentsummarizingthepromisingpracticeswehave identifiedthroughthissurveyandfollow-upinterviews.Wewillalsodevelopatechnicalassistanceguidefor promisingbehavioralhealthpracticesinruralareastolearnmoreaboutthestrategies,methodologies,and
resourcesneededtobecomeevidence-basedpractices.Pleaseindicatewhichdocumentsyouwouldliketoreceive uponprojectcompletion,ifany.
2. Nominated Practice Characteristics
2.1. Inwhattypeoforganizationisthenominatedpracticebased(e.g.,CMHC,not-for-profit)?
2.2. Whattypeofservicesdoesyourorganizationprovide(e.g.,outpatient,residentialnominatedpractice,psychiatric rehabilitation,preventionprogram,educationalcampaign)?
2.3. Whattypeofpracticeareyounominating(e.g.,prevention,promotion,treatment,recruitment/retention, nontraditionalreimbursementsuchaspeerspecialistthroughMedicaid)?
2.4. Whatarethemainservices/componentsofthenominatedpracticedescribedabove? PromisingPracticesDocument
TechnicalAssistanceGuide Both
25
2.5. Whatistherationale/theoryfortheintervention?
2.6. Howintegratedisthenominatedpracticeintotheagency/community(i.e.,doesthenominatedpracticerelyon servicesprovidedbyanothernominatedpracticeoragency)?
2.7. Howdoesyournominatedpracticetakeculturaldifferencesintoaccount?
2.8. Whatindicatorsofsuccessisyournominatedpracticelookingfor(e.g.,decreasingmooddisorders,increasing retentionoforganizationstaff,etc.)?
2.9. Pleasetellussomethingaboutyournominatedpracticeyouthinkisimportantbutthatwehaven’taskedinthe questionsabove.
3. Population Characteristics
3.1. Describethepopulationservedbythenominatedpractice(i.e.,age,gender,race/ethnicity,etc.) 3.2. Onaverage,howmanypeopledoesyournominatedpracticereacheachyear?
3.3. Ifyournominatedpracticeistreatment-based,whatpercentageoftheclientsyouserveusepublicinsurance (Medicaid,Medicare,orSCHIP)?
3.4. What
71%-80% 81%-90% 91%-100% Not applicable
isyourbestestimateofthepercentageofpeopleyournominatedpracticereachesthatliveinaruralarea? 10% or less 11%-20% 21%-30% 31%-40% 41%-50% 51%-60% 61%-70% 10% or less 11%-20% 21%-30% 31%-40% 41%-50%
26
4. Staff Training
4.1. Howarestaffmemberstrainedtoimplementnewproceduresintheorganization? 4.2. Howarenewstaffmemberstrainedtoimplementthenominatedpractice?
4.3. Whatopportunitiesdostaffmembershaveforcontinuingeducationand/orprofessionaldevelopment(e.g., distancelearning,CMEs,etc.)?
4.4. Howisstaffevaluatedonproceduralcompliancetoorganizationalpolicy? 4.5. Howisstaffevaluatedonproceduralcompliancetothenominatedpractice? 4.6. Howisstaffperformancetrackedandrecorded?
4.7. Isstaffco-locatedwithotherprograms?
4.8. Whateducationalbackgroundsdokeystaffmembershave?
5. Documented Nominated Practice Information
5.1. Whatisthemissionofthenominatedpractice?Pleaseindicateifyourmissionisformalizedinamission statement.
5.2. Doyouhaveapolicy/proceduremanualthatguidesyourorganization?
Ifyes,pleasebrieflydescribetheformatandcontentofthemanualorotherrelateddocument.
5.3. Doyouhaveapolicy/proceduremanualthatguidesyourimplementationofthenominatedpractice?
No Yes
Ifyes,pleasebrieflydescribetheformatandcontentofthemanualorotherrelateddocument. 51%-60% 61%-70% 71%-80% 81%-90% 91%-100% Not applicable No Yes
27
5.4. Aretherebrochuresormaterialsthatprovideinformationabouttheservicesandnominatedpracticeofferedby your
No Yes organization?
Ifyes,pleasebrieflydescribetheformatandcontentofanybrochures,materials,orotherrelateddocuments. 5.5. Doyouhaveanygrantsyournominatedpracticehasappliedfororreceived?
Ifyes,pleasebrieflydescribethegrants(i.e.,fundingorganization,purpose,etc.).
5.6. Doyouhaveadocumentthatdescribesyournominatedpractice(treatment/intervention),eitherformallyor informally?
Ifyes,pleasebrieflydescribetheformatandcontentofthedocument.
6. Data Collection/Research Capacity
6.1. Doesyournominatedpracticecollectdatainanyform?
Ifyes,whatkindofdatadoyoucollect(e.g.,demographic,outcomes)?
6.2. Ifyournominatedpracticedoescollectdatainanyform,howarethedatacollected(e.g.,electronicrecords, treatmentnotes)?
6.3. Describetheevidenceyouhavethatyournominatedpracticehasapositiveoutcomeforthepopulationyouserve. 6.4. Howdoyoumakesureyournominatedpracticeisimplementedconsistently?
6.5. Istheresomeoneinthenominatedpracticewhoworksonevaluation,and/orhaveyoueverworkedwithan evaluator?
No Yes
Ifyes,pleasedescribetheevaluationexperience. No Yes No Yes No Yes
28
6.6. Isthereanevaluationplantomeasuretheeffectivenessofyournominatedpractice?
Ifyes,pleasesummarizetheevaluationplan.
7. Additional Information
7.1. Ifyouhaveanyadditionalinformationyouwouldlikeustoknowaboutyournominatedpractice,pleaseinclude thatinformationhere.
7.2. Pleaseletusknowifyouhaveanyadditionalcommentsand/orconcernsaboutthesurvey. No
Appendix B: Phone Interview Follow-Up Questions
1. Summarizetheprogram/practiceyouarenominating. 2. Describetherolesofthemainstaffinvolvedinthepractice. 3. Whatareyoudoingthatisunique?
4. Inwhatwaysisyourprogramhavinganimpactinruralareas?
5. Whatbenefitsisthepro
