Needs assessment and evaluation are distinct but related activities. A needs assessment should provide information not only about the types and levels of service available to PLWH, but also about service access, barriers, and appropriateness; client satisfaction with services is also a typical component of needs assessment. These factors are also a part of program evaluation.
Often planning bodies, grantees, and providers do not feel comfortable including evaluative information about specific providers in their needs assessment reports. For example, they may not want to publicly present specific barriers to services at a particular clinic or health center. However, they usually do need to include such information for the overall service system and for particular categories of services, populations, and geographic locations. For example, a profile of provider capacity and capability often summarizes common barriers to care for particular population groups such as gay men of color, women, or Latinos with limited-English proficiency. Such information is frequently collected from both PLWH and providers, through focus groups and surveys. Already available information from client satisfaction surveys can also contribute to the provider profile.
In planning your provider profile, consider what types and levels of data on service barriers and access and client satisfaction you wish to collect, aggregate, and report.
• geographic location;
• proximity to public transportation; • service hours;
• eligibility requirements; and
• languages spoken by provider personnel.
◗ Appropriateness: to what extent the available services are appropriate for PLWH and for specific populations (e.g., specific racial/ethnic minorities, women, women with children, gay and bisexual men or color) in terms of:
• HIV experience,
• staff composition and cultural competence; • service approach or model; and
• sensitivity to needs of PLWH. (This information is particularly important for providers whose clients are not limited to PLWH and their families.)
Also of interest is information about actual services provided to PLWH during the most recent completed program year, and key characteristics of those clients. Such information is available for CARE Act-funded providers through AAR or URS reports, but not for other providers.
A sample provider survey—including questions that address availability, accessibility, and appropriateness—is provided in the appendix.
Suggested Information Sources
A comprehensive and accurate provider profile includes not only information obtained directly f rom providers describing their own services but also client and other provider personnel perc e p t i o n s regarding service accessibility and appropriateness. This information may involve particular providers but is more likely to address categories of services (e.g., substance abuse treatment, primary medical care), populations (e.g., women with children), and/or locations (e.g., services available in a particular jurisdiction or community).
Thus a comprehensive provider profile typically includes four types of information from three sources:
◗ provider self-reports, with providers asked to describe their own service availability and provide both factual information and their own perspectives on the accessibility and appropriateness of their services for PLWH;
◗ reports on the broader service network by provider personnel, who describe service access, barriers, and appropriateness of services provided by other providers in the network based on their own referrals and their clients’ experiences;
◗ consumer information from clients who have received particular categories or types of services (e . g ., primary medical care, case management), who re p o rt on both satisfaction with and barriers to services through PLWH surveys, focus groups, and/or client satisfaction surveys; and
◗ secondary source data such as existing studies or reports or AAR or URS data summa- rizing (by provider and by service category) the services provided to PLWH and the characteristics of the clients receiving these services.
Each of these information sources are discussed in detail below. Information from Providers
Providers can be asked not only for information about their own agency’s services but also about referral experiences with other categories of service. For example, case managers or other personnel responsible for referrals might be asked about the extent to which they are able to obtain slots in detoxification centers, substance abuse treatment programs, housing assistance programs, or other types of services for their clients, and the most typical barriers encountered.
Information from Consumers
PLWH and other consumers (such as parent of children with HIV/AIDS) can be asked about their specific experiences in seeking and receiving particular categories of services. Information addressing service barriers and service appropriateness is especially valuable when it is collected, categorized, and analyzed for particular special needs populations.
P LWH perspectives on provider capacity and capability can be ve ry illuminating. For example, primary care providers may report that they have the capability to serve Latinos, but Spanish- speaking clients may report going to appointments to discover that the health care professionals do not speak Spanish and no interpreter is available—or being unable to communicate with the staff who answer the telephone so they never get to a Spanish-speaking health care professional or paraprofessional.
Secondary Source Data
Use of secondary source (existing) data can enhance a provider profile and reduce the cost of compiling it. Before seeking new information from or about providers, determine what infor- mation is already available. For example:
◗ AAR data are reported by each funded provider, and are readily aggregated by category of service, geographic location, and client characteristics.
◗ Funded providers may report other information about service capacity and capability in their quarterly narrative reports to the grantee.
◗ Client satisfaction surveys and other evaluation studies may provide useful information. ◗ CARE Act grantees, applicants, and providers often have substantial data bases and/or
research reports that address issues related to service availability, accessibility, and appro- priateness.
Methods of Information Gathering
Choosing appropriate methods for gathering provider profile information requires consideration of the following:
◗ whether resource inventory data are being collected at the same time; ◗ the number and range of providers to be included;
◗ the other sources of provider information to be tapped (e.g., case managers or clients); ◗ the types and amount of information to be obtained from each provider or other
information source;
◗ the costs and benefits of specific data collection methods for reaching particular populations; and
◗ available time and resources.
Client perspectives on service accessibility and appropriateness can be obtained as part of the consumer surveys or focus groups conducted as a part of the needs assessment. (See Chapter 4 of this section for information on planning and conducting these components of the needs assessment.)
For providers, consider the following approaches:
◗ Provider surveys, administered using one or a combination of the following methods: • mail or fax surveys designed for self-administration by provider personnel;
• telephone surveys with provider representatives; and/or • personal interviews with provider representatives.
◗ Provider focus groups, which are particularly useful for obtaining case manager or other provider perspectives on categories of services or service providers (rather than factual information or assessments of their own services).
◗ Key informant interviews with a carefully selected group of diverse and particularly knowledgeable provider personnel, which together can provide expert perspectives on the availability, accessibility, and appropriateness of particular categories of services, services for particular populations, and services in particular geographic areas.
Focus groups and key informant interviews will provide valuable qualitative (non-statistical) information about the network of providers and services and about service access and barriers. Provider surveys will probably be needed for specific information about aggregate service capacity and capability, and for information on specific providers or groups of providers (e.g., primary care providers, nutrition services providers).
The trade-offs involved in using different types of survey approaches are similar to those for PLWH surveys, as described in Chapter 4. Personal face-to-face or telephone interviews can provide high response rates and tend to generate the most complete and accurate data but also take considerable time and effort. Fax or mail questionnaires that are self-administered obtain l ower response rates but are far less expensive to implement. Providers receiving CARE Act funds a re somewhat more likely to respond to such surveys and to provide complete information than those that are not receiving CARE Act funds and/or not specifically targeting PLWH. Larger organizations with more staff may be more likely to respond than smaller providers with few staff.
To maximize the provider response rate, ensure responses from a mix of providers, obtain good data quality and completeness, and still control costs, consider using a combination of methods. For example:
1. SELECT A SAMPLE OF PROVIDERS TO RECEIVE PERSONAL INTERVIEWS, face-to-face or by telephone. This might include a “purposive” (deliberately selected) mix of providers chosen based on such factors as size, services provided, location, and/or targeted population groups. 2. SEND FAX OR MAIL SURVEYS TO THE OTHER PROVIDERS ON YOUR LIST.Give them the
option of contacting you to request a telephone or personal interview. Request completed surveys by a specified date not more than three weeks after they receive the surveys.
3. IF THE RESPONSE RATE IS UNACCEPTABLY LOW BASED ON YOUR NEEDS ASSESSMENT PLAN, CONTACT NON-RESPONDING PROVIDERS by sending a reminder card, faxing a reminder, or making a telephone call. Offer to do personal interviews if necessary to obtain a response. 4. SUPPLEMENT MAIL RESPONSES BY TELEPHONE until you have an acceptable response rate
and mix of responding providers.