Framework for Curriculum Evaluation

The No Más Hambre curriculum evaluation had several components that were conducted in various stages throughout the lifespan of the program. Methods included formative, process, and impact evaluation strategies. The formative evaluation began with previously discussed needs assessments that included the Senior Focus Groups, research team brainstorming and group-think sessions, and an extensive literature review. Curriculum program objectives and strategies were developed and refined.

Evaluation components were developed and curriculum/educational materials were pre-and pilot-tested for content, comprehension, cultural-sensitivity, pre-and linguistic

appropriateness. During the two pre-tests and final pilot-test, observations and field notes were completed by two team evaluators. In addition, each evaluator met with their assigned team of promotoras after each lesson to de-brief and identify strengths and weaknesses of the lesson format, content, supplementary materials and activities, and instruction/delivery. Research team de-briefings also aided in the identification of participant problems and barriers (e.g., literacy, language/translation, interpretation of information provided, and cultural-appropriateness of lesson materials). Curriculum lessons were modified as indicated by the project PI, curriculum developer, curriculum evaluators, and promotoras. Process and impact evaluation strategies, including

constructive and conclusive evaluation functions, are addressed in a subsequent feasibility and acceptability study of the curriculum.

During the initial stages of curriculum development, formative evaluation questions were developed to aid in the improvement of the final curriculum and all supplementary materials. The evaluation questions included:

 What do promotoras know about nutrition and nutrition-related topics?

 What nutrition concerns are most common among MH seniors?

 What types of food assistance programs, health services, and information are currently available in the priority communities?

 What are the overall social, economic, and political characteristics of the community?

 How do these characteristics affect individual and community health status?

 How will the curriculum lessons be created (and in what sequence) in order to provide education and skills that meet the needs and demands of MH seniors?

Two research team evaluators/observers were utilized throughout the process of conducting two curriculum pre-tests and one pilot implementation. The

evaluators/observers were divided so that each two-promotora team and their respective participants could be observed during implementation. The evaluators did not speak (unless spoken to) or play an active role in the lessons during which the curriculum was being delivered. During observation, the evaluators paid close attention to contextual aspects of the curriculum, participant engagement, and the delivery by promotoras.

Evaluator notes were guided by questions, such as:

 Were curriculum lessons delivered in a manner that was organic and conversational rather than directive?

 How did the senior participants respond to the lessons? Were they receptive or disengaged?

 Which elements of the lessons were senior participants most engaged (e.g., hands-on activities and use of “tool kits,” discussions accompanied by color handouts)?

 Did the promotoras deliver the lessons with fidelity? How often did promotoras diverge from the established lessons?

 To what extent were the learning objectives met for each lesson?

 Were the lessons developed and sequenced appropriately in order to provide the education/knowledge that is most pressing?

All evaluation notes arising from the observations were utilized for lesson modifications, training, and professional development for promotoras. Oral feedback from the promotoras and participants allowed the research team to revise and improve the curriculum when necessary in an iterative process. All protocols were approved by the Texas A&M University Institutional Review Board.

4.3. Results

During the pilot-test phase, 63 total lessons were delivered, during which 9 MH seniors participated (mean age = 66 years, range = 60-75 years). Of the 12 participants recruited, 9 participants (8 females, 1 male) agreed to the study and completed all seven lessons. Of the three participants who attrited, two declined to participate citing personal reasons, and one completed all but the final lesson and post-intervention interview because she migrated north when work became available. The lessons lasted an average

of 68.7 minutes each (range = 35-143 minutes). Two-thirds of the participants were born in Mexico (n = 6) and none of the participants received formal education beyond the sixth grade. Monthly household incomes reported by the participants ranged from less than $500 to $1000 - $1200. Of the nine participants, only two reported they did not receive SNAP benefits. However, these two participants reported they sought fresh and processed food items from a local church or food pantry. All but two participants reported they owned an automobile; the two without reported they received transportation from family members. Three of the participants self-reported being overweight or obese and two-thirds of all participants reported their general health as

“poor” or “bad.”

4.3.1. Observations & Reflexive Journals

A recurrent theme that surfaced throughout the pre-and pilot-testing was the importance of shared learning. MH seniors preferred group learning though individual instruction was completed to best assess curriculum feasibility. Despite informing each participant that the education was limited to them alone (for feasibility testing purposes), many of the participants were joined by spouses, immediate and extended family

members, and neighbors. Desire for communal learning and information sharing was a prominent subtheme in the Senior Focus Groups and this assertion was elucidated during the pre-and pilot-testing of the lessons.

Most of the participants had a difficult time remembering to write in their reflexive journals from one lesson to the next and often had to request assistance from their promotora educators or from family members and friends. Much of the participant

reflexive journals contained nutrition-related goals and ideas for adopting and/or maintaining coping strategies and maximization of resources. Only one participant was avid about writing in her journal on a daily basis, and she offered a wealth of insight into her thoughts of each lesson, personal goals, and self-reflections on improved food security and health. The promotora educators were also encouraged to write in their reflexive journals, as well as share some of their journal passages during team meetings and lesson de-briefings. A few of the promotoras expressed apprehension regarding their own knowledge and abilities to deliver the lessons, though responses were mostly

positive:

“I feel much better and I thank God…I feel good about this week.”

“I am motivated to study [curriculum lessons] more now than before.”

“I am glad we are in a project where we can teach the people.”

“I loved the practice and confidence. I have confidence in the team and in teaching the classes [lessons].”

“I like that we give many important tips for change.”

These shared experiences provided invaluable opportunities for the improvement of the curriculum in addition to satisfying a critical element in the formative evaluation process.

Observations by the evaluator/observers were also important elements of data in the reflection process. Important of note, participants were mostly excited to be

connected with information and given the skills to make the most of limited resources.

The importance of frequent social support and social interaction was another critical

observation point made by the observers. Though we limited each individual lesson delivered to just one participant, most pre- and pilot-test participants were eager to invite others (e.g., friends, neighbors, family) to listen in and receive the education. Illiteracy was an excluding factor; however, one participant later admitted she was embarrassed initially to admit she was unable to read or write. Therefore, the participant was joined by her young grand-daughter (age 11) who happily assisted her with the lesson

materials, journal writing, and reading of supplementary handouts. Examples such as this gave observers the impression that group learning, whether in a community or private residence setting, is likely the ideal method of delivery within this population.

4.4. Discussion

This work adds to a growing body of literature by describing innovative concepts of applying post-modern pedagogical theories of curriculum development and novel methods of applying cultural tenets to programs designed to improve nutrition knowledge, skills, and behaviors, in an effort to reduce hunger risks and encourage resource maximization. Special consideration was given in the curriculum content to the financial hardships and access challenges of MH seniors within the target geographical areas. Promotoras and participants were encouraged to share their new knowledge, skills, and resources within their social networks (e.g., family members, friends, neighbors, church groups, senior center peers) because behavior change is achieved when learners hear consistent messages from different people in different contexts.¹³³

Complete standardization of curriculum is an archaic pedagogical practice.

Though a few of the promotoras diverged from the curriculum lesson plans, each

promotora had her own unique method of delivery, and we encouraged this display of autonomy. However, team de-briefings were put into place—not only as an element of formative evaluation—to identify areas of concern if the delivery was straying too far from the core lesson concepts. Content validity and reliability were not measured in this study, as we only aimed to examine the feasibility and acceptability, not curriculum effectiveness and outcomes.

This study has some limitations. The sample size for the pilot intervention was small and was a convenience sample. Additionally, one of the evaluator/observers developed the curriculum and supplemental materials; this could have introduced bias. A limitation of the observations is that the participants may have interacted with

promotoras in an atypical fashion because they knew they were being observed.

Furthermore, the promotoras knew they were being observed as well, which may have impacted their method of delivery and adherence to the curriculum. Having only one evaluator/observer per promotora team limited recorded data to the perceptions of only one observer. However, curriculum feasibility and reliability assessments included in-depth interviews in addition to the evaluators’ observations which do increase the validity of the data. Finally, because this was a pilot intervention focused on

development, formative evaluation, and implementation of No Más Hambre, the units of analyses were the participants (i.e., non-verbal responses, behaviors) and promotoras and their perceptions and suggestions regarding the curriculum. The next stages of evaluation involved data collection of participant responses (i.e., in-depth interviews)

and promotora insights (i.e., focus group) regarding the curriculum, and assessment of theoretical constructs.

Not unlike the goals of numerous organizations whose missions are to address and combat hunger and food insecurity, we seek to reduce the incidence and prevalence of hunger and food insecurity through pragmatic and sustainable solutions. Development of learner-based educational programs that include considerations of socio-ecological context, cultural and linguistic factors, and address needs and root causes may be successful in achieving community food security in underserved and marginalized populations. Furthermore, successful grass-roots and community-based efforts may lead to policy initiation and reformations that achieve a wide-spread, population-level reach and address and combat hunger and food insecurity across all sub-populations and communities at risk.

4.4.1. Lessons Learned

The employment of promotoras in the curriculum development process, and later as community educators, elucidated information and cultural insight for how the final curriculum should be structured and implemented. We encouraged promotoras to be flexible and energetic versus robotic in guiding their lessons, though we urged them to drive home key objectives highlighted in each lesson. Variations in lesson delivery (e.g., time, depth, activities) were unique to each promotora educator. The project

evaluator/observers noticed that, in a few of the lessons, some promotoras neglected to follow a complete lesson plan, either by omitting parts or by adding extraneous

information to the discussion. Researchers who implemented a similar program within a

low-income Hispanic population reported similar findings for nutrition education programs guided by peers.¹⁵⁹ Though the fidelity of the curriculum delivery may be impeded, within this particular cultural setting, flexibility of delivery and contextual focus may be essential to a promotora educator. While delivering the lessons,

promotoras enjoyed offering personal examples of their own struggles, successes, and behavior modifications.

Recruitment and selection of promotoras is an important element to the success of this curriculum because lesson delivery, teaching styles, enthusiasm, and general interest in the lesson topics can impact how a participant learns and absorbs the knowledge and skills. It is also important for the promotora educators to be

conversational and self-reflective when delivering lessons, as well as to develop and maintain their own knowledge of the nutrition-related topics addressed in lesson plans.

Training and preparation of promotoras and program staff must be conducted in a manner that allows for adequate peer-to-peer lesson “rehearsals,” the opportunity to ask questions, and have refresher courses or continuing education opportunities as the intervention progresses. Both promotoras and participants requested more training and educational opportunities in the future. A few participants noted they would like to teach their peers in the community about the information they learned and wanted the

opportunity to do so.

5. PAPER 3: FEASIBILITY AND ACCEPTABILITY OF A SENIOR HUNGER CURRICULUM

5.1. Introduction

Older adults living in Texas-Mexico border areas known as colonias are primarily of Mexican heritage and are one of the most disadvantaged, hard-to-reach groups in the United States.²⁵ Colonias are often substandard residential areas with variable housing conditions, inadequate roads and basic infrastructure, limited access to adequate sewer systems and safe, potable water sources.²⁶ In these areas, Mexican-heritage (MH, i.e., individuals who trace their origin or descent to Mexico) seniors experience numerous nutrition-related health disparities and encounter many barriers to adopting and maintaining healthy behaviors. For the most part, colonias residents are monolingual Spanish and have significant financial constraints, limited education, and few health-supporting resources such as access to affordable healthful foods, healthcare, health information, and efficacious primary prevention programs.25,26,30-32 MH seniors in the colonias experience the critical problems of food insecurity and hunger, overweight or obesity, and chronic disease disproportionately to their ethnic counterparts.^31,138

Culturally-grounded nutrition education programs have gained popularity over the last decade as an effective and feasible method of deploying behavior modification strategies among ethnic minority populations.118,149,160-163 These programs not only highlight the importance of nutritional health, but they increase knowledge and skills-based behaviors through experiential learning. Though numerous nutrition education interventions are currently being implemented in community settings nationwide, none

of these programs have specifically identified seniors (ages 60 and older) from low-income, MH populations who experience frequent hunger and/or food insecurity as a priority population, per extant literature. Good nutritional health is a critical element to healthy aging, yet many older adults experience chronic malnutrition and increased risk of hunger. In fact, in 2011 researcher Craig Gundersen found that, “8.35 percent of Americans over age 60 faced the threat of hunger,” which translates to approximately 4.8 million people.¹⁶⁴ Though numerous hunger and food insecurity interventions focus on combating the problem at the institutional and policy sectors, community grass-roots efforts that focus on individuals and households have the opportunity to have lasting intrinsic impacts. There are neither published studies nor existing data, to our

knowledge, on individualized, home-based intervention strategies to reduce the risk for hunger.

Though there are numerous peer-reviewed feasibility studies, very little published literature offers recommendations to guide the design and evaluation of feasibility research. Feasibility studies can be defined as studies which are designed to establish a foundation for a designed intervention study.^165,166 Though not unusual in community health interventions, feasibility studies are more commonly found in drug efficacy trials and are typically designed as efficacy and/or randomized controlled trials (RCTs).¹⁶⁵ According to Bowen and colleagues (2009), “Feasibility studies encompass any sort of study that can help investigators prepare for full-scale research leading to an intervention…and are relied on to produce a set of findings that help determine whether an intervention is relevant, sustainable, and should be recommended for efficacy

testing.”¹⁶⁶ Feasibility studies are set into motion in an effort to answer the question,

“Can this study be done?” Of the previously published studies that examine the

feasibility of a curriculum, few have employed an intervention strategy similar to that of the No Más Hambre [No More Hunger] curriculum. Additionally, our hard-to-reach population has been shown empirically to necessitate consideration and innovative intervention strategies.

The literature on home-based, nutrition interventions tailored to families with infants and children is sizeable; however, limited evidence on home-based interventions for seniors exists.¹⁶⁷ Following extensive formative and process evaluation in the early stages of curriculum development, data from participants confirmed the preference for a home-based program that would accommodate their needs. Home-based interventions are essential for seniors in this population as many of them have limited safe and reliable transportation options that would allow them the freedom to attend community-based programs.^25,42 Other senior adults have functional limitations that would inhibit their opportunity to attend events outside of their homes, warranting the need for home-based programs.¹⁶⁸ Furthermore, home-based interventions for seniors may reduce future costs of healthcare.¹⁶⁷ Unfortunately, of the few home-based studies which include senior adults, the majority are randomized controlled trials which limit the argument for practical applicability. According to Sanson-Fischer and colleagues, “Commitment to randomized controlled trial may limit innovation in population-based health

interventions…other research designs may be more practical for a wider variety of interventions.”¹⁶⁹ Therefore, we designed a home-based curriculum, tailored to the home

environment and to meet the needs of our priority population, which provides nutrition knowledge and skills in a practical and real-world setting.

The purpose of this paper is to provide the results of the feasibility and

acceptability testing of the No Más Hambre curriculum among MH seniors (ages 60 and older) who reside in Texas-Mexico border colonias. With the overarching goals of establishing feasibility and acceptability, we hypothesized that the development and implementation of No Más Hambre, a culturally- and linguistically-focused, home-based nutrition curriculum, will: 1) be feasible, 2) be acceptable, and 3) aid in reducing the risk and presence of hunger among MH seniors in Texas-Mexico border colonias.

5.2. Methods

The work of Bowen and colleagues¹⁶⁶ inspired the guidelines used to assess the feasibility of this study. Our study was not powered to measure outcomes, rather we sought to address the feasibility and acceptability of the curriculum, specifically with regard to the following questions:

 Will limited participant recruitment allow for a meaningful assessment of the feasibility and acceptability of the curriculum?

 What collaborators and resources are necessary?

 Are there other data collection and analysis techniques specific to answering our research questions that must be identified and utilized?

 Should this intervention be subject to greater efficacy and effectiveness studies?

5.2.1 Participant Recruitment

Study recruitment lasted 12 non-consecutive weeks (between January 2014 and May 2014). Eligible participants were low-income MH seniors residing in functionally-rural colonias in four geographical areas (Alton, San Carlos, Progreso, and Penitas) in Hidalgo County in the Lower Rio Grande Valley of Texas. Due to the nature of this study, we recruited purposive samples of MH seniors using convenience sampling strategies (i.e. door-to-door) which increased the possibility of recruiting individuals within our priority population who met the inclusion criteria. Inclusion criteria for this

Study recruitment lasted 12 non-consecutive weeks (between January 2014 and May 2014). Eligible participants were low-income MH seniors residing in functionally-rural colonias in four geographical areas (Alton, San Carlos, Progreso, and Penitas) in Hidalgo County in the Lower Rio Grande Valley of Texas. Due to the nature of this study, we recruited purposive samples of MH seniors using convenience sampling strategies (i.e. door-to-door) which increased the possibility of recruiting individuals within our priority population who met the inclusion criteria. Inclusion criteria for this