ALE Spotlight: Evaluating the Preventive Food Pantry at Boston Medical Center
By Eva Greenthal, MS/MPH ‘18
When I started the mock program evaluation project for my Methods of Public Health Inquiry class in Fall 2017, I had no idea this assignment would turn into my Applied Learning Experience (ALE). For the project, each student chose a public health program in the Boston area and developed an evaluation plan for that program. I chose the Preventive Food Pantry at Boston Medical Center (BMC). I did not know much about this program, except that it was a hospital-based, medically-tailored food pantry and one of the first food-as-medicine interventions of its kind. Excited to learn more, I started by looking for past evaluation reports on the hospital’s website, but there were none to be found. This led me to reach out to the pantry manager who informed me that aside from periodically administering a three-question patient satisfaction survey, in its 17 years of operation the Preventive Food Pantry had never undergone a comprehensive program evaluation! I briefly considering changing topics for my class assignment but decided instead to put in some extra work and create an evaluation plan from scratch.
Towards the end of the semester, I met with the pantry manager, showed him my mock evaluation plan, and asked whether an evaluation of this kind would be helpful. After a few more months of discussions, volunteering at the pantry, obtaining approval from the hospital’s chief medical officers, and securing funding from the Morton A. Madoff Fellowship, it was decided—I would spend the summer implementing my evaluation plan and completing my ALE at the BMC Preventive Food Pantry.
The purpose of the pilot evaluation was to identify strengths and weaknesses in program functioning and common aspects of patients’ experiences at the pantry in order to make recommendations for quality improvement and lay the groundwork for an outcome evaluation. The pilot included two key parts: an online survey on food insecurity and pantry referral for medical providers, and in-depth interviews with patients who use the pantry.
I developed the provider survey with input from stakeholders at BMC, Greater Boston Food Bank, and Children’s HealthWatch. It included 14 multiple choice and short response questions asking about demographics, experience and perceptions about discussing food insecurity with patients, and experience and perceptions about the Preventive Food Pantry. After recruiting participants by making announcements at department meetings and sending follow-up emails, I was pleased that a total of 89 providers responded to the survey—a response rate of 34 percent! While the sample was not big enough for rigorous statistical analysis, I gained valuable experience analyzing raw survey data, creating descriptive statistics, and making critical decisions about how to present my findings.
Meanwhile, I conducted 30 interviews with patients at the pantry. Thanks to my funding from the Madoff Fellowship, I was able to offer patients $10 gift cards to Stop and Shop as an incentive for participation, which made recruitment easy. I asked questions regarding food assistance, food security, diet, nutrition education, and perceptions of the pantry. Recognizing that food insecurity is typically a symptom of a larger problem (i.e. poverty), I concluded each interview by asking, “What would have to happen for you not to need the food pantry?” After taking an online tutorial to learn how to use NVivo qualitative data analysis software and attending a timely qualitative data analysis seminar through the Boston Nutrition Obesity Research Center (BNORC), I was ready to analyze the interview data. The process of conducting, transcribing, coding, and presenting the qualitative data from these interviews was perhaps the most valuable part of my ALE . Speaking with patients gave me a much deeper understanding of the lived experience of food insecurity and hunger, and the entire process helped me build a strong skill base in qualitative data collection and analysis.
Designing my own ALE was a challenging endeavor, and there were times I felt completely overwhelmed along the way. But with support from colleagues and advisors at Tufts and BMC, I met the challenge head-on. I built valuable relationships and developed project management skills. I am continuing to work with researchers and doctors at BMC to prepare a manuscript for submission to a peer-review journal, and I am involved with the follow-up outcome evaluation study led by an internal medicine physician at BMC. Seeing this project through from start to finish has been incredibly rewarding, and I am so glad to have completed my ALE evaluating the BMC Preventive Food Pantry.