Evaluating Success Across the US: Impact Evaluation Unit ensures federal dollars are effective

Tucked into the 176-year-old Lumpkin House on UGA’s campus, the Impact Evaluation Unit (IEU) may seem as historical as the building that houses it. The IEU was born much later, but the work done inside shows how publicly funded projects are influencing the future.

The Impact Evaluation Unit was founded in 2018 to provide faculty with evaluation services. Dr. Kay Kelsey, IEU Director and ALEC Professor, says the unit began to serve faculty’s need to evaluate both “outputs and impacts of their projects.”

 

Why Publicly Funded Projects Require Evaluation

All federal agencies who provide grants require impact studies of the projects they’re funding. These studies provide transparency and display how the federal funds are put to use.

“Taxpayers demand accountability for publicly funded projects, and we work to gather credible evidence to help determine ‘what happened’ as a result of the program,” says Kelsey.

According to Kelsey, the evaluation process involves partnering with faculty to design both an evaluation plan and logic model. Then, IEU works with them to collect data about project outcomes and impacts. Finally, they report the findings to U.S. government agencies.

The data collected and reported measures the “broader impact” by seeking answers to the question, “What happened in terms of changing people’s knowledge, attitudes and behaviors as a result of the program?” according to Kelsey.

The broader impact goes beyond what people learn and tries to determine if their lives were changed for the better by the publicly funded programs.

“It’s one thing to learn about new innovations by sitting in a classroom, and another thing entirely to actually make a change in our behavior,” says Kelsey. “IEU works to determine if and how people changed their behavior for long term improvement of society.”

With more than 20 years’ experience as a program evaluator, adult educator, and qualitative research methodologist, Kelsey heads up a team of five individuals who lend their expertise to IEU. One post-doctoral scholar, two doctoral students on assistantship, and a student intern work together to evaluate a variety of programs across campus.

 

ALEC Alumna Serves UGA through the IEU

One of the doctoral students working in the IEU is Amanda Marabesi, a 2019 graduate of the Master of Agricultural and Environmental Education program. Marabesi earned her bachelor’s degree in Agronomic Engineering from the University of Sao Paolo before coming to UGA to gain expertise in the social science aspects of agriculture.

Through her master’s thesis work, Marabesi explored Georgia producers’ experiences growing organic produce and the Extension agents who support them. During this process, she realized a desire to learn more about agricultural production and decided to focus more on the “hard sciences.”

“I believe that what I learned from growers and Extension agents during my research made me want to help solve problems they faced, mainly the ones related to the agricultural production itself. Therefore, I applied to the Horticulture department to earn a PhD at UGA,” Marabesi says.

This translated well into her work in IEU because Marabesi is currently working on a USDA-funded project called “Lighting Approaches to Maximize Profits” (LAMP). She says this 5-year project examines lighting strategies for controlled environment agriculture.

“I love to work on this project because it is horticulture-related. I’m very interested in the topic and I think CEA will play a big role in the future of agriculture. By working on the evaluation for the project I had the opportunity to meet and work closely with many researchers and industry representatives across the country.”

Because of her unique position in the UGA Horticulture department, Amanda is able to uniquely navigate the work she does with LAMP and says “… by being a student in horticulture I have been developing a good understanding of the research they are doing in the project, I am familiar with scientific terms and also with the science behind the experiments. I think that is helpful when making evaluative judgments.”

Kelsey also recognizes the benefit of Marabesi’s background and expresses that her strong STEM framework and social sciences approach gives the work she does at IEU a special edge.

“She has a great mix of both social sciences and STEM science to understand a variety of projects. She is able to collect data using both qualitative and quantitative methods, and write research reports to summarize the findings that will help project leaders to better understand how their programs have affected clients,” Kelsey says.

The interdisciplinary work Amanda performs gives her valuable experience, especially in pursuing a career in academia. She says her work in a variety of disciplines provides perspective to see the “bigger picture” and the benefits of working together with other experts to draw connections.

“When I tell people that I am a Ph.D. student in horticulture and also work for the IEU, they get confused and ask me why. I tell them that I am trying to get the best out of my graduate school experience,” Marabesi says. “… I think that when you work with people from different research areas you expand your perspective and then you can see the bigger picture.”

 

Current Projects and Measuring Future Impacts

As the team at IEU look toward the future, they seek not only to evaluate impacts, but also help researchers begin their projects with an end result in mind.

Lewis Carroll once expressed a sentiment that went something like this, “If you don’t know where you are going, ending up anywhere will do.”

Kelsey says, “To help us from ending up ‘anywhere,’ evaluators create a road map to help project leaders accomplish their educational goals effectively, and to help them to stay on course so that their goals are accomplished.”

By guiding faculty to begin thinking about impacts while they are mapping their projects, IEU helps them “…to design the curriculum and activities with the end in mind.”

If you’re interested in working with the IEU, they partner with any agency that uses public money to deliver educational programs.

Learn more about IEU by visiting their website at ieu.uga.edu/.

By Allison Fortner