Learn how to move beyond vanity metrics to show the tangible impact of your work to key stakeholders. Vanity metrics are things like number of people served, satisfaction scores, and referral rates. These metrics are important (as we do need to capture how many people our programs and services support). However, this data is not sufficient to tell stakeholders the impact of the programs and people served. Impact is not defined by “how many” and “how much.” It’s defined by “so what.” Workshop participants will work through a 4-part process to tell their impact story using credible data.

In this session, you will:

• Share stories of measurement challenges and successes in a guided icebreaker.
  
• Explore the impact hypothesis framework and how it connects learning initiatives to business outcomes.
  
• Identify the right business questions: The impact hypothesis helps you focus on the business problems learning is intended to solve—before jumping to metrics—so your measurement strategy is grounded in what stakeholders actually care about.
• Apply the impact hypothesis to a real-world case study to see the framework in action.
• Identify the right data to collect based upon the metrics outlined in the hypothesis.
• Translate data into compelling stakeholder narratives. The impact hypothesis gives you a ready-made structure to communicate impact clearly—strengthening your credibility, influence, and career positioning with leadership.
• Troubleshoot and refine your impact hypothesis during a closing Q&A session.
  
  

About our Speaker

Alaina Szlachta, PhD, is an academic-turned-entrepreneur and founder of By Design Development Solutions. She helps teams and businesses optimize their programs and services using original data. With a PhD in education and human behavior and experience conducting original research studies for universities, small businesses, and for her own firm, she offers expertise in the art and science of data. She is also author of "Measurement and Evaluation on a Shoestring" (ATD Press, 2024) and host of Measurement Made Easy, a free community for anyone eager to leverage the benefits of data science.

Connect with Dr. Szlachta on LinkedIn

Demonstrating the return on investment of training programs is an important, though often intimidating, challenge in L&D. Even when we understand what a Level 4 evaluation requires, the lack of statistical background or data infrastructure makes it difficult to execute. This session makes that work more accessible.

Using real and synthetic datasets, participants will work through an end-to-end workflow for using AI tools to analyze training data, interpret results, and communicate impact—all without a statistics degree. We'll cover what to measure and how to work with the data you actually have, how to use AI to clean messy datasets, key data governance considerations for practitioners working with institutional data, and how to use large language models to run analyses, explain statistical results in plain language, and generate compelling visualizations. Throughout, we will touch on some statistical foundations, like what t-tests tell us, to help you ask the right questions and trust your conclusions.

Whether you are constructing a Level 4 evaluation from scratch or trying to make sense of data (however messy) you already have, you'll leave with practical tools and a workflow you can apply immediately.

In this session, you will:

• Identify the opportunities and constraints for leveraging AI to conduct Level 4 evaluations in your organizational context
  
• Leverage large language models to clean messy data, perform statistical analyses, interpret results in plain language, and generate data visualizations.
  
• Articulate basic statistical concepts (like what t-tests measure) well enough to communicate findings to organizational stakeholders.

About our Speaker

Eric de Araujo, PhD is a Lead Instructional Designer at Purdue University, where he works at the intersection of online learning, faculty development, artificial intelligence, and data-informed decision making. With a background spanning philosophy, program development, and applied AI, he brings both theoretical grounding and practical experience to the challenges facing today's L&D professionals.

Eric has developed AI-powered tools that streamline learning design workflows and has presented on AI applications in education and training at universities and national conferences including OLC Accelerate and UPCEA. He recently completed the Data Science Methods for Digital Learning Platforms graduate certificate at the University of Pennsylvania. 

Part of his current work focuses on using AI to make data analysis accessible to learning professionals who are not trained data scientists.

Connect with Dr. de Araujo on LinkedIn