In this talk, I aim to present insights into the design of evaluations of machine learning and AI systems to assess properties related to algorithmic fairness and health equity. I argue that such evaluations are contextual and require specification of the intended use, target population, and assumptions regarding the data generating process and measurement mechanism. Through examples from my research, I argue that causal graphical models can serve as key tools for context specification and can aid in understanding and appropriate use of analytical algorithmic fairness techniques. I will further discuss recent work related to evaluation of equity-related biases in large language models.
Speaker: Stephen Pfohl, PhD
Affiliation: Google Research
Position: Senior Research Scientist
Host: Anurag Vaidya, Mahmood Lab
Date: Monday January 27, 2025
Time: 4:00-5:00PM ET
Zoom: https://partners.zoom.us/j/82163676866
Meeting ID: 821 6367 6866
Research Links: Personal site, Google Scholar
Stephen Pfohl is a senior research scientist at Google Research. His work focuses on the incorporation of fairness, distribution shift, and equity considerations into the design and evaluation of machine learning systems in healthcare contexts. Stephen earned his PhD in Biomedical Informatics from Stanford University.