In this talk, I aim to present insights into the design of evaluations of machine learning and AI systems to assess properties related to algorithmic fairness and health equity. I argue that such evaluations are contextual and require specification of the intended use, target population, and assumptions regarding the data generating process and measurement mechanism. Through examples from my research, I argue that causal graphical models can serve as key tools for context specification and can aid in understanding and appropriate use of analytical algorithmic fairness techniques. I will further discuss recent work related to evaluation of equity-related biases in large language models.
Speaker: Stephen Pfohl, PhD
Affiliation: Google Research
Position: Senior Research Scientist
Host: Anurag Vaidya, Mahmood Lab
Date: Monday January 27, 2025
Time: 4:00-5:00PM ET
Zoom: https://partners.zoom.us/j/82163676866
Meeting ID: 821 6367 6866
Research Links: Personal site, Google Scholar
Stephen Pfohl is a senior research scientist at Google Research. His work focuses on the incorporation of fairness, distribution shift, and equity considerations into the design and evaluation of machine learning systems in healthcare contexts. Stephen earned his PhD in Biomedical Informatics from Stanford University.
The genome is a sequence that encodes the DNA, RNA, and proteins orchestrating an organism’s function. We present Evo, a long-context genomic foundation model with a frontier architecture trained on millions of prokaryotic and phage genomes, and report the first scaling laws on DNA to complement observations in language and vision. Evo generalizes across DNA, RNA, and proteins, enabling zero-shot function prediction competitive with domain-specific language models and the generation of functional CRISPR-Cas and transposon systems, representing the first examples of protein-RNA and protein-DNA co-design with a language model. Evo also learns how small mutations affect whole-organism fitness and generates megabase-scale sequences with plausible genomic architecture. These prediction and generation capabilities span molecular to genome scales of complexity, advancing our understanding and control of biology.
The integration of Artificial Intelligence (AI) in clinical pathology has faced significant hurdles due to constraints in data collection and challenges associated with model transparency and interpretability. In this talk, we introduce a novel digital pathology AI framework named nuclei.io, which leverages active learning and incorporates real-time feedback from human experts. This innovative approach empowers pathologists to quickly generate diverse datasets and develop models for various clinical applications. To demonstrate the effectiveness of our framework, we conducted two user studies employing such human–AI collaboration strategy. These studies focused on two key areas: the identification of plasma cells in endometrial biopsies and the detection of colorectal cancer metastasis in lymph nodes. The results from these studies showed significant enhancements in sensitivity, accuracy, and diagnostic efficiency with the integration of AI. Our findings underscore the benefits of the human-in-the-loop AI framework, highlighting its potential to transform the field of digital pathology.
Advancements in single-cell and spatial-omics technologies have created a need for integrating datasets across modalities with limited and weakly correlated features, such as those between spatial proteomics and transcriptomics. Existing tools, usually designed for strongly linked data, often fail in these scenarios. Recently, we have developed a series of methods (Mario and MaxFuse), that improves integration by refining weak correlations between modalities through an iterative smoothing and co-embedding process, and achieves single-cell level matching across these weakly linked modalities, enabling in-depth understanding of tissue micro-environments.
BWH Computational Pathology Special Seminar