Name
Applications
Date & Time
Friday, October 27, 2023, 1:30 PM - 3:20 PM
Speakers
Tesca Fitzgerald, Yale University "Benefits And Barriers To Using LLMs In Interactive Robots" As robots become more commonplace in our lives, our expectations of their ability to interact and adapt to our lives will grow as well. Meeting this challenge will require robots to (i) personalize their knowledge and action models to the specific settings in which they are deployed, (ii) incorporate and learn from feedback provided by end-users, and (iii) communicate their intentions and queries to those end-users. In this talk, I will discuss the potential benefits and challenges to utilizing LLMs for each of these research problems.
David Vandijk, Yale University Foundation Model Abc: Algorithms, Brains, And Cells Foundation models are revolutionizing the way we interact with expansive datasets, presenting unparalleled opportunities in the biomedical field. In this talk, I will showcase three pioneering projects from our lab, each aiming to harness the potential of foundation models in biomedicine. At the forefront is our cellular language model, trained on millions of human and mouse cells, tailored for functional cross-species translation. This model seamlessly bridges the biological nuances between humans and mice. Its strength not only paves the way for the development of enhanced mouse models of human diseases but also aids in better translating mouse research into effective and safe human treatments. We then transition to the Cell2Sentence method. By translating single-cell gene expression profiles into textual sequences, this approach paves the way for Large Language Model (LLM) fine-tuning. Cell2Sentence can generate biologically accurate cells from natural language prompts as well as interpret cellular data in human language terms. The synergy of natural language pretraining with cellular fine-tuning empowers Cell2Sentence to grasp genomic intricacies, all while preserving its linguistic comprehension. Our exploration culminates with the Brain Language Model (BrainLM). Anchored on a vast dataset of 6,700 hours of fMRI recordings, BrainLM emerges as a robust tool. It excels in forecasting clinical outcomes, mapping neural trajectories, and identifying functional networks. Through BrainLM, we exemplify the profound insights attainable when foundation models and LLMs converge with large biomedical datasets.
Alex Wong, Yale University Unsupervised Learning Of Depth Perception And Beyond Deep neural networks are highly parameterized functions that have seen empirical successes across a number of computer vision tasks. Due to their size, they require tens of thousands to millions of training examples. Curating vision datasets amounts to numerous man-hours; tasks like depth estimation requires an even more massive effort. In this talk, I will introduce an alternative form of supervision that leverages multi-sensor validation as an unsupervised training objective for depth estimation. Additionally, I will demonstrate how one can leverage synthetic data and the abundance of pretrained models available online, which has traditionally relied largely on expensive manual labeling, to learn regularities of our visual world. In doing so, I show that one can design smaller and faster models that can operate in real-time with comparable performance to state-of-the-art methods trained on human annotated ground truth. I will discuss challenges in how we train these models and finally by scaling up the training we show that they are a viable form of pretraining for a downstream semantic task.
Anjalie Field, Johns Hopkins University Risks Of Racial Bias In Nlp For Child Protective Services Practitioners are increasingly using algorithmic tools in high-stakes settings, like healthcare, social services, policing, and education with particular recent interest in natural language processing (NLP). These domains raise a number of challenges, including ensuring model reliability, preserving data privacy, and developing approaches that can mitigate, rather than exacerbate historical bias. In this talk, I will discuss recent work in one of these areas: the risks of perpetuating racial bias when deploying NLP in child protective services (CPS). CPS workers often write copious free-form text notes about families they are working with, and agencies are actively seeking to deploy NLP models to leverage these data. However, these models are well-documented to perpetuate and amplify data biases, though model biases are often measured over data designed for probing models, rather than reflective of real use cases. In investing potential NLP deployment in this domain, we find surprisingly little evidence of exacerbated racial bias in risk prediction, but we do identify algorithmic unfairness in information extraction tools, which could lead to direct harms. While there is existing pronounced criticism of risk prediction, our results expose previously undocumented risks of racial bias in realistic information extraction systems, highlighting potential concerns in deploying them, even though they may appear more benign. Our work serves as a starting point for exploring potential real-word impacts of NLP models in high-stakes domains.
Shreya Saxena, Yale University Foundation Models In Neuroscience TBD
David Vandijk, Yale University Foundation Model Abc: Algorithms, Brains, And Cells Foundation models are revolutionizing the way we interact with expansive datasets, presenting unparalleled opportunities in the biomedical field. In this talk, I will showcase three pioneering projects from our lab, each aiming to harness the potential of foundation models in biomedicine. At the forefront is our cellular language model, trained on millions of human and mouse cells, tailored for functional cross-species translation. This model seamlessly bridges the biological nuances between humans and mice. Its strength not only paves the way for the development of enhanced mouse models of human diseases but also aids in better translating mouse research into effective and safe human treatments. We then transition to the Cell2Sentence method. By translating single-cell gene expression profiles into textual sequences, this approach paves the way for Large Language Model (LLM) fine-tuning. Cell2Sentence can generate biologically accurate cells from natural language prompts as well as interpret cellular data in human language terms. The synergy of natural language pretraining with cellular fine-tuning empowers Cell2Sentence to grasp genomic intricacies, all while preserving its linguistic comprehension. Our exploration culminates with the Brain Language Model (BrainLM). Anchored on a vast dataset of 6,700 hours of fMRI recordings, BrainLM emerges as a robust tool. It excels in forecasting clinical outcomes, mapping neural trajectories, and identifying functional networks. Through BrainLM, we exemplify the profound insights attainable when foundation models and LLMs converge with large biomedical datasets.
Alex Wong, Yale University Unsupervised Learning Of Depth Perception And Beyond Deep neural networks are highly parameterized functions that have seen empirical successes across a number of computer vision tasks. Due to their size, they require tens of thousands to millions of training examples. Curating vision datasets amounts to numerous man-hours; tasks like depth estimation requires an even more massive effort. In this talk, I will introduce an alternative form of supervision that leverages multi-sensor validation as an unsupervised training objective for depth estimation. Additionally, I will demonstrate how one can leverage synthetic data and the abundance of pretrained models available online, which has traditionally relied largely on expensive manual labeling, to learn regularities of our visual world. In doing so, I show that one can design smaller and faster models that can operate in real-time with comparable performance to state-of-the-art methods trained on human annotated ground truth. I will discuss challenges in how we train these models and finally by scaling up the training we show that they are a viable form of pretraining for a downstream semantic task.
Anjalie Field, Johns Hopkins University Risks Of Racial Bias In Nlp For Child Protective Services Practitioners are increasingly using algorithmic tools in high-stakes settings, like healthcare, social services, policing, and education with particular recent interest in natural language processing (NLP). These domains raise a number of challenges, including ensuring model reliability, preserving data privacy, and developing approaches that can mitigate, rather than exacerbate historical bias. In this talk, I will discuss recent work in one of these areas: the risks of perpetuating racial bias when deploying NLP in child protective services (CPS). CPS workers often write copious free-form text notes about families they are working with, and agencies are actively seeking to deploy NLP models to leverage these data. However, these models are well-documented to perpetuate and amplify data biases, though model biases are often measured over data designed for probing models, rather than reflective of real use cases. In investing potential NLP deployment in this domain, we find surprisingly little evidence of exacerbated racial bias in risk prediction, but we do identify algorithmic unfairness in information extraction tools, which could lead to direct harms. While there is existing pronounced criticism of risk prediction, our results expose previously undocumented risks of racial bias in realistic information extraction systems, highlighting potential concerns in deploying them, even though they may appear more benign. Our work serves as a starting point for exploring potential real-word impacts of NLP models in high-stakes domains.
Shreya Saxena, Yale University Foundation Models In Neuroscience TBD
Location Name
FDS in Kline Tower: 13th Floor
Full Address
Kline Tower - 13th and 14th Floors
219 Prospect St
New Haven, CT 06511
United States
219 Prospect St
New Haven, CT 06511
United States
Session Type
Workshop