Part I: Background on LLMs
Conference Room A
- Introduction to LLMs
- Training Process of LLMs
- Root Causes of Bias in LLMs
- Training Process of LLMs
- Root Causes of Bias in LLMs
Overview
Large Language Models (LLMs), such as BERT, GPT-3, and LLaMA, have shown powerful performance and development prospects in various tasks of Natural Language Processing due to their robust text encoding and decoding capabilities and discovered emergent capabilities (e.g., reasoning). Despite their great performance, LLMs tend to inherit bias from multiple sources, including training data, encoding processes, and fine-tuning procedures, which may result in biased decisions against certain groups defined by the sensitive attribute (e.g., age, gender, or race). The biased prediction has raised significant ethical and societal concerns, severely limiting the adoption of LLMs in high-risk decision-making scenarios such as hiring, loan approvals, legal sentencing, and medical diagnoses.
The necessity for a comprehensive understanding of how different fair LLM methodologies are implemented and understood across diverse studies. Lacking clarity on these correspondences, the design of future fair LLMs can become challenging. Consequently, there is a pressing need for a systematic tutorial elucidating the recent advancements in fair LLMs. However, although there are several tutorials that address fairness in machine learning algorithms, these primarily focus on fairness in broader machine learning algorithms. There is a noticeable gap in inclusive resources that specifically address fairness within LLMs, distinguishing it from traditional models and discussing recent developments. To address this need, we present a tutorial on fairness in Large Language Models: Recent Advances and Future. This tutorial aims to provide researchers, developers, and practitioners an up-to-date and comprehensive review of existing work on fair LLMs.
Our tutorial is structured into five key parts:
- Background on LLMs
- Quantifying Bias in LLMs
- Mitigating Bias in LLMs
- Resources for Evaluating Bias
- Challenges and Future Directions
Speakers
Agenda
9:00 - 9:30
9:30 - 10:00
Part II: Quantifying Bias in LLMs
Conference Room A
- Demographic representation
- Stereotypical association
- Counterfactual fairness
- Performance disparities
- Stereotypical association
- Counterfactual fairness
- Performance disparities
Coffee Break
Room A5
Coffee and cakes lorem ipsum dolor sit amet, consectetur adipiscing elit.
10:30 - 11:00
Part III: Mitigating Bias in LLMs
Conference Room A
- Pre-processing
- In-training
- Intra-processing
- Post-processing
- In-training
- Intra-processing
- Post-processing
11:00 - 12:00
Part IV: Resources for Evaluating Bias
Conference Room B
- Toolkits
- Datasets
- Datasets
13:30 - 14:30
Part V: Challenges and Future Directions
Conference Room C
- Formulating Fairness Notions
- Rational Counterfactual Data Augmentation
- Balancing Performance and Fairness in LLMs
- Fulfilling Multiple Types of Fairness
- Developing More and Tailored Datasets
- Rational Counterfactual Data Augmentation
- Balancing Performance and Fairness in LLMs
- Fulfilling Multiple Types of Fairness
- Developing More and Tailored Datasets