Recordings
Welcome and Introduction
Session I: Opportunities and Responsibility
Session II: Technological Foundations
Session III: Industry and Applications
Session IV: Harms and Society
Workshop Agenda
* Times are in Pacific Time
Day 1: August 23, 2021 9:30am - 2:30pm
Welcome and Introduction
9:30am - 9:35am
Introduction
Fei-Fei Li, Sequoia Professor, Computer Science Department, Stanford University; Denning Co-Director, Stanford Institute for Human-Centered Artificial Intelligence
9:35am - 10:00am
Foundation Models
Percy Liang, Associate Professor of Computer Science, Stanford University
Session I: Opportunities and Responsibility
Presentations
10:00am - 10:30am
What Has Happened, Where Are We Going, and Who Gets to Build Them
Jack Clark, Co-Founder, Anthropic; Co-chair of the AI Index; Co-chair of the OECD's working group on classifying and defining AI systems
10:30am - 10:40am
Threshold Effects
Michael Bernstein, Associate Professor of Computer Science, Stanford University
10:40am - 10:50am
Foundation Models for Law & The Law of Foundation Models: A U.S. Perspective
Dan Ho, William Benjamin Scott and Luna M. Scott Professor of Law, Professor of Political Science, Associate Director for the Stanford Institute for Human-Centered Artificial Intelligence (HAI)
10:50am - 11:00am
Joint Q&A
Panel
11:00am – 12:00pm
Jack Clark, Co-Founder, Anthropic; Co-chair of the AI Index; Co-chair of the OECD's working group on classifying and defining AI systems
Su Lin Blodgett, Postdoctoral Researcher, Microsoft
Eric Horvitz, Technical Fellow; Chief Scientific Officer, Microsoft
Joelle Pineau, Co-Managing Director, Facebook AI Research; Associate Professor and William Dawson Scholar of Computer Science, McGill University
Jacob Steinhardt, Assistant Professor of Statistics, University of California, Berkeley
Percy Liang (moderator), Associate Professor of Computer Science, Stanford University
12:00pm – 12:15pm
Break
Session II: Technological Foundations
Presentations
12:15pm - 12:45pm
David V.S. Goliath: the Art of Leaderboarding in the Era of Extreme-Scale Neural Models
Yejin Choi, Brett Helsel Professor at the Paul G. Allen School of Computer Science & Engineering, University of Washington; Senior Research Manager, Allen Institute for AI
12:45pm - 12:55pm
Broad Robot Generalization Requires Broad Offline Data
Chelsea Finn, Assistant Professor of Computer Science and Electrical Engineering, Stanford University
12:55pm - 1:05pm
Theory for Foundations Models: Analysis Framework, Recent Results, and Challenges
Tengyu Ma, Assistant Professor of Computer Science and Statistics, Stanford University
1:05pm - 1:15pm
Tatsu Hashimoto, Assistant Professor of Computer Science, Stanford University
1:15pm - 1:30pm
Joint Q&A
Panel
1:30pm – 2:30pm
Yejin Choi, Brett Helsel Professor at the Paul G. Allen School of Computer Science & Engineering, University of Washington; Senior Research Manager, Allen Institute for AI
Sanjeev Arora, Charles C. Fitzmorris Professor of Computer Science, Princeton University
Kavita Bala, Dean of the Ann S. Bowers College of Computing and Information Science, Cornell University
Jitendra Malik, Arthur J. Chick Professor of Electrical Engineering and Computer Science, University of California, Berkeley
Natalie Schluter, Senior Research Scientist, Google Brain; Associate Professor of Computer Science, IT University of Copenhagen
Chris Manning (moderator), Thomas M. Siebel Professor in Machine Learning, Professor of Linguistics and Computer Science, Stanford University; Associate Director, Stanford Institute for Human-Centered Artificial Intelligence
Day 2: August 24, 2021 9:30am - 2:30pm
Session III: Industry and Applications
Presentations
9:30am - 10:00am
Is Scale All We Need?
Slav Petrov, Distinguished Scientist and Senior Research Director, Google
10:00am - 10:10am
The Economic Implications of Foundation Models
Erik Brynjolfsson, Jerry Yang and Akiko Yamazaki Professor and Senior Fellow, HAI; Director of the Stanford Digital Economy Lab; Ralph Landau Senior Fellow, SIEPR, Stanford University
10:10am - 10:20am
Breaking the Systems Bottleneck: Faster and Cheaper Model Training
Matei Zaharia, Assistant Professor of Computer Science, Stanford University
10:20am - 10:30am
Towards Transparent Foundations -- Building Accessible Infrastructure for Training Large-Scale Language Models
Siddharth Karamcheti, PhD Student in Computer Science, Stanford University
Laurel Orr, Postdoctoral Fellow in Computer Science, Stanford University
10:30am - 10:45am
Joint Q&A
Panel
10:45am – 11:45am
Slav Petrov, Distinguished Scientist and Senior Research Director, Google
Michael Carbin, Associate Professor of Electrical Engineering and Computer Science, MIT
Pascale Fung, Director, Center for Artificial Intelligence Research; Professor of Electronic and Computer Engineering and Professor of Computer Science and Engineering, Hong Kong University of Science and Technology
Ilya Sutskever, Co-Founder and Chief Scientific Officer, OpenAI
Jakob Uszkoreit, Co-Founder and Chief Technology Officer, Inceptive
Thomas Wolf, Chief Scientific Officer, Hugging Face
Chris Ré (moderator), Associate Professor of Computer Science, Stanford University
11:45am – 12:00pm
Break
Session IV: Harms and Society
Presentations
12:00pm - 12:30pm
Cementing a Foundation of Inequity in AI
Margaret Mitchell, Research Scientist, Ethical AI, Hugging Face
12:30pm - 12:40pm
Anti-Muslim biases in large language models
James Zou, Assistant Professor of Biomedical Data Science, Stanford University
12:40pm - 12:50pm
How Foundation Models will Shape Disinformation, and Implications for Human Detection
Shelby Grossman, Research Scholar on Disinformation in Africa, Stanford Internet Observatory
12:50pm - 1:00pm
Homogenization and the Ethics of Scale
Katie Creel, Postdoctoral Research Fellow of Philosophy, McCoy Family Center for Ethics in Society; Embedded EthiCS Fellow, HAI, Stanford University
1:00pm - 1:15pm
Joint Q&A
Panel
1:15am – 2:15pm
Margaret Mitchell, Research Scientist, Ethical AI
Angèle Christin, Assistant Professor of Communication, Stanford University
Sarah Kreps, Chair and John L. Wetherill Professor of Government, Cornell University
Sameer Singh, Associate Professor of Computer Science, University of California, Irvine
Rob Reich (moderator), Professor of Political Science; Director of the Center for Ethics in Society; Co-director of the Center on Philanthropy and Civil Society; Associate Director, Stanford Institute for Human-Centered Artificial Intelligence, Stanford University
Closing Remarks
2:30pm
Percy Liang, Associate Professor of Computer Science, Stanford University
Description
The Center for Research on Foundation Models (CRFM), a new initiative of the Stanford Institute for Human-Centered Artificial Intelligence (HAI), invites you to the Workshop on Foundation Models from August 23-24, 2021. By foundation model (e.g. BERT, GPT-3, DALL-E), we mean a single model that is trained on raw data, potentially across multiple modalities, which can be usefully adapted to a wide range of tasks. These models have demonstrated clear potential, which we see as the beginnings of a sweeping paradigm shift in AI. They represent a dramatic increase in capability in terms of accuracy, generation quality, and extrapolation to new tasks, but they also pose clear risks such as use for widespread disinformation, potential exacerbation of historical inequities, and problematic centralization of power.
Given their anticipated impact, we invite you to join us at this workshop, where scholars reflecting a diverse array of perspectives, disciplinary backgrounds (e.g. social science, economics, computer science, law, philosophy, information science) and sectors (academia and industry) will convene to provide vital expertise on the many dimensions of foundation models. Broadly, we will address the opportunities, challenges, limitations, and societal impact of foundation models. Given that future AI systems will likely rely heavily on foundation models, it is imperative that we, as a community, come together to develop more rigorous principles for foundation models and guidance for their responsible development and deployment.
Specific points of emphasis include:
- What applications and communities might benefit the most from foundation models and what are some of the unique application-specific obstacles?
- How do we characterize and mitigate the disparate, and likely inequitable, effects of foundation models?
- How do multimodal methods and grounding impact conversations around meaning and semantics in foundation models?
- When foundation models are used in applications that cause harm, how do we handle matters of responsibility, accountability, and recourse?
- What should be the professional norms and ethical and legal considerations around the release and deployment of foundation models?
- How should various groups (e.g. academia, industry, government), given their complementary strengths, productively collaborate on developing foundation models?
- Given foundation models must be adapted for specific tasks, how do we evaluate them in ways that capture the needs of diverse stakeholders?
- Foundation models generally coincide with the centralization of power: how do we reason about this centralization, and its potential harms, and build ecosystems that better distribute the benefits of foundation models?
- Data plays a central role in foundation models: how do we think about data sourcing, selection, documentation, and how do we build principles to guide how data shapes foundation models?
- The scale of foundation models complicates principled scientific study: how do we build foundation models in a sound manner given the potential inability to run comprehensive experiments, and how do we reaffirm our commitments to open and reproducible science in spite of this scale?
Keynote Speakers
Jack Clark
Co-Founder, Anthropic; Co-chair of the AI Index; Co-chair of the OECD's working group on classifying and defining AI systems
Yejin Choi
Brett Helsel Professor at the Paul G. Allen School of Computer Science & Engineering, University of Washington; Senior Research Manager, Allen Institute for AI
Slav Petrov
Distinguished Scientist and Senior Research Director, Google
Margaret Mitchell
Research Scientist, Ethical AI
Workshop Organizers
Percy Liang
Associate Professor of Computer Science, Stanford University
Rishi Bommasani
PhD Student in Computer Science, Stanford University