Recordings

Welcome and Introduction

Session I: Opportunities and Responsibility

Session II: Technological Foundations

Session III: Industry and Applications

Session IV: Harms and Society

Workshop Agenda

* Times are in Pacific Time

Day 1: August 23, 2021 9:30am - 2:30pm


Welcome and Introduction


9:30am - 9:35am

Introduction
Fei-Fei Li, Sequoia Professor, Computer Science Department, Stanford University; Denning Co-Director, Stanford Institute for Human-Centered Artificial Intelligence


9:35am - 10:00am

Foundation Models
Percy Liang, Associate Professor of Computer Science, Stanford University


Session I: Opportunities and Responsibility


Presentations

10:00am - 10:30am

What Has Happened, Where Are We Going, and Who Gets to Build Them
Jack Clark, Co-Founder, Anthropic; Co-chair of the AI Index; Co-chair of the OECD's working group on classifying and defining AI systems

10:30am - 10:40am

Threshold Effects
Michael Bernstein, Associate Professor of Computer Science, Stanford University

10:40am - 10:50am

Foundation Models for Law & The Law of Foundation Models: A U.S. Perspective
Dan Ho, William Benjamin Scott and Luna M. Scott Professor of Law, Professor of Political Science, Associate Director for the Stanford Institute for Human-Centered Artificial Intelligence (HAI)

10:50am - 11:00am

Joint Q&A


Panel

11:00am – 12:00pm

Jack Clark, Co-Founder, Anthropic; Co-chair of the AI Index; Co-chair of the OECD's working group on classifying and defining AI systems

Su Lin Blodgett, Postdoctoral Researcher, Microsoft

Eric Horvitz, Technical Fellow; Chief Scientific Officer, Microsoft

Joelle Pineau, Co-Managing Director, Facebook AI Research; Associate Professor and William Dawson Scholar of Computer Science, McGill University

Jacob Steinhardt, Assistant Professor of Statistics, University of California, Berkeley

Percy Liang (moderator), Associate Professor of Computer Science, Stanford University


12:00pm – 12:15pm

Break

Session II: Technological Foundations


Presentations

12:15pm - 12:45pm

David V.S. Goliath: the Art of Leaderboarding in the Era of Extreme-Scale Neural Models
Yejin Choi, Brett Helsel Professor at the Paul G. Allen School of Computer Science & Engineering, University of Washington; Senior Research Manager, Allen Institute for AI

12:45pm - 12:55pm

Broad Robot Generalization Requires Broad Offline Data
Chelsea Finn, Assistant Professor of Computer Science and Electrical Engineering, Stanford University

12:55pm - 1:05pm

Theory for Foundations Models: Analysis Framework, Recent Results, and Challenges
Tengyu Ma, Assistant Professor of Computer Science and Statistics, Stanford University

1:05pm - 1:15pm

On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies

Tatsu Hashimoto, Assistant Professor of Computer Science, Stanford University

1:15pm - 1:30pm

Joint Q&A


Panel

1:30pm – 2:30pm

Yejin Choi, Brett Helsel Professor at the Paul G. Allen School of Computer Science & Engineering, University of Washington; Senior Research Manager, Allen Institute for AI

Sanjeev Arora, Charles C. Fitzmorris Professor of Computer Science, Princeton University

Kavita Bala, Dean of the Ann S. Bowers College of Computing and Information Science, Cornell University

Jitendra Malik, Arthur J. Chick Professor of Electrical Engineering and Computer Science, University of California, Berkeley

Natalie Schluter, Senior Research Scientist, Google Brain; Associate Professor of Computer Science, IT University of Copenhagen

Chris Manning (moderator), Thomas M. Siebel Professor in Machine Learning, Professor of Linguistics and Computer Science, Stanford University; Associate Director, Stanford Institute for Human-Centered Artificial Intelligence


Day 2: August 24, 2021 9:30am - 2:30pm


Session III: Industry and Applications


Presentations

9:30am - 10:00am

Is Scale All We Need?
Slav Petrov, Distinguished Scientist and Senior Research Director, Google

10:00am - 10:10am

The Economic Implications of Foundation Models
Erik Brynjolfsson, Jerry Yang and Akiko Yamazaki Professor and Senior Fellow, HAI; Director of the Stanford Digital Economy Lab; Ralph Landau Senior Fellow, SIEPR, Stanford University

10:10am - 10:20am

Breaking the Systems Bottleneck: Faster and Cheaper Model Training
Matei Zaharia, Assistant Professor of Computer Science, Stanford University

10:20am - 10:30am

Towards Transparent Foundations -- Building Accessible Infrastructure for Training Large-Scale Language Models
Siddharth Karamcheti, PhD Student in Computer Science, Stanford University
Laurel Orr, Postdoctoral Fellow in Computer Science, Stanford University

10:30am - 10:45am

Joint Q&A


Panel

10:45am – 11:45am

Slav Petrov, Distinguished Scientist and Senior Research Director, Google

Michael Carbin, Associate Professor of Electrical Engineering and Computer Science, MIT

Pascale Fung, Director, Center for Artificial Intelligence Research; Professor of Electronic and Computer Engineering and Professor of Computer Science and Engineering, Hong Kong University of Science and Technology

Ilya Sutskever, Co-Founder and Chief Scientific Officer, OpenAI

Jakob Uszkoreit, Co-Founder and Chief Technology Officer, Inceptive

Thomas Wolf, Chief Scientific Officer, Hugging Face

Chris Ré (moderator), Associate Professor of Computer Science, Stanford University


11:45am – 12:00pm

Break

Session IV: Harms and Society


Presentations

12:00pm - 12:30pm

Cementing a Foundation of Inequity in AI
Margaret Mitchell, Research Scientist, Ethical AI, Hugging Face

12:30pm - 12:40pm

Anti-Muslim biases in large language models
James Zou, Assistant Professor of Biomedical Data Science, Stanford University

12:40pm - 12:50pm

How Foundation Models will Shape Disinformation, and Implications for Human Detection
Shelby Grossman, Research Scholar on Disinformation in Africa, Stanford Internet Observatory

12:50pm - 1:00pm

Homogenization and the Ethics of Scale
Katie Creel, Postdoctoral Research Fellow of Philosophy, McCoy Family Center for Ethics in Society; Embedded EthiCS Fellow, HAI, Stanford University

1:00pm - 1:15pm

Joint Q&A


Panel

1:15am – 2:15pm

Margaret Mitchell, Research Scientist, Ethical AI

Angèle Christin, Assistant Professor of Communication, Stanford University

Sarah Kreps, Chair and John L. Wetherill Professor of Government, Cornell University

Sameer Singh, Associate Professor of Computer Science, University of California, Irvine

Rob Reich (moderator), Professor of Political Science; Director of the Center for Ethics in Society; Co-director of the Center on Philanthropy and Civil Society; Associate Director, Stanford Institute for Human-Centered Artificial Intelligence, Stanford University


Closing Remarks

2:30pm

Percy Liang, Associate Professor of Computer Science, Stanford University


Description

The Center for Research on Foundation Models (CRFM), a new initiative of the Stanford Institute for Human-Centered Artificial Intelligence (HAI), invites you to the Workshop on Foundation Models from August 23-24, 2021. By foundation model (e.g. BERT, GPT-3, DALL-E), we mean a single model that is trained on raw data, potentially across multiple modalities, which can be usefully adapted to a wide range of tasks. These models have demonstrated clear potential, which we see as the beginnings of a sweeping paradigm shift in AI. They represent a dramatic increase in capability in terms of accuracy, generation quality, and extrapolation to new tasks, but they also pose clear risks such as use for widespread disinformation, potential exacerbation of historical inequities, and problematic centralization of power.

Given their anticipated impact, we invite you to join us at this workshop, where scholars reflecting a diverse array of perspectives, disciplinary backgrounds (e.g. social science, economics, computer science, law, philosophy, information science) and sectors (academia and industry) will convene to provide vital expertise on the many dimensions of foundation models. Broadly, we will address the opportunities, challenges, limitations, and societal impact of foundation models. Given that future AI systems will likely rely heavily on foundation models, it is imperative that we, as a community, come together to develop more rigorous principles for foundation models and guidance for their responsible development and deployment.

Specific points of emphasis include:

  1. What applications and communities might benefit the most from foundation models and what are some of the unique application-specific obstacles?
  2. How do we characterize and mitigate the disparate, and likely inequitable, effects of foundation models?
  3. How do multimodal methods and grounding impact conversations around meaning and semantics in foundation models?
  4. When foundation models are used in applications that cause harm, how do we handle matters of responsibility, accountability, and recourse?
  5. What should be the professional norms and ethical and legal considerations around the release and deployment of foundation models?
  6. How should various groups (e.g. academia, industry, government), given their complementary strengths, productively collaborate on developing foundation models?
  7. Given foundation models must be adapted for specific tasks, how do we evaluate them in ways that capture the needs of diverse stakeholders?
  8. Foundation models generally coincide with the centralization of power: how do we reason about this centralization, and its potential harms, and build ecosystems that better distribute the benefits of foundation models?
  9. Data plays a central role in foundation models: how do we think about data sourcing, selection, documentation, and how do we build principles to guide how data shapes foundation models?
  10. The scale of foundation models complicates principled scientific study: how do we build foundation models in a sound manner given the potential inability to run comprehensive experiments, and how do we reaffirm our commitments to open and reproducible science in spite of this scale?

Keynote Speakers

Jack Clark

Co-Founder, Anthropic; Co-chair of the AI Index; Co-chair of the OECD's working group on classifying and defining AI systems

Yejin Choi

Brett Helsel Professor at the Paul G. Allen School of Computer Science & Engineering, University of Washington; Senior Research Manager, Allen Institute for AI

Slav Petrov

Distinguished Scientist and Senior Research Director, Google

Margaret Mitchell

Research Scientist, Ethical AI

Workshop Organizers

Percy Liang

Associate Professor of Computer Science, Stanford University

Rishi Bommasani

PhD Student in Computer Science, Stanford University