Blog Posts
HELM Safety: Towards Standardized Safety Evaluations of Language Models
Farzaan Kaiyom
and
Ahmed Ahmed
and
Yifan Mai
and
Kevin Klyman*
and
Rishi Bommasani
and
Percy Liang
Advancing Customizable Benchmarking in HELM via Unitxt Integration
Yifan Mai
and
Elron Bandel
and
Michal Shmueli-Scheuer
and
Percy Liang
ThaiExam Leaderboard in HELM
Yifan Mai
and
Potsawee Manakul
and
Sitipong Sripaisarnmongkol
and
Kunat Pipatanakul
and
Percy Liang
Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models
Andy K. Zhang
and
Neil Perry
and
Riya Dulepet
and
Joey Ji
and
Justin W. Lin
and
Eliot Jones
and
Celeste Menders
and
Gashon Hussein
and
Samantha Liu
and
Donovan Jasper
and
Pura Peetathawatchai
and
Ari Glenn
and
Vikram Sivashankar
and
Daniel Zamoshchin
and
Leo Glikbarg
and
Derek Askaryar
and
Mike Yang
and
Teddy Zhang
and
Rishi Alluri
and
Nathan Tran
and
Rinnara Sangpisit
and
Polycarpos Yiorkadjis
and
Kenny Osele
and
Gautham Raghupathi
and
Dan Boneh
and
Daniel E. Ho
and
Percy Liang
Co-Composition with an Anticipatory Music Transformer
John Thickstun
and
Carol Reiley
and
Percy Liang
Foundation Models under the EU AI Act
Rishi Bommasani
and
Alice Hau
and
Kevin Klyman
and
Percy Liang
The AI Executive Order through the lens of the AI Index
Jonathan Xue
and
Rishi Bommasani
The Foundation Model Transparency Index after 6 months
Rishi Bommasani
and
Kevin Klyman
and
Sayash Kapoor
and
Shayne Longpre
and
Betty Xiong
and
Nestor Maslej
and
Percy Liang
The First Steps to Holistic Evaluation of Vision-Language Models
Tony Lee
and
Yifan Mai
and
Chi Heem Wong
and
Josselin Somerville Roberts
and
Michihiro Yasunaga
and
Farzaan Kaiyom
and
Rishi Bommasani
and
Percy Liang
Massive Multitask Language Understanding (MMLU) on HELM
Yifan Mai
and
Percy Liang
Acceptable Use Policies for Foundation Models
Kevin Klyman
HELM Instruct: A Multidimensional Instruction Following Evaluation Framework with Absolute Ratings
Yian Zhang†
and
Yifan Mai
and
Josselin Somerville Roberts
and
Rishi Bommasani
and
Yann Dubois
and
Percy Liang
Plans for v1.1 of the Foundation Model Transparency Index: Self-Assessment
Foundation Model Transparency Index
HELM Lite: Lightweight and Broad Capabilities Evaluation
Percy Liang
and
Yifan Mai
and
Josselin Somerville
and
Farzaan Kaiyom
and
Tony Lee
and
Rishi Bommasani
Towards compromise: A concrete two-tier proposal for foundation models in the EU AI Act
Rishi Bommasani
and
Tatsunori Hashimoto
and
Daniel E. Ho
and
Marietje Schaake
and
Percy Liang
Drawing Lines: Tiers for Foundation Models
Rishi Bommasani
Flash-Decoding for long-context inference
Tri Dao
and
Daniel Haziza
and
Francisco Massa
and
Grigory Sizov
Observations from HALIE:
A Closer Look at Human-LM Interactions in Information-Seeking Contexts
Megha Srivastava
and
John Thickstun
Robust Distortion-free Watermarks for Language Models
Rohith Kuditipudi
and
John Thickstun
and
Tatsunori Hashimoto
and
Percy Liang
HyenaDNA: learning from DNA with 1 Million token context
Eric Nguyen*
and
Michael Poli*
and
Marjan Faizi
Levanter — Legible, Scalable, Reproducible Foundation Models with JAX
David Hall
and
Ivan Zhou
and
Percy Liang
Anticipatory Music Transformer: A Controllable Infilling Model for Music
John Thickstun
and
David Hall
and
Chris Donahue
and
Percy Liang
Do Foundation Model Providers Comply with the Draft EU AI Act?
Rishi Bommasani
and
Kevin Klyman
and
Daniel Zhang
and
Percy Liang
RA-CM3: Retrieval-Augmented Multimodal Modeling
Michihiro Yasunaga
and
Armen Aghajanyan
and
Weijia Shi
and
Rich James
and
Jure Leskovec
and
Percy Liang
and
Mike Lewis
and
Luke Zettlemoyer
and
Wen-tau Yih
AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
Yann Dubois*
and
Xuechen Li*
and
Rohan Taori*
and
Tianyi Zhang*
and
Ishaan Gulrajani
and
Jimmy Ba
and
Carlos Guestrin
and
Percy Liang
and
Tatsunori B. Hashimoto
Ecosystem Graphs: The Social Footprint of Foundation Models
Rishi Bommasani
and
Thomas Liao
and
Percy Liang
From Deep to Long Learning?
Dan Fu
and
Michael Poli
and
Chris Ré
Alpaca: A Strong, Replicable Instruction-Following Model
Rohan Taori*
and
Ishaan Gulrajani*
and
Tianyi Zhang*
and
Yann Dubois*
and
Xuechen Li*
and
Carlos Guestrin
and
Percy Liang
and
Tatsunori B. Hashimoto
Hyena Hierarchy: Towards Larger Convolutional Language Models
Michael Poli*
and
Stefano Massaroli*
and
Eric Nguyen*
and
Dan Fu
and
Tri Dao
and
Stephen A. Baccus
and
Yoshua Bengio
and
Stefano Ermon
and
Chris Ré
Meerkat and the Path to Foundation Models as a Reliable Software Abstraction
Karan Goel
and
Sabri Eyuboglu
and
Arjun Desai
and
James Zou
and
Chris Ré
Year in Review 2022: Technical Advances, Applications, and Social Responsibility
Michael Wornow
and
Nandita Bhaskhar
and
Simran Arora
and
Percy Liang
Data Wrangling with Foundation Models
Avanika Narayan
and
Ines Chami
and
Laurel Orr
and
Chris Ré
BioMedLM
Elliot Bolton
and
David Hall
and
Michihiro Yasunaga
and
Tony Lee
and
Chris Manning
and
Percy Liang
Language Models are Changing AI: The Need for Holistic Evaluation
Rishi Bommasani
and
Percy Liang
and
Tony Lee
Borrowing from the Law to Filter Training Data for Foundation Models
Katharine Miller
How does in-context learning work? A framework for understanding the differences from traditional supervised learning
Sang Michael Xie
and
Sewon Min
Can Foundation Models Help Us Achieve "Perfect Secrecy"?
Nikki Goth Itoi
Percy Liang on CRFM's first and next 30 years
Chris Potts
Can Longer Sequences Help Take the Next Leap in AI?
Chris Re
and
Tri Dao
and
Dan Fu
and
Karan Goel
LinkBERT: Improving Language Model Training with Document Link
Michihiro Yasunaga
The Time Is Now to Develop Community Norms for the Release of Foundation Models
Percy Liang
and
Rishi Bommasani
and
Kathleen Creel
and
Rob Reich
Reflections on Foundation Models
Rishi Bommasani
and
Percy Liang
Mistral — A Journey towards Reproducible Language Model Training
Siddharth Karamcheti
and
Laurel Orr