2022
Domino: Discovering Systematic Errors with Cross-Modal Embeddings [ Paper ]
Sabri Eyuboglu*, Maya Varma*, Khaled Saab*, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, Christopher Ré
Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution [ Paper ]
Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, Percy Liang
An Explanation of In-context Learning as Implicit Bayesian Inference [ Paper ]
Sang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu Ma
Meta-Learning with Fewer Tasks through Task Interpolation [ Paper ]
Huaxiu Yao, Linjun Zhang, Chelsea Finn
LinkBERT: Pretraining Language Models with Document Links [ Paper ]
Michihiro Yasunaga, Jure Leskovec, Percy Liang
MetaMorph: Learning Universal Controllers with Transformers [ Paper ]
Agrim Gupta, Linxi Fan, Surya Ganguli, Li Fei-Fei
GreaseLM: Graph REASoning Enhanced Language Models for Question Answering [ Paper ]
Xikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren, Percy Liang, Christopher D. Manning, Jure Leskovec
Self-supervised Learning is More Robust to Dataset Imbalance [ Paper ]
Hong Liu, Jeff Z. HaoChen, Adrien Gaidon, Tengyu Ma
Large language models can be strong differentially private learners [ Paper ]
Xuechen Li, Florian Tramèr, Percy Liang, Tatsunori Hashimoto
CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities [ Paper ]
Mina Lee, Percy Liang, Qian Yang
2021
Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data [ Paper ]
Colin Wei, Kendrick Shen, Yining Chen, Tengyu Ma
Relevance-guided Supervision for OpenQA with ColBERT [ Paper ]
Omar Khattab, Christopher Potts, Matei Zaharia
Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for Improved Generalization [ Paper ]
Sang Michael Xie, Tengyu Ma, Percy Liang
In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness [ Paper ]
Sang Michael Xie*, Ananya Kumar*, Robbie Jones*, Fereshte Khani, Tengyu Ma, Percy Liang
Efficient Large-Scale Language Model Training on GPU Clusters [ Paper ]
Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Anand Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, Matei Zaharia
Memory-Efficient Pipeline-Parallel DNN Training [ Paper ]
Deepak Narayanan, Amar Phanishayee, Kaiyu Shi, Xie Chen, Matei Zaharia
On the Inductive bias of Masked Language Modeling: From Statistical to Syntactic Dependencies [ Paper ]
Tianyi Zhang, Tatsunori B. Hashimoto
Viewmaker Networks: Learning Views for Unsupervised Representation Learning [ Paper ]
Alex Tamkin, Mike Wu, Noah Goodman
Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors [ Paper ]
Michelle A. Lee, Matthew Tan, Yuke Zhu, Jeannette Bohg
Concept2Robot: Learning Manipulation Concepts from Instructions and Human Demonstrations [ Paper ]
Lin Shao, Toki Migimatsu, Qiang Zhang, Karen Yang, Jeannette Bohg
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning [ Paper ]
Colin Wei, Sang Michael Xie, Tengyu Ma
DABS: A Domain-Agnostic Benchmark for Self-Supervised Learning [ Paper ]
Alex Tamkin, Vincent Liu, Rongfei Lu, Daniel Fein, Colin Schultz, Noah Goodman
When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset [ Paper ]
Lucia Zheng, Neel Guha, Brandon R. Anderson, Peter Henderson, Daniel E. Ho
Swords ⚔️: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality [ Paper ]
Mina Lee, Chris Donahue, Robin Jia, Alexander Iyabor, Percy Liang
Learning Generalizable Robotic Reward Functions from “In-The-Wild” Human Videos [ Paper ]
Annie S. Chen, Suraj Nair, Chelsea Finn
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss [ Paper ]
Jeff Z. HaoChen, Colin Wei, Adrien Gaidon, Tengyu Ma
Combiner: Full Attention Transformer with Sparse Computation Cost [ Paper ]
Hongyu Ren, Hanjun Dai, Zihang Dai, Mengjiao Yang, Jure Leskovec, Dale Schuurmans, Bo Dai
LM-Critic: Language Models for Unsupervised Grammatical Error Correction [ Paper ]
Michihiro Yasunaga, Jure Leskovec, Percy Liang
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering [ Paper ]
Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, Jure Leskovec
CoCon: Cooperative-Contrastive Learning [ Paper ]
Nishant Rai, Ehsan Adeli, Kuan-Hui Lee, Adrien Gaidon, Juan Carlos Niebles
Identification of disease treatment mechanisms through the multiscale interactome [ Paper ]
Camilo Ruiz, Marinka Zitnik, Jure Leskovec
Codified Audio Language Modeling Learns Useful Representations for Music Information Retrieval [ Paper ]
Rodrigo Castellon, Chris Donahue, Percy Liang
System Error: Where Big Tech Went Wrong and How We Can Reboot [ Paper ]
Rob Reich, Mehran Sahami, Jeremy M. Weinstein
Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation [ Paper ]
Suraj Nair, Eric Mitchell, Kevin Chen, Brian Ichter, Silvio Savarese, Chelsea Finn
2020
ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT [ Paper ]
Omar Khattab, Matei Zaharia
Generalization through Memorization: Nearest Neighbor Language Models [ Paper ]
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, Mike Lewis
Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models [ Paper ]
Dan Iter, Kelvin Guu, Larry Lansing, Dan Jurafsky
Do Language Embeddings Capture Scales? [ Paper ]
Xikun Zhang, Deepak Ramachandran, Ian Tenney, Yanai Elazar, Dan Roth
DrRepair: Graph-based, Self-Supervised Program Repair from Diagnostic Feedback [ Paper ]
Michihiro Yasunaga, Percy Liang
Language Through a Prism: A Spectral Approach for Multiscale Language Representations [ Paper ]
Alex Tamkin, Dan Jurafsky, Noah Goodman
Investigating Transferability in Pretrained Language Models [ Paper ]
Alex Tamkin, Trisha Singh, Davide Giovanardi, Noah Goodman
Learning Task-Oriented Grasping from Human Activity Datasets [ Paper ]
Mia Kokic, Danica Kragic, Jeannette Bohg
Contextual Embeddings: When Are They Worth It? [ Paper ]
Simran Arora, Avner May, Jian Zhang, Christopher Ré
With Little Power Comes Great Responsibility [ Paper ]
Dallas Card, Peter Henderson, Urvashi Khandelwal, Robin Jia, Kyle Mahowald, Dan Jurafsky
Michihiro Yasunaga, Percy Liang [ Paper ]
Graph-based, Self-Supervised Program Repair from Diagnostic Feedback
Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning [ Paper ]
Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, Joelle Pineau
2019
Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks [ Paper ]
Michelle A. Lee*, Yuke Zhu*, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg, Jeannette Bohg
Self-Supervised Representation Learning via Neighborhood-Relational Encoding [ Paper ]
Mohammad Sabokrou, Mahmood Khalooei, Ehsan Adeli