2022

Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution [ Paper ]

Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, Percy Liang
ICLR (International Conference on Learning Representations) 2022

Grounding Predicates through Actions [ Paper ]

Toki Migimatsu, Jeannette Bohg
ICRA (International Conference on Robotics and Automation) 2022

An Explanation of In-context Learning as Implicit Bayesian Inference [ Paper ]

Sang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu Ma
ICLR (International Conference on Learning Representations) 2022

Meta-Learning with Fewer Tasks through Task Interpolation [ Paper ]

Huaxiu Yao, Linjun Zhang, Chelsea Finn
ICLR (International Conference on Learning Representations) 2022

LinkBERT: Pretraining Language Models with Document Links [ Paper ]

Michihiro Yasunaga, Jure Leskovec, Percy Liang
ACL (Annual Meeting of the Association for Computational Linguistics) 2022

MetaMorph: Learning Universal Controllers with Transformers [ Paper ]

Agrim Gupta, Linxi Fan, Surya Ganguli, Li Fei-Fei
ICLR (International Conference on Learning Representations) 2022

GreaseLM: Graph REASoning Enhanced Language Models for Question Answering [ Paper ]

Xikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren, Percy Liang, Christopher D. Manning, Jure Leskovec
ICLR (International Conference on Learning Representations) 2022

Self-supervised Learning is More Robust to Dataset Imbalance [ Paper ]

Hong Liu, Jeff Z. HaoChen, Adrien Gaidon, Tengyu Ma
ICLR (International Conference on Learning Representations) 2022

Large language models can be strong differentially private learners [ Paper ]

Xuechen Li, Florian Tramèr, Percy Liang, Tatsunori Hashimoto
ICLR (International Conference on Learning Representations) 2022

CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities [ Paper ]

Mina Lee, Percy Liang, Qian Yang
CHI (Conference on Human Factors in Computing Systems) 2022

2021

Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data [ Paper ]

Colin Wei, Kendrick Shen, Yining Chen, Tengyu Ma
ICLR (International Conference on Learning Representations) 2021

Relevance-guided Supervision for OpenQA with ColBERT [ Paper ]

Omar Khattab, Christopher Potts, Matei Zaharia
TACL (Transactions of the Association for Computational Linguistics) 2021

Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for Improved Generalization [ Paper ]

Sang Michael Xie, Tengyu Ma, Percy Liang
ICML (International Conference on Machine Learning) 2021

In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness [ Paper ]

Sang Michael Xie*, Ananya Kumar*, Robbie Jones*, Fereshte Khani, Tengyu Ma, Percy Liang
ICLR (International Conference on Learning Representations) 2021

Generative Adversarial Transformers [ Paper ]

Drew A. Hudson, C. Lawrence Zitnick
ICML (International Conference on Machine Learning) 2021

Efficient Large-Scale Language Model Training on GPU Clusters [ Paper ]

Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Anand Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, Matei Zaharia
SC (SuperComputing) 2021

Memory-Efficient Pipeline-Parallel DNN Training [ Paper ]

Deepak Narayanan, Amar Phanishayee, Kaiyu Shi, Xie Chen, Matei Zaharia
ICML (International Conference on Machine Learning) 2021

On the Inductive bias of Masked Language Modeling: From Statistical to Syntactic Dependencies [ Paper ]

Tianyi Zhang, Tatsunori B. Hashimoto
NAACL (North American Chapter of the Association for Computational Linguistics) 2021

Viewmaker Networks: Learning Views for Unsupervised Representation Learning [ Paper ]

Alex Tamkin, Mike Wu, Noah Goodman
ICLR (International Conference on Learning Representations) 2021

Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors [ Paper ]

Michelle A. Lee, Matthew Tan, Yuke Zhu, Jeannette Bohg
ICRA (International Conference on Robotics and Automation) 2021

Concept2Robot: Learning Manipulation Concepts from Instructions and Human Demonstrations [ Paper ]

Lin Shao, Toki Migimatsu, Qiang Zhang, Karen Yang, Jeannette Bohg
RSS (Robotics: Science and Systems) 2021

Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning [ Paper ]

Colin Wei, Sang Michael Xie, Tengyu Ma
NeurIPS (Conference on Neural Information Processing Systems) 2021

DABS: A Domain-Agnostic Benchmark for Self-Supervised Learning [ Paper ]

Alex Tamkin, Vincent Liu, Rongfei Lu, Daniel Fein, Colin Schultz, Noah Goodman
NeurIPS (Conference on Neural Information Processing Systems) 2021

When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset [ Paper ]

Lucia Zheng, Neel Guha, Brandon R. Anderson, Peter Henderson, Daniel E. Ho
ICAIL (International Conference on Artificial Intelligence and Law) 2021

Swords ⚔️: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality [ Paper ]

Mina Lee, Chris Donahue, Robin Jia, Alexander Iyabor, Percy Liang
NAACL (North American Chapter of the Association for Computational Linguistics) 2021

Learning Generalizable Robotic Reward Functions from “In-The-Wild” Human Videos [ Paper ]

Annie S. Chen, Suraj Nair, Chelsea Finn
Robotics: Science and Systems (RSS) 2021

Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss [ Paper ]

Jeff Z. HaoChen, Colin Wei, Adrien Gaidon, Tengyu Ma
NeurIPS (Conference on Neural Information Processing Systems) 2021

Combiner: Full Attention Transformer with Sparse Computation Cost [ Paper ]

Hongyu Ren, Hanjun Dai, Zihang Dai, Mengjiao Yang, Jure Leskovec, Dale Schuurmans, Bo Dai
NeurIPS (Conference on Neural Information Processing Systems) 2021

LM-Critic: Language Models for Unsupervised Grammatical Error Correction [ Paper ]

Michihiro Yasunaga, Jure Leskovec, Percy Liang
EMNLP (Empirical Methods in Natural Language Processing) 2021

Break-It-Fix-It: Unsupervised Learning for Program Repair [ Paper ]

Michihiro Yasunaga, Percy Liang
ICML (International Conference on Machine Learning) 2021

QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering [ Paper ]

Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, Jure Leskovec
NAACL 2021

Longitudinal Self-Supervised Learning [ Paper ]

Qingyu Zhao*, Zixuan Liu*, Ehsan Adeli, Kilian Pohl
Medical Image Analysis (MedIA) 2021

CoCon: Cooperative-Contrastive Learning [ Paper ]

Nishant Rai, Ehsan Adeli, Kuan-Hui Lee, Adrien Gaidon, Juan Carlos Niebles
IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop (CVPRW 2021), Holistic Video Understanding (HVU) 2021

Identification of disease treatment mechanisms through the multiscale interactome [ Paper ]

Camilo Ruiz, Marinka Zitnik, Jure Leskovec
Nature Communications 2021

Codified Audio Language Modeling Learns Useful Representations for Music Information Retrieval [ Paper ]

Rodrigo Castellon, Chris Donahue, Percy Liang
ISMIR (International Society for Music Information Retrieval) 2021

System Error: Where Big Tech Went Wrong and How We Can Reboot [ Paper ]

Rob Reich, Mehran Sahami, Jeremy M. Weinstein
HarperCollins 2021

Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation [ Paper ]

Suraj Nair, Eric Mitchell, Kevin Chen, Brian Ichter, Silvio Savarese, Chelsea Finn
CORL (Conference on Robot Learning) 2021

2020

ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT [ Paper ]

Omar Khattab, Matei Zaharia
SIGIR (Special Interest Group on Information Retrieval) 2020

Generalization through Memorization: Nearest Neighbor Language Models [ Paper ]

Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, Mike Lewis
ICLR (International Conference on Learning Representations) 2020

Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models [ Paper ]

Dan Iter, Kelvin Guu, Larry Lansing, Dan Jurafsky
ACL (Association for Computational Linguistics) 2020

Do Language Embeddings Capture Scales? [ Paper ]

Xikun Zhang, Deepak Ramachandran, Ian Tenney, Yanai Elazar, Dan Roth
EMNLP (Empirical Methods in Natural Language Processing) 2020

Graph Structure of Neural Networks [ Paper ]

Jiaxuan You, Jure Leskovec, Kaiming He, Saining Xie
ICML (International Conference on Machine Learning) 2020

DrRepair: Graph-based, Self-Supervised Program Repair from Diagnostic Feedback [ Paper ]

Michihiro Yasunaga, Percy Liang
ICML (International Conference on Machine Learning) 2020

Sparse GPU Kernels for Deep Learning [ Paper ]

Trevor Gale, Matei Zaharia, Cliff Young, Erich Elsen
SC (SuperComputing) 2020

Language Through a Prism: A Spectral Approach for Multiscale Language Representations [ Paper ]

Alex Tamkin, Dan Jurafsky, Noah Goodman
NeurIPS (Conference on Neural Information Processing Systems) 2020

Investigating Transferability in Pretrained Language Models [ Paper ]

Alex Tamkin, Trisha Singh, Davide Giovanardi, Noah Goodman
Findings of EMNLP 2020

Learning Task-Oriented Grasping from Human Activity Datasets [ Paper ]

Mia Kokic, Danica Kragic, Jeannette Bohg
ICRA (International Conference on Robotics and Automation) 2020

Contextual Embeddings: When Are They Worth It? [ Paper ]

Simran Arora, Avner May, Jian Zhang, Christopher Ré
ACL (Association for Computational Linguistics) 2020

With Little Power Comes Great Responsibility [ Paper ]

Dallas Card, Peter Henderson, Urvashi Khandelwal, Robin Jia, Kyle Mahowald, Dan Jurafsky
EMNLP (Empirical Methods in Natural Language Processing) 2020

Michihiro Yasunaga, Percy Liang [ Paper ]

Graph-based, Self-Supervised Program Repair from Diagnostic Feedback
ICML (International Conference on Machine Learning) 2020

Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning [ Paper ]

Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, Joelle Pineau
JMLR (Journal of Machine Learning Research) 2020

2019

Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks [ Paper ]

Michelle A. Lee*, Yuke Zhu*, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg, Jeannette Bohg
ICRA (International Conference on Robotics and Automation) 2019

Self-Supervised Representation Learning via Neighborhood-Relational Encoding [ Paper ]

Mohammad Sabokrou, Mahmood Khalooei, Ehsan Adeli
ICCV (International Conference on Computer Vision) 2019