HELM Instruct: A Multidimensional Instruction Following Evaluation Framework with Absolute Ratings

Authors: Yian Zhang and Yifan Mai and Josselin Somerville Roberts and Rishi Bommasani and Yann Dubois and Percy Liang


† Yian Zhang conducted this work while he was at Stanford CRFM.

We introduce HELM Instruct, a multidimensional evaluation framework for instruction following with absolute ratings. We use HELM Instruct to compare several instruction-following models as well as some human- and LM- evaluators. All raw results and the leaderboard can be found on this HELMet page.

HELM Instruct

1. Introduction

Instruction-following language models like ChatGPT are impressive: They can compose a business plan, write a poem, and answer questions just about anything. But how should we evaluate the outputs given the open-ended nature of the generation task? Consider the instruction from Anthropic’s HH-RLHF dataset 1 2 and the corresponding response from Cohere’s Command-Xlarge-Beta below:

Instruction: How do you make a good campfire?

Response: There are many ways to make a good campfire, but some of the most important things to consider are the type of wood you use, the size of the fire, and the location of the fire.

How good is this response?

1.1 Background

Here are three well-known ways of evaluating instruction following:

  • Automatic evaluation against references. Metrics such as F1, ROUGE, BERTscore 3 all compare the model-generated output to a small set of reference outputs. F1 and ROUGE are based on lexical overlaps, while BERTscore is based on cosine similarity between the generation and reference embeddings.
  • Human pairwise evaluation. In this case, a human is given a pair of LM generations and asked to select the “better” one and sometimes how much it is better. As an example, Chatbot Arena 4 is an LM battleground where internet users engage in conversations with two LMs before rating them.
  • LM-judged pairwise evaluation. This method is similar to the one above but uses LM evaluators instead of humans. One such example is AlpacaEval,5 which uses LMs such as GPT-4 to score candidate models by comparing them to a reference model (e.g., text-davinci-003).

Table 1: A Comparison of different evaluation methods of instruction following.

Table 1: A Comparison of different evaluation methods of instruction following.

Stepping back, we propose the following three desiderata for instruction following evaluation: open-ended, multidimensional, absolute. We will discuss how the existing work falls short on at least one of these axes, and how our proposed HELM Instruct evaluation captures all three. Table 1 is a comparison among the existing work and HELM Instruct based on the desiderata.

i. Open-ended. While automatic methods computed against references (e.g. ROUGE-L, BLEU, BERTscore) might be effective for tasks such as machine translation or summarization, given the open-ended nature of instruction following, a small set of references is inadequate to represent the full space of possible acceptable generations. Consider an instruction asking for a weekend activity and a reference about doing sports. A response that recommends having dinner with friends will have a very low ROUGE-L and BERTscore.

ii. Multidimensional. Consider the following example and think about which of response A and response B is better:

Instruction: How do you get rid of the smell in your gym shoes?

Response A: One way is to wash them with a mild detergent and then let them air dry. Another way is to use a shoe deodorizer or spray.

Response B: To get rid of the smell in your gym shoes, you can follow these steps:

  • Wash them: Remove any detachable inserts or insoles and wash them separately if possible. Hand wash or machine wash the shoes using a mild detergent and cold water. Avoid using hot water as it can damage the shoes.
  • Dry them properly: After washing, let your shoes air dry completely. Avoid using a dryer or direct heat sources as it can warp or shrink the shoes. Place them in a well-ventilated area, preferably outdoors, and let them dry naturally.
  • Use baking soda: Sprinkle some baking soda inside the shoes to absorb the odor. Leave it overnight or for a few hours, then shake out the excess baking soda. You can also place baking soda in a small fabric pouch or sock and keep it inside the shoes when you’re not wearing them.
  • Freeze them: Put your shoes in a plastic bag and place them in the freezer overnight. The extreme cold temperature helps kill odor-causing bacteria. Take them out in the morning and let them thaw naturally.
  • Use odor absorbers: There are several odor-absorbing products available in the market specifically designed for shoes. These can be in the form of charcoal inserts, shoe deodorizers, or moisture-absorbing sachets. Place them inside your shoes when not in use to eliminate odors.
  • Alternate shoes: Wearing the same pair of shoes every day without giving them time to dry can contribute to the odor. Try alternating between two pairs of shoes to allow them to air out and dry between uses.
  • Maintain cleanliness: Keep your feet clean and dry before wearing your shoes. Use foot powders or antiperspirants to help reduce excessive sweating and odor.

    By following these steps and maintaining good hygiene practices, you should be able to effectively get rid of the smell in your gym shoes.

You may rate response B higher because it gives more ideas and richer details, or prefer response A for its conciseness and efficient information delivery. This illustrates that looking at a response from different angles yields varied assessments of its quality. Such multifaceted nature of response quality cannot be captured by approaches like Chatbot Arena or AlpacaEval that use a single general criterion (e.g. “vote for the better response”). Conversely, our study adopts five different criteria — Helpfulness, Understandability, Completeness, Conciseness, and Harmlessness. More details of these criteria will be discussed in 2.2. Multidimensional evaluation also comes up in other settings like dialogue 6 7 and summarization.8

iii. Absolute. Both Chatbot Arena and AlpacaEval use relative scores, which do not reveal the absolute quality of a model response. For example, at the time we write this post, GPT-4 (0613) has an Elo rating of 1162 on Chatbot Arena. It shows it is more “powerful” than a model with a lower score, say Vicuna-33B, but its actual effectiveness remains unclear. How often does it hallucinate – only 1% of the time, or as much as 50%? And in terms of answer completeness, does it address 90% of a question, or just 40%? Using absolute ratings avoids this issue — the scores show how far a model is from a “perfect” AI model according to the evaluator and are thus more interpretable.

1.2 Overview of HELM Instruct

To capture all these desiderata, we propose the HELM Instruct evaluation framework. Below is a schematic diagram of HELM Instruct. In HELM Instruct, we take an instruction and use a model to generate a response to it. We then feed the response and a criterion, such as conciseness, to a human- or model- based evaluator to produce a numerical score from 1 to 5.

Figure 1: The schematic diagram of HELM Instruct.

Figure 1: The HELM Instruct framework: A model takes in an instruction and generates a response. Given a criterion and a response, the evaluator (which could be model- or human-based) produces a numerical score. The pipeline is implemented and open-sourced using the HELM framework.

Our contributions are as follows:

  1. We create HELM Instruct, a multidimensional evaluation framework of open-ended instruction following with absolute ratings.
  2. Using HELM Instruct, we evaluate multiple instruction-following LMs and compare several evaluators including both LMs (GPT-4 (0314) & Anthropic Claude v1.3) and human crowdworkers on Scale and Amazon Mechanical Turk.

2. Methodology

In this section, we describe the design of HELM Instruct in greater detail. Figure 2 shows the specific scenarios, models, evaluators, and criteria that we use. We conduct a thorough evaluation consisting of the combination of each of the 7 scenarios, 4 models, 4 evaluators, and 5 criteria.

Figure 2: The scenarios, criteria, candidate LMs, and evaluators used in HELM Instruct.

Figure 2: The scenarios, criteria, candidate LMs, and evaluators used in HELM Instruct.

  1. Instruction following scenarios

    We use a total of seven scenarios. Initially, we incorporate the six scenarios from AlpacaEval 1.0: the two Anthropic datasets 1 2 are labelled dialogues released to facilitate research in model helpfulness and harmlessness; Koala Eval 9 and Vicuna Eval 10 are test sets the LM creators use to evaluate their own models; OASST1 11 is a dialogue dataset collected by LAION; Self Instruct represents the set of manually-curated instructions from Wang et al., 2023.12 For the dialogue datasets, we only use the first utterances in our experiments as instructions. Additionally, we incorporate Best ChatGPT Prompts, a set of prompts covering a range of topics such as personal life, health, and marketing. More information about each scenario can be found in Table 2.

    Table 2: Size and example of each scenario.

    Table 2: Size and example of each scenario. For scenarios consisting of more than 100 instances, we randomly sample 100.

  2. Evaluation criteria

    The details of our multidimensional evaluation design can be found in Table 3. We formulate the following multiple-choice evaluation task: For each prompt-response pair, the evaluator answers five multiple choice questions, each corresponding to a criterion — Helpfulness, Understandability, Completeness, Conciseness, and Harmlessness. The questions and options of the multiple choice are shown in Table 3. Each option implies an integer score ranging from 1 to 5.

    Table 3: The evaluation criteria and detailed grading rubrics used for HELM Instruct.

    Table 3: The evaluation criteria and detailed grading rubrics used for HELM Instruct.

  3. Human & LM evaluators

    We use two platforms as human evaluators:

    • Amazon Mechanical Turk: We work closely with a verified group of 16 workers. We start with 19 workers who did a good job in Liu et al., 2023 13 and keep those whose tolerant accuracy* > 0.85 on a 17-example test set we manually create and label. During the annotation process, we continuously monitor annotator behavior and provide personalized feedback. For each response we collect 2 annotations, which in total cost $0.76 plus platform charges. This means on average, if an annotator evaluates one model output every minute, then their hourly pay will be $22.80.

      * Tolerant accuracy measures the percentage of times an annotator’s prediction is +/-1 within our manual labels. E.g. If the gold label is 3 and if an annotator answers 2 or 3 or 4, they are considered correct.

    • Scale: We carefully select and label some test examples, which the Scale AI platform uses to recruit and manage workers for us. Specifically, a worker will only be able to contribute if they pass the initial screening, which is based on the same 17 examples we gave to the MTurk workers. They also need to demonstrate a consistent accuracy on our hidden test cases throughout the process. Scale partners with the Global Living Wage Coalition and analyzes local costs of living to determine a fair compensation. Consistent with MTurk, for each response we collect two annotations on Scale.

    We additionally include two LMs as evaluators: GPT-4 (0314) and Anthropic Claude v1.3.

    We create the following prompt that is used for both the human and LM evaluators:

    The following is an instruction written by a human, and a response to the instruction written by an AI model. Please answer the following questions about the AI model’s response.

    Instruction:
    {instruction}

    Response:
    {response}

    {criterion-specific question}
    Options:
    {criterion-specific options (1-5 points)}

    For the human evaluators, we additionally share some evaluation guidelines with a short task description and a list of grading examples to help them better understand the criteria. For cost considerations, we do not provide the LM evaluators with these detailed guidelines. Future work could explore how the level of detail impacts LM-based evaluation.

We apply the above framework to evaluate 4 models: GPT-4 (0314), GPT-3.5 Turbo (0613), Anthropic Claude v1.3, and Cohere-Command-Xlarge-Beta. All experiment procedures are implemented and open-sourced via the HELM framework.14

3. Results

3.1 Instruction following performance

Below are bar charts showing the multidimensional performance of different models. Each bar chart shows the evaluation results based on one of the five criteria and macro-averaged over all scenarios. Each bar in the plot represents the score of a candidate model (specified by color and texture) rated by the evaluator specified on the x axis. Every candidate model is rated by every evaluator, resulting in 4 * 4 = 16 bars in each bar chart.

Figure 3: Ratings of model responses from different evaluators with different criteria.

Figure 3: Ratings of model responses from different evaluators based on different criteria. The numbers are macro averages over all 7 scenarios. The x axis is the evaluator; the y axis is the rating.

Some of our main findings about the LM candidates are:

GPT-4 (0314) is overall the best. Figure 3 shows average scores over scenarios, and GPT-4 (0314) shows the highest score most frequently (10 out of 20 bar groups in the figure). Similarly, if we average over the criteria instead of scenarios, out of 28 evaluation settings (scenario X evaluator), GPT-4 (0314) has the highest score in 16 of them.

Anthropic Claude v1.3 is most understandable and harmless. GPT-4 (0314) is not the strongest on every aspect: When averaged over all scenarios and evaluators, Anthropic Claude v1.3 has the highest score on Understandability and Harmlessness. It is also the most frequent winner on these two aspects in Figure 3, although it often wins by a narrow margin. On the other hand, GPT-4 (0314)’s biggest strength seems to be in helpfulness and completeness.

Room for improvement in helpfulness and conciseness. Having absolute ratings allows us to understand how far a model’s ability is from the ceiling. Although all evaluators agree that many models have been close to “perfect” on aspects like understandability, the models show the lowest scores on helpfulness and conciseness. It is also interesting to find that LM evaluators are stricter than humans on these two criteria.

3.2 Comparing different evaluators

In this experiment, we include two human evaluators (Scale, MTurk) and two LM-based evaluators (Anthropic Claude v1.3, GPT-4 (0314)). We calculate the instance-level Pearson correlation coefficient between every pair of evaluators, as shown in Table 4 below. Each cell represents the correlation coefficient between the two evaluators it connects. A larger number is marked with a darker background color in the table and indicates that the corresponding evaluators tend to give more similar judgments.

Table 4: Instance-level correlations between different evaluators averaged over all scenarios.

Table 4: Instance-level Pearson correlations between different evaluators on all scenarios. "GPT4" represents the evaluator based on GPT-4 (0314). "Claude" represents the evaluator based on Anthropic Claude v1.3.

From the results in Table 4, we find that there is considerable variation across evaluators. The lower right corner of the table shows that with all criteria considered, the Pearson correlation coefficients between evaluator pairs fall in the range between 0.48 and 0.72. Moreover, the level of disagreement varies across the criteria. The evaluators disagree the most on conciseness and understandability, and the correlation coefficients on other aspects are relatively larger. We encourage future work to study the factors that affect the ratings of different evaluators and result in the disagreements.

We also find that GPT-4 (0314) is better correlated with human evaluators than Anthropic Claude v1.3. As shown in Table 4, on the instance level, the GPT-4 (0314) evaluator’s Pearson correlation coefficients with the MTurk evaluator and the Scale evaluator are 0.66 and 0.56 respectively, while the Anthropic Claude v1.3 evaluator shows 0.50 and 0.48. In addition, we average the per-instance ratings over all the instances (examples) in each scenario (dataset) to get the per-scenario average scores and then calculate the Pearson correlation coefficients between them. The scenario-level coefficients between GPT-4 (0314) and the human evaluators are 0.85 (MTurk) and 0.90 (Scale). This suggests that GPT-4 (0314) can be used as a pretty decent approximator of human evaluators when results are aggregated over hundreds of examples but its judgment of each individual example is less reliable.

Additionally, the LM evaluators have different strengths for different criteria. As revealed in Table 4, out of the 5 aspects, the LM evaluators are best correlated with humans on evaluating harmlessness and completeness and worse at evaluating understandability and conciseness. When evaluating understandability, the LM evaluators seem to be too lenient to differentiate the candidates. They rate 5/5 for understandability in a majority of cases as shown in Figure 3, while human evaluators give lower scores more often. On the other hand, in terms of conciseness evaluation, both LM evaluators are harsher than humans.

To better understand the behavior of each evaluator, we calculate the means and standard deviations of their ratings based on different criteria.

Table 5: Means and standard deviations of the ratings given by each evaluator to different instances.

Table 5: Means and standard deviations of the ratings given by each evaluator to different instances. An instance is a pair of instruction and response, and we consider instances from all seven scenarios. The row headers show the evaluation criteria adopted. "GPT-4" represents the evaluator based on GPT-4 (0314). "Claude v1.3" represents the evaluator based on Anthropic Claude v1.3.

Similar to in AlpacaEval,5 we also find that human ratings have a higher variance than LM ones. In Table 5, the LM evaluators’ standard deviations are always lower than those of their human counterparts except in completeness evaluation. This is likely because each human evaluator consists of multiple humans and is thus noisier. Also, Table 5 again shows that LM evaluators are too lenient for understandability evaluation.

Explore it yourself

We release all the raw model responses and ratings of different evaluators. Browse the results at https://crfm.stanford.edu/helm/instruct/v1.0.0/#/runs.

Acknowledgements

We thank Scale for their support and providing credits. We thank Nelson Liu for advice on MTurk worker selection and collaboration. We thank the MTurk workers for their contributions as evaluators. This work was supported in part by the AI2050 program at Schmidt Futures (Grant G-22-63429). We thank OpenAI, Anthropic, and Cohere for the model credits.

Citation

@misc{helm-instruct,
  author = {Yian Zhang and Yifan Mai and Josselin Somerville Roberts and Rishi Bommasani and Yann Dubois and Percy Liang},
  title = {HELM Instruct: A Multidimensional Instruction Following Evaluation Framework with Absolute Ratings},
  month = {February},
  year = {2024},
  url = {https://crfm.stanford.edu/2024/02/18/helm-instruct.html},
}

References

  1. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, and Jared Kaplan. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback.  2

  2. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022. Red teaming language models with language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3419–3448, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.  2

  3. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020b. BERTScore: Evaluating text generation with bert. In International Conference on Learning Representations. 

  4. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-judge with MT-bench and chatbot arena. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track. 

  5. Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori Hashimoto. 2023. AlpacaFarm: A simulation framework for methods that learn from human feedback. In Thirty-seventh Conference on Neural Information Processing Systems.  2

  6. Mina Lee, Megha Srivastava, Amelia Hardy, John Thickstun, Esin Durmus, Ashwin Paranjape, Ines Gerard-Ursin, Xiang Lisa Li, Faisal Ladhak, Frieda Rong, Rose E Wang, Minae Kwon, Joon Sung Park, Hancheng Cao, Tony Lee, Rishi Bommasani, Michael S. Bernstein, and Percy Liang. 2023. Evaluating human-language model interaction. Transactions on Machine Learning Research. 

  7. Shikib Mehri and Maxine Eskenazi. 2020. USR: An unsupervised and reference free evaluation metric for dialog generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 681–707, Online. Association for Computational Linguistics. 

  8. Alexander R. Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, Dragomir Radev; SummEval: Re-evaluating Summarization Evaluation. Transactions of the Association for Computational Linguistics 2021; 9 391–409. doi: https://doi.org/10.1162/tacl_a_00373 . 

  9. Xinyang Geng, Arnav Gudibande, Hao Liu, Eric Wallace, Pieter Abbeel, Sergey Levine and Dawn Song. 2023. Koala: A Dialogue Model for Academic Research. Berkeley Artificial Intelligence Research. https://bair.berkeley.edu/blog/2023/04/03/koala/ . 

  10. The Vicuna Team. 2023. Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality. LMSYS Org. https://lmsys.org/blog/2023-03-30-vicuna/ . 

  11. Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Minh Nguyen, Oliver Stanley, Richárd Nagyfi, Shahul ES, Sameer Suri, David Alexandrovich Glushkov, Arnav Varma Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, Alexander Julian Mattick. 2023. OpenAssistant Conversations - Democratizing Large Language Model Alignment. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track. 

  12. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13484–13508, Toronto, Canada. Association for Computational Linguistics. 

  13. Nelson Liu, Tianyi Zhang, and Percy Liang. 2023. Evaluating verifiability in generative search engines. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7001–7025, Singapore. Association for Computational Linguistics. 

  14. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Alexander Cosgrove, Christopher D Manning, Christopher Re, Diana Acosta-Navas, Drew Arad Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue WANG, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Andrew Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda. 2023. Holistic evaluation of language models. Transactions on Machine Learning Research. Featured Certification, Expert Certification.