† Yian Zhang conducted this work while he was at Stanford CRFM.

1. Introduction

Instruction-following language models like ChatGPT are impressive: They can compose a business plan, write a poem, and answer questions just about anything. But how should we evaluate the outputs given the open-ended nature of the generation task? Consider the instruction from Anthropic’s HH-RLHF dataset ^{1 2} and the corresponding response from Cohere’s Command-Xlarge-Beta below:

Instruction: How do you make a good campfire?

Response: There are many ways to make a good campfire, but some of the most important things to consider are the type of wood you use, the size of the fire, and the location of the fire.

How good is this response?

1.1 Background

Here are three well-known ways of evaluating instruction following:

• Automatic evaluation against references. Metrics such as F1, ROUGE, BERTscore ³ all compare the model-generated output to a small set of reference outputs. F1 and ROUGE are based on lexical overlaps, while BERTscore is based on cosine similarity between the generation and reference embeddings.
• Human pairwise evaluation. In this case, a human is given a pair of LM generations and asked to select the “better” one and sometimes how much it is better. As an example, Chatbot Arena ⁴ is an LM battleground where internet users engage in conversations with two LMs before rating them.
• LM-judged pairwise evaluation. This method is similar to the one above but uses LM evaluators instead of humans. One such example is AlpacaEval,⁵ which uses LMs such as GPT-4 to score candidate models by comparing them to a reference model (e.g., text-davinci-003).

Table 1: A Comparison of different evaluation methods of instruction following.

Stepping back, we propose the following three desiderata for instruction following evaluation: open-ended, multidimensional, absolute. We will discuss how the existing work falls short on at least one of these axes, and how our proposed HELM Instruct evaluation captures all three. Table 1 is a comparison among the existing work and HELM Instruct based on the desiderata.

i. Open-ended. While automatic methods computed against references (e.g. ROUGE-L, BLEU, BERTscore) might be effective for tasks such as machine translation or summarization, given the open-ended nature of instruction following, a small set of references is inadequate to represent the full space of possible acceptable generations. Consider an instruction asking for a weekend activity and a reference about doing sports. A response that recommends having dinner with friends will have a very low ROUGE-L and BERTscore.

ii. Multidimensional. Consider the following example and think about which of response A and response B is better:

Instruction: How do you get rid of the smell in your gym shoes?

Response A: One way is to wash them with a mild detergent and then let them air dry. Another way is to use a shoe deodorizer or spray.

Response B: To get rid of the smell in your gym shoes, you can follow these steps:

• Wash them: Remove any detachable inserts or insoles and wash them separately if possible. Hand wash or machine wash the shoes using a mild detergent and cold water. Avoid using hot water as it can damage the shoes.
• Dry them properly: After washing, let your shoes air dry completely. Avoid using a dryer or direct heat sources as it can warp or shrink the shoes. Place them in a well-ventilated area, preferably outdoors, and let them dry naturally.
• Use baking soda: Sprinkle some baking soda inside the shoes to absorb the odor. Leave it overnight or for a few hours, then shake out the excess baking soda. You can also place baking soda in a small fabric pouch or sock and keep it inside the shoes when you’re not wearing them.
• Freeze them: Put your shoes in a plastic bag and place them in the freezer overnight. The extreme cold temperature helps kill odor-causing bacteria. Take them out in the morning and let them thaw naturally.
• Use odor absorbers: There are several odor-absorbing products available in the market specifically designed for shoes. These can be in the form of charcoal inserts, shoe deodorizers, or moisture-absorbing sachets. Place them inside your shoes when not in use to eliminate odors.
• Alternate shoes: Wearing the same pair of shoes every day without giving them time to dry can contribute to the odor. Try alternating between two pairs of shoes to allow them to air out and dry between uses.

• Maintain cleanliness: Keep your feet clean and dry before wearing your shoes. Use foot powders or antiperspirants to help reduce excessive sweating and odor.

  By following these steps and maintaining good hygiene practices, you should be able to effectively get rid of the smell in your gym shoes.
  

You may rate response B higher because it gives more ideas and richer details, or prefer response A for its conciseness and efficient information delivery. This illustrates that looking at a response from different angles yields varied assessments of its quality. Such multifaceted nature of response quality cannot be captured by approaches like Chatbot Arena or AlpacaEval that use a single general criterion (e.g. “vote for the better response”). Conversely, our study adopts five different criteria — Helpfulness, Understandability, Completeness, Conciseness, and Harmlessness. More details of these criteria will be discussed in 2.2. Multidimensional evaluation also comes up in other settings like dialogue ^{6 7} and summarization.⁸

iii. Absolute. Both Chatbot Arena and AlpacaEval use relative scores, which do not reveal the absolute quality of a model response. For example, at the time we write this post, GPT-4 (0613) has an Elo rating of 1162 on Chatbot Arena. It shows it is more “powerful” than a model with a lower score, say Vicuna-33B, but its actual effectiveness remains unclear. How often does it hallucinate – only 1% of the time, or as much as 50%? And in terms of answer completeness, does it address 90% of a question, or just 40%? Using absolute ratings avoids this issue — the scores show how far a model is from a “perfect” AI model according to the evaluator and are thus more interpretable.

1.2 Overview of HELM Instruct

To capture all these desiderata, we propose the HELM Instruct evaluation framework. Below is a schematic diagram of HELM Instruct. In HELM Instruct, we take an instruction and use a model to generate a response to it. We then feed the response and a criterion, such as conciseness, to a human- or model- based evaluator to produce a numerical score from 1 to 5.

Figure 1: The HELM Instruct framework: A model takes in an instruction and generates a response. Given a criterion and a response, the evaluator (which could be model- or human-based) produces a numerical score. The pipeline is implemented and open-sourced using the HELM framework.

Our contributions are as follows:

1. We create HELM Instruct, a multidimensional evaluation framework of open-ended instruction following with absolute ratings.
2. Using HELM Instruct, we evaluate multiple instruction-following LMs and compare several evaluators including both LMs (GPT-4 (0314) & Anthropic Claude v1.3) and human crowdworkers on Scale and Amazon Mechanical Turk.

2. Methodology

In this section, we describe the design of HELM Instruct in greater detail. Figure 2 shows the specific scenarios, models, evaluators, and criteria that we use. We conduct a thorough evaluation consisting of the combination of each of the 7 scenarios, 4 models, 4 evaluators, and 5 criteria.

Figure 2: The scenarios, criteria, candidate LMs, and evaluators used in HELM Instruct.

1. Instruction following scenarios

   We use a total of seven scenarios. Initially, we incorporate the six scenarios from AlpacaEval 1.0: the two Anthropic datasets ^{1 2} are labelled dialogues released to facilitate research in model helpfulness and harmlessness; Koala Eval ⁹ and Vicuna Eval ¹⁰ are test sets the LM creators use to evaluate their own models; OASST1 ¹¹ is a dialogue dataset collected by LAION; Self Instruct represents the set of manually-curated instructions from Wang et al., 2023.¹² For the dialogue datasets, we only use the first utterances in our experiments as instructions. Additionally, we incorporate Best ChatGPT Prompts, a set of prompts covering a range of topics such as personal life, health, and marketing. More information about each scenario can be found in Table 2.

   

   Table 2: Size and example of each scenario. For scenarios consisting of more than 100 instances, we randomly sample 100.

2. Evaluation criteria

   The details of our multidimensional evaluation design can be found in Table 3. We formulate the following multiple-choice evaluation task: For each prompt-response pair, the evaluator answers five multiple choice questions, each corresponding to a criterion — Helpfulness, Understandability, Completeness, Conciseness, and Harmlessness. The questions and options of the multiple choice are shown in Table 3. Each option implies an integer score ranging from 1 to 5.

   

   Table 3: The evaluation criteria and detailed grading rubrics used for HELM Instruct.

3. Human & LM evaluators

   We use two platforms as human evaluators:

   • Amazon Mechanical Turk: We work closely with a verified group of 16 workers. We start with 19 workers who did a good job in Liu et al., 2023 ¹³ and keep those whose tolerant accuracy* > 0.85 on a 17-example test set we manually create and label. During the annotation process, we continuously monitor annotator behavior and provide personalized feedback. For each response we collect 2 annotations, which in total cost $0.76 plus platform charges. This means on average, if an annotator evaluates one model output every minute, then their hourly pay will be $22.80.

     * Tolerant accuracy measures the percentage of times an annotator’s prediction is +/-1 within our manual labels. E.g. If the gold label is 3 and if an annotator answers 2 or 3 or 4, they are considered correct.

   • Scale: We carefully select and label some test examples, which the Scale AI platform uses to recruit and manage workers for us. Specifically, a worker will only be able to contribute if they pass the initial screening, which is based on the same 17 examples we gave to the MTurk workers. They also need to demonstrate a consistent accuracy on our hidden test cases throughout the process. Scale partners with the Global Living Wage Coalition and analyzes local costs of living to determine a fair compensation. Consistent with MTurk, for each response we collect two annotations on Scale.

   We additionally include two LMs as evaluators: GPT-4 (0314) and Anthropic Claude v1.3.

   We create the following prompt that is used for both the human and LM evaluators:

   The following is an instruction written by a human, and a response to the instruction written by an AI model. Please answer the following questions about the AI model’s response.

   Instruction:
   {instruction}

   Response:
   {response}

   {criterion-specific question}
   Options:
   {criterion-specific options (1-5 points)}

   For the human evaluators, we additionally share some evaluation guidelines with a short task description and a list of grading examples to help them better understand the criteria. For cost considerations, we do not provide the LM evaluators with these detailed guidelines. Future work could explore how the level of detail impacts LM-based evaluation.

We apply the above framework to evaluate 4 models: GPT-4 (0314), GPT-3.5 Turbo (0613), Anthropic Claude v1.3, and Cohere-Command-Xlarge-Beta. All experiment procedures are implemented and open-sourced via the HELM framework.¹⁴

3. Results

3.1 Instruction following performance

Below are bar charts showing the multidimensional performance of different models. Each bar chart shows the evaluation results based on one of the five criteria and macro-averaged over all scenarios. Each bar in the plot represents the score of a candidate model (specified by color and texture) rated by the evaluator specified on the x axis. Every candidate model is rated by every evaluator, resulting in 4 * 4 = 16 bars in each bar chart.

Figure 3: Ratings of model responses from different evaluators based on different criteria. The numbers are macro averages over all 7 scenarios. The x axis is the evaluator; the y axis is the rating.

Some of our main findings about the LM candidates are:

GPT-4 (0314) is overall the best. Figure 3 shows average scores over scenarios, and GPT-4 (0314) shows the highest score most frequently (10 out of 20 bar groups in the figure). Similarly, if we average over the criteria instead of scenarios, out of 28 evaluation settings (scenario X evaluator), GPT-4 (0314) has the highest score in 16 of them.

Anthropic Claude v1.3 is most understandable and harmless. GPT-4 (0314) is not the strongest on every aspect: When averaged over all scenarios and evaluators, Anthropic Claude v1.3 has the highest score on Understandability and Harmlessness. It is also the most frequent winner on these two aspects in Figure 3, although it often wins by a narrow margin. On the other hand, GPT-4 (0314)’s biggest strength seems to be in helpfulness and completeness.

Room for improvement in helpfulness and conciseness. Having absolute ratings allows us to understand how far a model’s ability is from the ceiling. Although all evaluators agree that many models have been close to “perfect” on aspects like understandability, the models show the lowest scores on helpfulness and conciseness. It is also interesting to find that LM evaluators are stricter than humans on these two criteria.

3.2 Comparing different evaluators

In this experiment, we include two human evaluators (Scale, MTurk) and two LM-based evaluators (Anthropic Claude v1.3, GPT-4 (0314)). We calculate the instance-level Pearson correlation coefficient between every pair of evaluators, as shown in Table 4 below. Each cell represents the correlation coefficient between the two evaluators it connects. A larger number is marked with a darker background color in the table and indicates that the corresponding evaluators tend to give more similar judgments.

Table 4: Instance-level Pearson correlations between different evaluators on all scenarios. "GPT4" represents the evaluator based on GPT-4 (0314). "Claude" represents the evaluator based on Anthropic Claude v1.3.

From the results in Table 4, we find that there is considerable variation across evaluators. The lower right corner of the table shows that with all criteria considered, the Pearson correlation coefficients between evaluator pairs fall in the range between 0.48 and 0.72. Moreover, the level of disagreement varies across the criteria. The evaluators disagree the most on conciseness and understandability, and the correlation coefficients on other aspects are relatively larger. We encourage future work to study the factors that affect the ratings of different evaluators and result in the disagreements.

We also find that GPT-4 (0314) is better correlated with human evaluators than Anthropic Claude v1.3. As shown in Table 4, on the instance level, the GPT-4 (0314) evaluator’s Pearson correlation coefficients with the MTurk evaluator and the Scale evaluator are 0.66 and 0.56 respectively, while the Anthropic Claude v1.3 evaluator shows 0.50 and 0.48. In addition, we average the per-instance ratings over all the instances (examples) in each scenario (dataset) to get the per-scenario average scores and then calculate the Pearson correlation coefficients between them. The scenario-level coefficients between GPT-4 (0314) and the human evaluators are 0.85 (MTurk) and 0.90 (Scale). This suggests that GPT-4 (0314) can be used as a pretty decent approximator of human evaluators when results are aggregated over hundreds of examples but its judgment of each individual example is less reliable.

Additionally, the LM evaluators have different strengths for different criteria. As revealed in Table 4, out of the 5 aspects, the LM evaluators are best correlated with humans on evaluating harmlessness and completeness and worse at evaluating understandability and conciseness. When evaluating understandability, the LM evaluators seem to be too lenient to differentiate the candidates. They rate 5/5 for understandability in a majority of cases as shown in Figure 3, while human evaluators give lower scores more often. On the other hand, in terms of conciseness evaluation, both LM evaluators are harsher than humans.

To better understand the behavior of each evaluator, we calculate the means and standard deviations of their ratings based on different criteria.

Table 5: Means and standard deviations of the ratings given by each evaluator to different instances. An instance is a pair of instruction and response, and we consider instances from all seven scenarios. The row headers show the evaluation criteria adopted. "GPT-4" represents the evaluator based on GPT-4 (0314). "Claude v1.3" represents the evaluator based on Anthropic Claude v1.3.

Similar to in AlpacaEval,⁵ we also find that human ratings have a higher variance than LM ones. In Table 5, the LM evaluators’ standard deviations are always lower than those of their human counterparts except in completeness evaluation. This is likely because each human evaluator consists of multiple humans and is thus noisier. Also, Table 5 again shows that LM evaluators are too lenient for understandability evaluation.

Explore it yourself

We release all the raw model responses and ratings of different evaluators. Browse the results at https://crfm.stanford.edu/helm/instruct/v1.0.0/#/runs.

Acknowledgements

We thank Scale for their support and providing credits. We thank Nelson Liu for advice on MTurk worker selection and collaboration. We thank the MTurk workers for their contributions as evaluators. This work was supported in part by the AI2050 program at Schmidt Futures (Grant G-22-63429). We thank OpenAI, Anthropic, and Cohere for the model credits.

Citation

@misc{helm-instruct,
  author = {Yian Zhang and Yifan Mai and Josselin Somerville Roberts and Rishi Bommasani and Yann Dubois and Percy Liang},
  title = {HELM Instruct: A Multidimensional Instruction Following Evaluation Framework with Absolute Ratings},
  month = {February},
  year = {2024},
  url = {https://crfm.stanford.edu/2024/02/18/helm-instruct.html},
}

References