uncertainty/README.md

metadata

license: apache-2.0
language:
  - en
pipeline_tag: text-generation
library_name: transformers

Uncertainty Quantification Adapters

Model Summary

This family of adapters provides calibrated uncertainty scores for answers generated by each adapter's corresponding base model, quantifying the certainty of the model as a percentage.

• Developer: IBM Research
• License: Apache 2.0

Model Sources

• Paper: Each adapter is tuned to provide certainty scores mimicking the output of a calibrator trained via the method in [Shen et al. ICML 2024] Thermometer: Towards Universal Calibration for Large Language Models

Usage

Intended use

Certainty score definition The model will respond with a certainty percentage, quantized to 10 possible values (i.e. 5%, 15%, 25%,...95%). This percentage is calibrated in the following sense: given a set of answers assigned a certainty score of X%, approximately X% of these answers should be correct. See the eval experiment below for out-of-distribution verification of this behavior.

Certainty score interpretation Certainty scores calibrated as defined above may at times seem biased towards moderate certainty scores for the following reasons. Firstly, as humans we tend to be overconfident in our evaluation of what we know and don't know - in contrast, a calibrated model is less likely to output very high or very low confidence scores, as these imply certainty of correctness or incorrectness. Examples where you might see very low confidence scores might be on answers where the model's response was something to the effect of "I don't know", which is easy to evaluate as not being the correct answer to the question (though it is the appropriate one). Secondly, remember that the model is evaluating itself - correctness/incorrectness that may be obvious to us or to larger models may be less obvious to an 8b model. Finally, teaching a model every fact it knows and doesn't know is not possible, hence it must generalize to questions of wildly varying difficulty (some of which may be trick questions!) and to settings where it has not had its outputs judged. Intuitively, it does this by extrapolating based on related questions it has been evaluated on in training - this is an inherently inexact process and leads to some hedging.

Possible downstream use cases

• Human usage: Certainty scores give human users an indication of when to trust answers from the model (which should be augmented by their own knowledge).
• Model routing/guards: If the model has low certainty (below a chosen threshold), it may be worth sending the request to a larger, more capable model or simply choosing not to show the response to the user.
• RAG: These models are calibrated on document-based question answering datasets, hence it can be applied to giving certainty scores for answers created using RAG. This certainty will be a prediction of overall correctness based on both the documents given and the model's own knowledge (e.g. if the model is correct but the answer is not in the documents, the certainty can still be high).

Important note Certainty is inherently an intrinsic property of a model and its abilitities. These modesl are not intended to predict the certainty of responses generated by any other models besides their corresponding base model. Additionally, certainty scores are distributional quantities, and so will do well on realistic questions in aggregate, but in principle may have surprising scores on individual red-teamed examples.

Usage steps There are two supported usage scenarios.

Scenario 1. Answering a question and obtaining a certainty score proceeds as follows. Given a user query written in the user role:

1. Use the base model to generate a response as normal (via the assistant role).
2. Generate a certainty score with the adapter.
3. The model will respond with an integer 0-9 indicating a certainty percentage quantized with steps of 10% (i.e. 05%, 15%, 25%,...95%).

Scenario 2. Predicting the certainty score from the question only, prior to generating an answer. Given a user query written in the user role:

1. Generate a certainty score with the adapter.
2. The model will respond with an integer 0-9 indicating a certainty percentage quantized with steps of 10% (i.e. 05%, 15%, 25%,...95%).
3. If desired, use the base model to generate a response as normal, with the original input as prompt.

Quickstart Example

First, see information elsewhere in this repo on how to start up a vLLM server hosting the LoRAs and/or aLoRAs. Once this server is started, it can be queried via the OpenAI API. An example for this intrinsic follows.

import os
import openai
import json
import granite_common

QUESTION = "What is IBM?"
RESPONSE = ... # this should be generated by the base model corresponding to the chosen adapter

request = {
  "messages": [
    {
      "content": QUESTION,
      "role": "user"
    },
    {
      "content": RESPONSE,
      "role": "assistant"
    }
  ],
  "model": "uncertainty",
  "temperature": 0.0
}
openai_base_url = ...
openai_api_key = ...
io_yaml_file = "./rag_intrinsics_lib/uncertainty/.../io.yaml"

rewriter = granite_common.IntrinsicsRewriter(config_file=io_yaml_file)
result_processor = granite_common.IntrinsicsResultProcessor(config_file=io_yaml_file)

rewritten_request = rewriter.transform(request)

client = openai.OpenAI(base_url=openai_base_url, api_key=openai_api_key)
chat_completion = client.chat.completions.create(**rewritten_request.model_dump())

transformed_completion = result_processor.transform(chat_completion)

print(transformed_completion.model_dump_json(indent=2))

Training and Evaluation

These models are adapters tuned to provide certainty scores mimicking the output of a calibrator trained via the method in [Shen et al. ICML 2024] Thermometer: Towards Universal Calibration for Large Language Models.

Evaluation: The mean absolute error (MAE) for the adapters in predicting the output of the calibrator are as follows. Here, "X% MAE" means the error in the original output units of % chance (not a further relative error). Recall that the output is quantized to steps of 10%, so errors less than that are on average less than a quantization level.

aLoRA adapters

• Granite 3.3 2B: 4.45% MAE
• Granite 3.3 8B: 3.45% MAE
• GPT-OSS 20B: 0.75% MAE

LoRA adapters

• Granite 3.3 2B: 5.25% MAE
• Granite 3.3 8B: 5.10% MAE
• GPT-OSS 20B: 1.35% MAE

Training Data

The following datasets were used for calibration and/or finetuning.

• BigBench
• MRQA
• newsqa
• trivia_qa
• search_qa
• openbookqa
• web_questions
• smiles-qa
• orca-math
• ARC-Easy
• commonsense_qa
• social_i_qa
• super_glue
• figqa
• riddle_sense
• ag_news
• medmcqa
• dream
• codah
• piqa

Model Card Authors

Kristjan Greenewald