Skip to main content
POST
/
v1
/
evaluation
/
evaluate
Evaluate a prompt using Qualifire's detectors
curl --request POST \
  --url https://api.qualifire.ai/api/v1/evaluation/evaluate \
  --header 'Content-Type: application/json' \
  --header 'X-Qualifire-API-Key: <api-key>' \
  --data '
{
  "input": "<string>",
  "output": "<string>",
  "messages": [
    {
      "role": "system",
      "content": "<string>",
      "tool_calls": [
        {
          "function": {
            "name": "<string>",
            "arguments": {}
          },
          "id": "<string>",
          "type": "<string>"
        }
      ]
    }
  ],
  "available_tools": [
    {
      "name": "<string>",
      "description": "<string>",
      "parameters": {}
    }
  ],
  "hallucinations_check": true,
  "hallucinations_mode": "speed",
  "grounding_check": true,
  "grounding_mode": "speed",
  "grounding_multi_turn_mode": true,
  "prompt_injections": true,
  "pii_check": true,
  "content_moderation_check": true,
  "tool_use_quality_check": true,
  "tuq_mode": "speed",
  "assertions": [
    "<string>"
  ],
  "assertions_mode": "speed",
  "policy_target": "input",
  "policy_multi_turn_mode": true,
  "policy_include_tools": true,
  "allowed_topics": [
    "<string>"
  ],
  "topic_scoping_mode": "speed",
  "topic_scoping_target": "input",
  "topic_scoping_multi_turn_mode": true,
  "metadata": {}
}
'
{
  "status": "success",
  "score": 123,
  "evaluationResults": [
    {
      "type": "<string>",
      "results": [
        {
          "name": "<string>",
          "score": 123,
          "label": "<string>",
          "confidence_score": 123,
          "reason": "<string>",
          "flagged": true,
          "quote": "<string>",
          "data": "<string>"
        }
      ]
    }
  ]
}

Authorizations

​
X-Qualifire-API-Key
string
header
required

Body

application/json

Evaluation request

At least one of input, output, or messages must be provided.

​
input
string

The input text (e.g., user prompt)

​
output
string

The output text (e.g., LLM response)

​
messages
object[]

Conversation messages for multi-turn evaluation

​
available_tools
object[]

List of tools available to the LLM. Required when tool_use_quality_check is enabled.

​
hallucinations_check
boolean

Check for factual inaccuracies or hallucinations

​
hallucinations_mode
enum<string>

Quality/speed tradeoff for hallucination checks

Available options:
speed,
balanced,
quality
​
grounding_check
boolean

Check if the output is grounded in the provided input/context

​
grounding_mode
enum<string>

Quality/speed tradeoff for grounding checks

Available options:
speed,
balanced,
quality
​
grounding_multi_turn_mode
boolean

Use full conversation context for grounding checks

​
prompt_injections
boolean

Check for prompt injection attempts

​
pii_check
boolean

Check for personally identifiable information

​
content_moderation_check
boolean

Check for harmful content (dangerous content, harassment, hate speech, sexual content)

​
tool_use_quality_check
boolean

Check the quality of tool selection and usage. Requires messages and available_tools.

​
tuq_mode
enum<string>

Quality/speed tradeoff for tool use quality checks

Available options:
speed,
balanced,
quality
​
assertions
string[]

Custom policy assertions to evaluate against (e.g., "don't give medical advice")

​
assertions_mode
enum<string>

Quality/speed tradeoff for assertion checks

Available options:
speed,
balanced,
quality
​
policy_target
enum<string>

Whether policy assertions apply to input, output, or both

Available options:
input,
output,
both
​
policy_multi_turn_mode
boolean

Use full conversation context for policy checks

​
policy_include_tools
boolean

Include tool definitions and tool calls in policy assertion context. When true, assertions can reference available_tools and tool_calls from messages.

​
allowed_topics
string[]

List of allowed topics for topic scoping checks

​
topic_scoping_mode
enum<string>

Quality/speed tradeoff for topic scoping checks

Available options:
speed,
balanced,
quality
​
topic_scoping_target
enum<string>

Whether topic scoping applies to input, output, or both

Available options:
input,
output,
both
​
topic_scoping_multi_turn_mode
boolean

Use full conversation context for topic scoping checks

​
metadata
object

Optional key-value pairs (string values only) to attach to the evaluation invocation

Response

OK

​
status
enum<string>
required

Overall evaluation status

Available options:
success,
warning,
failed
​
score
number | null
required

Overall evaluation score (0-100)

​
evaluationResults
object[]
required