Post-Evaluation Hooks

Post-evaluation hooks run after all evaluators complete. Use them for result export, notifications, and custom analysis.

Key Characteristics

• Run in parallel (not sequential)
• Never fail the main evaluation
• Have read-only access to results
• Execute after all evaluators complete

Available Hooks

Hook	Purpose
database	Export to JSON/JSONL files
webhook	POST to HTTP endpoints
script	Custom shell scripts

Database Hook

Export results to structured storage:

post_evaluation:
  - name: database
    config:
      type: json-file
      output_path: ./results-history.jsonl
      include_full_bundle: true
      append: true

Configuration

Option	Description	Default
type	Storage type (json-file)	Required
output_path	File path for output	Required
include_full_bundle	Include complete results	false
append	Append to existing file	false

Example: Build History

post_evaluation:
  - name: database
    config:
      type: json-file
      output_path: ./evaluation-history.jsonl
      append: true

Each run appends a JSON line:

{"timestamp":"2024-11-15T10:00:00Z","status":"passed","suite":"auth-eval"}
{"timestamp":"2024-11-15T11:00:00Z","status":"passed","suite":"auth-eval"}
{"timestamp":"2024-11-15T12:00:00Z","status":"failed","suite":"auth-eval"}

Webhook Hook

POST results to HTTP endpoints:

post_evaluation:
  - name: webhook
    config:
      url: ${SLACK_WEBHOOK_URL}
      method: POST
      headers:
        Content-Type: "application/json"
      retry_on_failure: true

Configuration

Option	Description	Default
url	HTTP endpoint URL	Required
method	HTTP method	POST
headers	Custom headers	{}
retry_on_failure	Retry on error	false
timeout_ms	Request timeout	30000

Example: Slack Notification

post_evaluation:
  - name: webhook
    config:
      url: ${SLACK_WEBHOOK_URL}
      method: POST
      headers:
        Content-Type: "application/json"

youBencha sends a JSON payload:

{
  "text": "Evaluation Complete",
  "attachments": [{
    "color": "#36a64f",
    "fields": [
      { "title": "Status", "value": "passed", "short": true },
      { "title": "Suite", "value": "auth-evaluation", "short": true }
    ]
  }]
}

Example: Custom API

post_evaluation:
  - name: webhook
    config:
      url: https://api.example.com/evaluations
      method: POST
      headers:
        Authorization: "Bearer ${API_TOKEN}"
        Content-Type: "application/json"
      retry_on_failure: true

Script Hook

Run custom analysis scripts:

post_evaluation:
  - name: script
    config:
      command: ./scripts/analyze-results.sh
      args: ["${RESULTS_PATH}"]
      env:
        SLACK_WEBHOOK: "${SLACK_WEBHOOK_URL}"

Configuration

Option	Description	Default
command	Script/command to run	Required
args	Command arguments	[]
env	Environment variables	{}
timeout_ms	Execution timeout	60000

Example: Custom Slack Message

scripts/notify-slack.sh

#!/bin/bash
RESULTS=$1

STATUS=$(jq -r '.summary.overall_status' "$RESULTS")
PASSED=$(jq -r '.summary.passed' "$RESULTS")
FAILED=$(jq -r '.summary.failed' "$RESULTS")

COLOR="good"
if [ "$STATUS" != "passed" ]; then
  COLOR="danger"
fi

curl -X POST "$SLACK_WEBHOOK" -H "Content-Type: application/json" -d "{
  \"attachments\": [{
    \"color\": \"$COLOR\",
    \"title\": \"youBencha Evaluation: $STATUS\",
    \"fields\": [
      {\"title\": \"Passed\", \"value\": \"$PASSED\", \"short\": true},
      {\"title\": \"Failed\", \"value\": \"$FAILED\", \"short\": true}
    ]
  }]
}"

post_evaluation:
  - name: script
    config:
      command: ./scripts/notify-slack.sh
      args: ["${RESULTS_PATH}"]
      env:
        SLACK_WEBHOOK: "${SLACK_WEBHOOK_URL}"

Environment Variables

Post-evaluation hooks have access to:

Variable	Description
WORKSPACE_DIR	Workspace directory path
REPO_DIR	Repository directory
ARTIFACTS_DIR	Artifacts directory
RESULTS_PATH	Path to results.json
TEST_CASE_NAME	Test case name

Multiple Hooks

All post-evaluation hooks run in parallel:

post_evaluation:
  # These all run at the same time
  - name: database
    config:
      type: json-file
      output_path: ./history.jsonl
      append: true

  - name: webhook
    config:
      url: ${SLACK_WEBHOOK_URL}

  - name: script
    config:
      command: ./scripts/upload-to-s3.sh
      args: ["${RESULTS_PATH}"]

Failure Handling

Post-evaluation hooks never fail the main evaluation:

• Hook failures are logged but don’t affect overall status
• Other hooks continue even if one fails
• Results are still saved regardless of hook failures

Best Practices

1. Use environment variables for sensitive data
2. Set appropriate timeouts for external calls
3. Enable retry for unreliable endpoints
4. Keep scripts idempotent for safe retries
5. Log hook actions for debugging

Related

• Hooks Overview
• Pre-Execution Hooks
• Slack Notifications Example
• CI/CD Integration