Prompt Files

youBencha supports loading prompts from external files for better organization, reusability, and version control.

Why Use Prompt Files?

Benefit	Description
Organization	Keep complex prompts separate from configuration
Reusability	Share prompts across multiple suites
Version Control	Track prompt changes with meaningful diffs
Editor Support	Use markdown editors with syntax highlighting
Collaboration	Non-developers can edit prompts without touching YAML

Using Prompt Files

Agent Prompts

Load the agent’s task prompt from a file:

suite.yaml

agent:
  type: copilot-cli
  config:
    prompt_file: ./prompts/add-auth.md

prompts/add-auth.md

# Add JWT Authentication

Implement JWT-based authentication for the API:

1. Add a login endpoint at POST /api/auth/login
2. Generate JWT tokens with 24-hour expiration
3. Add middleware to protect /api/users/* routes
4. Handle token refresh at POST /api/auth/refresh

## Requirements

- Use the jsonwebtoken package
- Store JWT secret in environment variable JWT_SECRET
- Return appropriate HTTP status codes (401, 403)

Evaluator Prompts

Load custom instructions for the agentic-judge evaluator:

suite.yaml

evaluators:
  - name: agentic-judge
    config:
      type: copilot-cli
      prompt_file: ./prompts/strict-eval.txt
      assertions:
        security: "Authentication is secure. Score 1 if yes, 0 if no."

prompts/strict-eval.txt

You are a senior security engineer reviewing code changes.
Be strict about security best practices.
Score 0 for any potential security vulnerabilities.

File Formats

Prompt files can use any text format:

Extension	Use Case	Syntax Support
.md	Complex prompts with formatting	Markdown headers, lists, code blocks
.txt	Simple prompts	Plain text
.prompt	Custom convention	Plain text

Markdown (.md)

prompts/add-auth.md

# Add JWT Authentication

Implement JWT-based authentication for the API.

## Requirements

1. Add a login endpoint at `POST /api/auth/login`
2. Generate JWT tokens with 24-hour expiration
3. Add middleware to protect `/api/users/*` routes

## Technical Details

- Use the `jsonwebtoken` package
- Store JWT secret in `JWT_SECRET` environment variable
- Return 401 for invalid tokens, 403 for insufficient permissions

## Expected Files

- `src/middleware/auth.ts`
- `src/routes/auth.ts`
- `src/types/auth.ts`

Plain Text (.txt)

prompts/strict-eval.txt

You are a senior security engineer reviewing code changes.

Evaluation criteria:
- Be strict about security best practices
- Score 0 for any potential security vulnerabilities
- Check for proper input validation
- Verify secrets are not hardcoded

Path Resolution

Paths are resolved relative to the suite configuration file:

project/
├── suite.yaml
├── prompts/
│   ├── task.md
│   └── eval.txt
└── evaluators/
    └── security.yaml

suite.yaml

agent:
  config:
    prompt_file: ./prompts/task.md  # Relative to suite.yaml

Mutual Exclusivity

You cannot specify both prompt and prompt_file:

# ❌ Invalid - both specified
agent:
  config:
    prompt: "Inline prompt"
    prompt_file: ./prompts/task.md

# ✅ Valid - only prompt_file
agent:
  config:
    prompt_file: ./prompts/task.md

Caution

If both prompt and prompt_file are specified, validation will fail with an error.

Best Practices

1. Use Descriptive Filenames

prompts/
├── add-jwt-auth.md
├── refactor-database.md
└── add-unit-tests.md

2. Include Context in Prompts

prompts/add-tests.md

# Add Unit Tests

## Context
This is a Node.js Express application using Jest for testing.

## Task
Add unit tests for the UserService class.

## Requirements
- Test all public methods
- Mock database calls
- Achieve 80% coverage

3. Version Control Prompts

Track prompts in git alongside your configuration:

git add prompts/
git commit -m "Add authentication evaluation prompts"

4. Organize by Feature

prompts/
├── auth/
│   ├── add-jwt.md
│   └── add-oauth.md
├── testing/
│   ├── add-unit-tests.md
│   └── add-e2e-tests.md
└── refactoring/
    └── improve-error-handling.md

Validation

The yb validate command checks that prompt files exist:

yb validate -c suite.yaml -v

Output:

✅ Prompt file: ./prompts/task.md (exists)

If a file is missing:

❌ Prompt file: ./prompts/missing.md (not found)

Prompt Writing Tips

Be Specific

❌ Vague

Add tests to the project.

✅ Specific

Add unit tests for the UserService class:

1. Test `createUser()` with valid input
2. Test `createUser()` with duplicate email (should throw)
3. Test `getUserById()` with existing user
4. Test `getUserById()` with non-existent user (should return null)

Use Jest and mock the database layer.

Provide Context

Include relevant background for the AI agent:

prompts/fix-bug.md

# Fix Authentication Bug

## Context

Users report being logged out randomly. Investigation shows:
- JWT tokens expire correctly (24 hours)
- Refresh tokens work when called
- Issue occurs during active sessions

## Root Cause

The `auth.middleware.ts` doesn't handle clock skew between servers.

## Fix Required

Add a 5-minute grace period when validating token expiration.

## Files to Modify

- `src/middleware/auth.ts`
- `src/utils/jwt.ts`

Structure Complex Prompts

Use markdown structure for readability:

prompts/complex-task.md

# Task: Implement Feature X

## Overview

Brief description of what needs to be done.

## Requirements

- [ ] Requirement 1
- [ ] Requirement 2
- [ ] Requirement 3

## Technical Constraints

- Must use existing patterns
- Performance target: <100ms response time
- Backward compatible with v1 API

## Success Criteria

The implementation is complete when:
1. All tests pass
2. Documentation is updated
3. No new linting errors

Related

• Suite YAML Reference
• agentic-judge Evaluator