How do I configure mcpbr to use my MCP server?

Configure the mcp_server section in your YAML config file with the command to start your server, args (using {workdir} as placeholder for the task repository path), and any required environment variables.

What configuration parameters are available in mcpbr?

Key parameters include mcp_server (command, args, env), provider (anthropic), model, benchmark, sample_size, timeout_seconds, max_concurrent, and max_iterations.

Configuration

mcpbr uses YAML configuration files to define your MCP server settings and evaluation parameters.

Quick Start

The fastest way to get started is with example configurations:

mcpbr run -c examples/quick-start/getting-started.yaml -v

Or generate a custom config:

mcpbr init

Full Configuration Example

# MCP Server Configuration
mcp_server:
  name: "mcpbr"
  command: "npx"
  args:
    - "-y"
    - "@modelcontextprotocol/server-filesystem"
    - "{workdir}"
  env: {}

# Provider and Harness
provider: "anthropic"
agent_harness: "claude-code"

# Model Configuration
model: "sonnet"

# Benchmark Selection
benchmark: "swe-bench-verified"
sample_size: 10

# Execution Parameters
timeout_seconds: 300
max_concurrent: 4
max_iterations: 10

# Docker Configuration
use_prebuilt_images: true

MCP Server Section

Field	Type	Description
`name`	string	Name to register the MCP server as (default: `mcpbr`)
`command`	string	Executable to run (e.g., `npx`, `uvx`, `python`)
`args`	list	Command arguments. Use `{workdir}` as placeholder
`env`	dict	Additional environment variables

Environment Variables

mcp_server:
  command: "npx"
  args: ["-y", "@supermodeltools/mcp-server"]
  env:
    SUPERMODEL_API_KEY: "${SUPERMODEL_API_KEY}"
    LOG_LEVEL: "${LOG_LEVEL:-info}"

Benchmark Selection

Field	Default	Description
`benchmark`	`swe-bench-verified`	Benchmark to run
`sample_size`	`null`	Number of tasks (`null` = full dataset)

Execution Parameters

Field	Default	Description
`timeout_seconds`	`300`	Timeout per task in seconds
`max_concurrent`	`4`	Maximum parallel task evaluations
`max_iterations`	`10`	Maximum agent iterations per task
`thinking_budget`	`null`	Extended thinking token budget (1024-31999)
`budget`	`null`	Maximum budget in USD

Example Configurations

Filesystem Server

mcp_server:
  command: "npx"
  args: ["-y", "@modelcontextprotocol/server-filesystem", "{workdir}"]

Fast Iteration (Development)

mcp_server:
  command: "npx"
  args: ["-y", "@modelcontextprotocol/server-filesystem", "{workdir}"]

model: "haiku"
sample_size: 3
max_concurrent: 1
timeout_seconds: 180

Full Benchmark Run

mcp_server:
  command: "npx"
  args: ["-y", "@modelcontextprotocol/server-filesystem", "{workdir}"]

model: "sonnet"
sample_size: null
max_concurrent: 8
timeout_seconds: 600

Next Steps

CLI Reference — Command options that override config values
Benchmarks — Available benchmarks and how to use them
Examples — Browse 25+ example configurations

Created by Grey Newell