MCP server exposing 8 tools for ai-model-experiments.
This URL is a JSON-RPC 2.0 endpoint over HTTP. Issue POST requests with a JSON-RPC body. Browsers and search crawlers land here on GET.
POST https://gateway.pipeworx.io/ai-model-experiments/mcp
Content-Type: application/json
{"jsonrpc":"2.0","id":1,"method":"tools/list"}experiment_models — List AI models available for experiments (about 300 across Anthropic, OpenAI, Google, Meta, Mistral, DeepSeek, Qwen and more), with context window and OUR per-token prices (provider cost × 1.5 — what experiments actually bill). Filter by name/vendor search, minimum context, or max price. Use the returned model ids in experiment_create. Example: experiment_models({ search: "claude", min_context: 100000 })experiment_estimate — Dry-run cost estimate for an experiment BEFORE creating it — cell count and estimated billed cost range (at our 1.5× pricing) for prompts × models × reps. Free to call, no side effects, does not need credit balance. Same spec shape as experiment_create. Example: experiment_estimate({ prompts: ["Summarize: ..."], models: ["anthropic/claude-sonnet-4.5", "openai/gpt-5"], reps: 2 })experiment_create — Model Lab: run one prompt across many AI models at once and compare their outputs, cost, and latency side by side. Create and start an experiment: run each prompt against each model (× reps), collecting output, tokens, latency, and billed cost per cell. PREPAID: requires Pipeworx credit balance ≥ max_spend_usd (top up via experiment_topup); bills actual provider cost × 1.5 with a $0.10 minimum per experiment. ASYNC: returns experiment_id immediately — execution starts within ~1 minute; poll experiment_status until complete, then call experiment_results. Do NOT wait synchronously. Set summary:false to skip the AI-written model-comparison summary. Example: experiment_create({ name: "tone test", prompts: ["Rewrite formally: ..."], models: ["anthropic/claude-haiku-4.5", "openai/gpt-5-mini"], reps: 2, max_spend_usd: 2 })experiment_status — Progress of an experiment: cell counts by state (pending/running/ok/error/skipped), spend so far vs cap, and whether it is complete. Poll this after experiment_create (every few seconds). Example: experiment_status({ experiment_id: "..." })experiment_results — Results of an experiment: per-model aggregates (mean latency, tokens, total billed cost, error rate), per-cell outputs, and the AI-written summary comparing how the models differed (if enabled). Use include_outputs:false for aggregates only. Example: experiment_results({ experiment_id: "..." })experiment_list — List your experiments, newest first, with status and spend. Example: experiment_list({ limit: 10 })experiment_cancel — Cancel a running experiment: pending cells are skipped (not billed); in-flight cells finish and bill. Example: experiment_cancel({ experiment_id: "..." })experiment_topup — How to add prepaid credits for experiments (and your current balance). Payment is x402 — USDC on Base, paid in-band by any wallet-equipped agent: POST https://gateway.pipeworx.io/credits/topup?amount_usd=10 responds HTTP 402 with payment requirements; retry with PAYMENT-SIGNATURE to settle and the credits land instantly. Example: experiment_topup({})Code samples (curl / TypeScript / one-click client install), schemas, and the live playground are on the pack page:
https://pipeworx.io/packs/ai-model-experiments/
Pipeworx is an open MCP gateway connecting AI agents to live data. pipeworx.io