Our Python SDK is live

bolder-ai 0.1.0 is live on PyPI. It's the official Python SDK for BEval Studio, our hosted evaluation platform for production LLM systems. This post covers the three ways to use it, what's inside, and what ships next.

Install

pip install bolder-ai

Set an API key and every LLM call your app makes shows up in your BEval dashboard with latency, token counts, cost and status.

The distribution name is bolder-ai. The import name is beval.

import beval

beval.init()  # reads BEVAL_API_KEY from env

Three ways to use it

1. Log anything directly

beval.log(
    kind="llm",
    model_id="gpt-4o-mini",
    input="What is the capital of France?",
    output="Paris.",
    latency_ms=312,
    tokens_in=7,
    tokens_out=2,
)

There's no schema to learn. The fields map 1:1 to what the dashboard displays.

Logging is fire-and-forget. The call returns in microseconds and the network POST happens on a background thread. If the gateway is down, your app doesn't care.

2. Auto-wrap your OpenAI or Anthropic client

import beval
from openai import OpenAI

beval.init()
client = beval.wrap(OpenAI())

client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello."}],
)

Every chat.completions.create call is captured: input messages, output, token usage, latency and errors. Image parts are detected automatically and logged as kind="vlm" with the image inlined. beval.wrap(Anthropic()) works the same way.

Your call sites don't change.

3. Decorate agent functions

@beval.trace
def run_agent(query: str) -> str:
    # ...
    return answer

@beval.trace(name="tool:search", kind="agent")
async def search(q): ...

@beval.trace captures arguments, return value, latency and exceptions, for both sync and async functions. An exception is logged with status="failure" plus its class and message, then re-raised.

What's inside

The SDK is deliberately small and boring.

Piece	Behavior
Dependencies	One hard dependency, `httpx`. Optional extras for the `openai` and `anthropic` integrations.
Delivery	One background thread drains an in-memory queue (10,000-entry capacity, drop-on-overflow).
Retries	408, 429 and 5xx responses retry with exponential backoff. Network errors never raise to your code.
Shutdown	An `atexit` hook drains the queue with a 5-second timeout, so logs buffered during the last request still go out.
Redaction	A hook at `init()` scrubs PII before anything is sent.

Installed footprint: a ~22 KB wheel.

Why we built it

BEval Studio has run for months on direct API ingest from the VLM mock app and the bolder-fit-agent production workload. Every team integrating BEval wrote roughly the same HTTP client, retry loop, background thread and OpenAI wrapper. We pulled all of that into one package.

The SDK talks to a single endpoint, POST /api/v1/logs/ingest, which has been stable since BEval launched. That means:

Existing custom integrations keep working.
The SDK stays backward compatible across BEval dashboard upgrades.
New SDK versions won't force backend changes.

Already running in production

The day we published 0.1.0 to PyPI, we integrated it into bolder-fit-agent, our coach-facing agentic service that drives multi-turn LiteLLM + MCP agent runs. Every agent turn now shows up in the BEval dashboard with session ID, plan type, turn index, tool calls and cost.

The integration diff was 7 files, 159 insertions. One file is the SDK init helper. Most of the rest is a single helper function that builds a log entry around the existing LLM call site.

What's next

This is 0.1. The short list for 0.2 and beyond:

Batch ingest. Today the SDK posts one log per call. That's fine for most workloads and wasteful for high-throughput ones, so a batch endpoint is the top priority.
Streaming support. Wrappers don't log when stream=True yet. We'll accumulate chunks, log on stream end, and track time-to-first-token as a first-class metric.
Traces and spans. Flat logs work for single LLM calls, but agent runs need a parent/child tree. Once the backend trace model lands, the SDK gets a context manager and nested @trace support.
Callback integrations for LangChain, LlamaIndex and LiteLLM.
Direct-to-S3 image upload for VLM payloads over ~64 KB.

Get started

Install with pip install bolder-ai.
Create an API key in your BEval dashboard under Settings → API Keys.
Set BEVAL_API_KEY in your environment.
Add three lines to your app: beval.init() at startup, then beval.log(...), beval.wrap(client) or @beval.trace.

Full docs are at docs.bolder.services. The package is on PyPI at pypi.org/project/bolder-ai.

Running LLMs in production and want structured visibility without standing up your own observability stack? Book a call and we'll get you on the dashboard the same day.