Bug Fixes

• Fix a bug where negative tag filters (e.g.user_id != 1) matched inferences and datapoints without that tag.
• Fix a bug where metric filters covering default values (e.g.exact_match = false) matched inferences without that metric.
• Fix a regression affecting the logger in the UI.

New Features

• Improve the performance of the inference and datapoint list pages in the UI.
• Support filtering inferences by whether they have a demonstration.

& multiple under-the-hood and UI improvements (thanks@jinnovation@ecalifornica@simeonlee)!

Contributors

Bug Fixes

• Fix a performance regression affecting the inference table in the UI.

New Features

• Allow users to customize the log level in the UI (TENSORZERO_UI_LOG_LEVEL).

& multiple under-the-hood and UI improvements

Bug Fixes

• Fixed a regression that broke the dataset builder in the UI.

& multiple under-the-hood and UI improvements

Bug Fixes

• Fix a regression in the "Try with..." modal in the UI that disregarded some parameters (e.g.allowed_tools).
• Fix a regression inallowed_tools when using custom display names for tools.
• Fix an edge case when using bothallowed_tools andtool_choice parameters with GCP Vertex AI Gemini.

New Features

• Support free-form search and filtering (e.g. by tags, metrics) the inference and datapoint tables in the UI.
• Support creating datapoints from scratch in the UI.
• Support editing TensorZero API key descriptions in the UI (thanks@nicoestrada!).
• Support editing any kind of datapoint input and output in the UI.
• Support peeking at inferences in the episode detail page in the UI (thanks@BrianLi23!).
• Support cloning datapoints in the UI.
• Optimize the rendering performance of the code editor in the UI.
• Makemime_type optional for base64 file inputs (now inferred from magic bytes when not provided).

& multiple under-the-hood and UI improvements

Contributors

Bug Fixes

• Handle a regression in ClickHouselatest that affected the endpoint for deleting datapoints.

New Features

• Support running evaluations programmatically on specific datapoints (datapoint_ids).
• Generatevalues.schema.json for the Helm chart. (thanks@Erin-Boehmer!)

Contributors

New Features

• Support Anthropic's Beta Structured Outputs feature natively (beta_structured_outputs).extra_headers is no longer necessary.
• Supportjson_mode="tool" in chat inferences that don't otherwise include tools.
• Supportextra_body andextra_headers supplied at inference time without scope filters.
• Supportextra_body andextra_headers supplied at inference time withmodel_name and optionalprovider_name scope filters.
• Support thought signatures for the GCP Vertex model providers.
• Support custom tools for the OpenAI model provider.
• Adddescription fields to evaluation and evaluator configuration.

& multiple under-the-hood and UI improvements

New Features

• Add adaptive stopping for evaluations in the UI and Python SDK.
• Support explicitcandidate_variants andfallback_variants when using uniform sampling.
• Support theinput_audio content block in the OpenAI-compatible inference endpoint.
• Support theinput_audio content block in the OpenAI, Azure, GCP Vertex Gemini, Google AI Studio, and OpenRouter model providers.
• Add optionalfilename field for input files.
• Move closer to parity between the GCP Vertex Anthropic model provider and the Anthropic model provider.
• Expose new observability and dataset management endpoints as methods in the TensorZero Python SDK.
• Add optionalpostgres.enabled field to the configuration.
• Handle missing usage information from model providers that don't report it.
• Add experimental method for searching inferences programmatically (search_query_experimental).
• Add a native OpenRouter embedding model provider.

& multiple under-the-hood and UI improvements

Bug Fixes

• Enable TLS support for Postgres connections.
• Fix handling of user-defined tags in batch inference.

& multiple under-the-hood and UI improvements

Bug Fixes

• Fix a bug that prevented some model inferences from being rendered correctly in the UI.
• Handle non-image base64 file inputs consistently in the OpenAI-compatible inference endpoint.
• Handleraw_response correctly for batch inference with GCP Vertex AI Gemini.

New Features

• Apply thetensorzero::api_key_public_id tag to inference and feedback when using auth.
• Add updated HTTP endpoint for creating datapoints (POST /v1/datasets/{dataset_name}/datapoints).
• Addgateway.global_outbound_http_timeout_ms configuration setting.

& multiple under-the-hood and UI improvements (thanks@omarraf!)

Contributors

Bug Fixes

• Fix a regression that prevented batch inferences from being rendered in the UI.
• Handle missing Postgres credentials gracefully in the UI.

New Features

• Support rate limiting by API key (api_key_public_id).
• Add nativeservice_tier inference parameter (supported providers: Anthropic, Azure, Groq, OpenAI).extra_body is no longer necessary.
• Add nativedetail parameter for input images (supported providers: Azure, OpenAI, xAI).extra_body is no longer necessary.
• Add updated HTTP endpoint for querying inferences by ID (POST /v1/inferences/get_inferences).
• Add updated HTTP endpoint for querying inferences with filters (POST /v1/inferences/list_inferences).

& multiple under-the-hood and UI improvements