MLflow

MLflow 3.14.0 includes several major features and improvements

Major New Features

• 🚀 One-command agent onboarding with mlflow agent setup: Install MLflow, set up tracing, and hand your favorite coding agent (Claude Code, OpenAI Codex, or OpenCode) the MLflow skills to instrument your app, all from a single command.
• ⚡ Durable, low-latency tracing for Claude Code: Roll out Claude Code tracing across a team with confidence: a write-ahead-log keeps it from slowing the agent, overwhelming the tracking server, or losing traces on a network blip or crash.
• 📝 Review Queues for traces: Assign traces to reviewers (or agents) and collect structured feedback and ground-truth annotations in the UI, written straight back onto the trace so they are immediately usable for evaluation.
• 🗂️ Revamped evaluation dataset UI: Browse, inspect, edit, and bulk-manage evaluation dataset records directly in the UI, with click-through to the source trace.
• 🧪 Pytest integration for regression testing: Write GenAI regression tests as plain pytest functions with the @mlflow.test marker, gate them in CI, and review test history and per-assertion judge results in the UI.
• 🎛️ LLM Playground: Iterate on prompts in the browser against your AI Gateway endpoints and Prompt Registry versions, with settings, tools, structured output, and template variables.

Breaking Changes

• [Models] Change mlflow.sklearn serialization_format default from cloudpickle to skops (#23987, @copilot-swe-agent)
• [Models] Change serialization_format default to "pt2" for mlflow.pytorch.log_model and mlflow.pytorch.save_model (#23988, @copilot-swe-agent)
• [Models] Change serialization_format default to "skops" in mlflow.lightgbm log_model/save_model (#23986, @copilot-swe-agent)

Other Assorted Features & Improvements:

• [Evaluation / UI] [3/3] Show regression-test results in the existing eval-run UI (#23985, @B-Step62)
• [Prompts / UI] Add "Save prompt to registry" action to the Prompt Playground (#24021, @B-Step62)
• [Prompts] Prompt Playground (#23273, @TomeHirata)
• [Evaluation] [2/3] Add EvaluationResult.passed/.reason for @mlflow.test assertions (#23869, @B-Step62)
• [UI] Review queues: list the affected queues in the delete-question confirmation (#24002, @kriscon-db)
• [UI] Add shareable review queue URLs with a startReview deep link (#23941, @harupy)
• [UI] Allow editing a completed review in place in focus mode (#23967, @kriscon-db)
• [Tracing] Add x-mlflow-run-id support to OTLP trace ingestion (#23664, @sanatb187)
• [Evaluation / Tracing] [1/3] Add @mlflow.test pytest marker and assertion framework (#23864, @B-Step62)
• [UI] Improve review queue empty states with onboarding content (#23903, @B-Step62)
• [UI] Add mlflow skills view/list CLI (#23907, @joshuawong-db)
• [UI] Improve review queue list: flat layout, sortable columns, status filter (#23902, @B-Step62)
• [Tracing] Add MLFLOW_WORKSPACE support to OSS auth provider (#23927, @Nehanth)
• [Gateway] Add cached token pricing to Databricks model catalog (#23901, @TomeHirata)
• [Evaluation] Add MLFLOW_GENAI_JUDGE_DEFAULT_MODEL environment variable (#23860, @B-Step62)
• [Evaluation] Wire "Run judge(s)" submission in "Run Eval" in Evaluations Run page to POST /mlflow/genai/evaluate/invoke (#23781, @aaronteo-db)
• [Evaluation] Add rule-based built-in scorers: RegexMatch, PIIDetection, ResponseLength (#22571, @debu-sinha)
• [Tracing] Support Databricks backend in mlflow agent setup (#23783, @harupy)
• [Ev

<h1 align="center" style="border-bottom: none">

<a href="https://mlflow.org/">

<img alt="MLflow logo" src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/assets/logo.svg" width="200" />

</a>

</h1>

<h2 align="center" style="border-bottom: none">The Open Source AI Engineering Platform for Agents, LLMs & Models</h2>

MLflow is the largest open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug,

evaluate, monitor, and optimize production-quality AI applications while

controlling costs and managing access to models and data. With over 60 million monthly downloads,

thousands of organizations rely on MLflow each day to ship AI to production with confidence.

MLflow's comprehensive feature set for agents and LLM applications includes production-grade observability, evaluation,

prompt management, prompt optimization and an AI Gateway for managing costs and model access.

Learn more at MLflow for LLMs and Agents.

<div align="center">

Python SDK

PyPI Downloads

License

<a href="https://twitter.com/intent/follow?screen_name=mlflow" target="_blank">

<img src="https://img.shields.io/twitter/follow/mlflow?logo=X&color=%20%23f5f5f5"

alt="follow on X(Twitter)"></a>

<a href="https://www.linkedin.com/company/mlflow-org/" target="_blank">

<img src="https://custom-icon-badges.demolab.com/badge/LinkedIn-0A66C2?logo=linkedin-white&logoColor=fff"

alt="follow on LinkedIn"></a>

Ask DeepWiki

</div>

<div align="center">

<div>

<a href="https://mlflow.org/"><strong>Website</strong></a> ·

<a href="https://demo.mlflow.org/"><strong>Try Demo</strong></a> ·

<a href="https://mlflow.org/docs/latest"><strong>Docs</strong></a> ·

<a href="https://mlflow.org/blog"><strong>News</strong></a> ·

<a href="https://lu.ma/mlflow?k=c"><strong>Events</strong></a>

</div>

</div>

<br>

Get Started in 3 Simple Steps

From zero to full-stack LLMOps in minutes. No complex setup or major code changes required. Get Started →

Fastest start — set up tracing with our CLI

```bash

uvx mlflow@latest agent setup

```

One command installs the MLflow skills and launches your coding agent of choice to add tracing to your app. Prefer to wire it up yourself? Follow the three steps below.

1. Start MLflow Server

uvx mlflow server

2. Enable Logging

import mlflow

mlflow.set_tracking_uri("http://localhost:5000")
mlflow.openai.autolog()

3. Run Your Code

from openai import OpenAI

client = OpenAI()
client.responses.create(
    model="gpt-5.4-mini",
    input="Hello!",
)

Explore traces and metrics in the MLflow UI at http://localhost:5000.

LLMs & Agents

MLflow provides everything you need to build, debug, evaluate, and deploy production-quality LLM applications and AI agents. Supports Python, TypeScript/JavaScript, Java and any other programming language. MLflow also natively integrates with OpenTelemetry and MCP.

<table>

<tr>

<td width="50%">

<img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/assets/readme-tracing.png" alt="Observability" width=100%>

<div align="center">

<br>

<a href="https://mlflow.org/docs/latest/genai/tracing/"><strong>Observability</strong></a>

<br><br>

<div>Capture complete traces of your LLM applications and agents for deep behavioral insights. Built on OpenTelemetry, supporting any LLM provider and agent framework. Monitor production quality, costs, and safety.</div><br>

<a href="https://mlflow.org/docs/latest/genai/tracing/quickstart/">Getting Started →</a>

<br>

<a href="https://demo.mlflow.org/#/experiments/1/traces?startTimeLabel=CUSTOM&startTime=2026-04-17T16%3A47%3A15.258Z&endTime=2026-04-24T21%3A20%3A50.781Z">Try Demo →</a>

<br><br>

</div>

</td>

<td width="50%">

<img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/assets/readme-llm-eval.png" alt="Evaluation" width=100%>

<div align="center">

<br>

<a href="https://mlflow.org/docs/latest/genai/eval-monitor/"><strong>Evaluation</strong></a>

<br><br>

<div>Run systematic evaluations, track quality metrics over time, and catch regressions before they reach production. Choose from 50+ built-in metrics and LLM judges, or define your own.</div><br>

<a href="https://mlflow.org/docs/latest/genai/eval-monitor/">Getting Started →</a>

<br>

<a href="https://demo.mlflow.org/#/experiments/1/runs/d690ad8bb7a546c5a74b79691bb32b27/evaluations">Try Demo →</a>

<br><br>

</div>

</td>

</tr>

<tr>

<td width="50%">

<img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/assets/readme-prompt.png" alt="Prompts & Optimization" width=100%>

<div align="center">

<br>

<a href="https://mlflow.org/docs/latest/genai/prompt-registry/"><strong>Prompts & Optimization</strong></a>

<br><br>

<div>Version, test, and deploy prompts with full lineage tracking. <a href="https://mlflow.org/prompt-optimization">Automatically optimize prompts</a> with state-of-the-art algorithms to improve performance.</div><br>

<a href="https://mlflow.org/docs/latest/genai/prompt-registry/create-and-edit-prompts/">Getting Started →</a>

<br>

<a href="https://demo.mlflow.org/#/experiments/1/prompts/mlflow-demo.prompts.code-reviewer?promptVersion=4">Try Demo →</a>

<br><br>

</div>

</td>

<td width="50%">

<img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/assets/readme-gateway.png" alt="AI Gateway" width=100%>

<div align="center">

<br>

<a href="https://mlflow.org/docs/latest/genai/governance/ai-gateway/"><strong>AI Gateway</strong></a>

<br><br>

<div>Unified API gateway for all LLM providers. Route requests, manage rate limits, handle fallbacks, and control costs through an OpenAI-compatible interface with built-in credential management, guardrails and traffic splitting for A/B testing.</div><br>

<a href="https://mlflow.org/docs/latest/genai/governance/ai-gateway/quickstart/">Getting Started →</a>

<br><br>

</div>

</td>

</tr>

</table>

Model Training

For machine learning and deep learning model development, MLflow provides a full suite of tools to manage the ML lifecycle:

• Experiment Tracking — Track models, parameters, metrics, and evaluation results across experiments
• Model Evaluation — Automated evaluation tools integrated with experiment tracking
• Model Registry — Collaboratively manage the full lifecycle of ML models
• Deployment — Deploy models to batch and real-time scoring on Docker, Kubernetes, Azure ML, AWS SageMaker, and more

Learn more at MLflow for Model Training.

Integrations

MLflow supports all agent frameworks, LLM providers, tools, and programming languages. We offer one-line automatic tracing for more than 60 frameworks. See the full integrations list.

OpenTelemetry

<table>

<tr>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/app-instrumentation/opentelemetry"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/opentelemetry-logo-only.png" height="40"><br>_{<b>OpenTelemetry</b>}</a></td>

</tr>

</table>

Agent Frameworks (Python)

<table>

<tr>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/langchain"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/langchain-logo-only.png" height="40"><br>_{<b>LangChain</b>}</a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/langgraph"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/langgraph-logo-only.png" height="40"><br>_{<b>LangGraph</b>}</a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/openai-agent"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/openai-logo-only.png" height="40"><br>_{<b>OpenAI Agent</b>}</a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/dspy"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/dspy-logo.png" height="40"><br>_<b>DSPy</b></a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/pydantic_ai"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/pydantic-ai-logo-only.png" height="40"><br>_{<b>PydanticAI</b>}</a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/google-adk"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/google-adk-logo.png" height="40"><br>_{<b>Google ADK</b>}</a></td>

</tr>

<tr>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/microsoft-agent-framework"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/microsoft-agent-framework-logo.png" height="40"><br>_{<b>Microsoft Agent</b>}</a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/crewai"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/crewai-logo.svg" height="40"><br>_{<b>CrewAI</b>}</a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/llama_index"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/llamaindex-logo.svg" height="40"><br>_{<b>LlamaIndex</b>}</a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/autogen"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/autogen-logo.png" height="40"><br>_{<b>AutoGen</b>}</a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/strands"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/strands-logo.png" height="40"><br>_{<b>Strands</b>}</a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/livekit"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/livekit-logo.png" height="40"><br>_{<b>LiveKit Agents</b>}</a></td>

</tr>

<tr>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/agno"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/agno-logo.png" height="40"><br>_<b>Agno</b></a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/bedrock-agentcore"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/bedrock-logo.png" height="40"><br>_{<b>Bedrock AgentCore</b>}</a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/smolagents"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/smolagents-logo.png" height="40"><br>_{<b>Smolagents</b>}</a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/semantic_kernel"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/semantic-kernel-logo.png" height="40"><br>_{<b>Semantic Kernel</b>}</a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/deepagent"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master/docs/static/images/logos/deepagent-logo.svg" height="40"><br>_{<b>DeepAgent</b>}</a></td>

<td align="center" width="110"><a href="https://mlflow.org/docs/latest/genai/tracing/integrations/listing/ag2"><img src="https://raw.githubusercontent.com/mlflow/mlflow/refs/heads/master