Haystack

⭐️ Highlights

Haystack 3.0 is a major release for building production-grade agents with full control and flexibility. It ships a wave of new capabilities: a more capable Agent with hooks and first-class skills, built-in run introspection, first-class async for serving, a leaner core, and safer pipeline loading.

A few small, intentional breaking changes come with it but our Migration Guide and Upgrading to Haystack 3.0 make it easy to upgrade.

🤖 A more capable, adaptable Agent

The Agent gained a general-purpose hooks system and now owns tool execution end to end (the standalone ToolInvoker has been removed). Hooks at before_run, before_llm, before_tool, after_tool, on_exit, and after_run let you shape behavior, enforce guardrails, and add human-in-the-loop checkpoints without touching the agent's internals:

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.hooks import hook

@hook
def audit_tool_calls(state):
    pending = state.data["messages"][-1].tool_calls
    print(f"about to run: {[tc.tool_name for tc in pending]}")

agent = Agent(
    chat_generator=OpenAIChatGenerator(),
    tools=[...],
    hooks={"before_tool": [audit_tool_calls]},
)

Built on this foundation:

• Skills as first-class citizens — SkillToolset discovers skills through progressive disclosure, so the model sees only names and one-line descriptions until it loads one, keeping context lean.
• Dynamic tool selection at runtime — pass tools=... to run / run_async so one reusable Agent serves different teams, tenants, and tasks.
• Native async tools — @tool routes async def callables to a Tool's new async_function.
• Human-in-the-Loop is now a before_tool hook (ConfirmationHook). ⚠️
• Tool result offloading (ToolResultOffloadHook) writes large results to a store and leaves a compact pointer in the conversation.

🔎 Built-in introspection and observability

Agents now expose step_count, token_usage, and tool_call_counts as built-in state. React to them to compact context when the window fills, cap runaway tool loops, or route to a cheaper approach past a budget threshold. Tracing now emits dedicated step-level spans (haystack.agent.step with nested .llm / .tool children) tagged with the tools actually used, so you can see exactly what your agent did. ⚠️

⚡ A core built for serving: first-class async

Pipeline and AsyncPipeline are now one class — no more switching between them. A single Pipeline exposes run, run_async, run_async_generator, and stream; serialized pipelines, SuperComponents, and PipelineTools load as Pipeline instances. ⚠️

from haystack import Pipeline

pipeline = Pipeline()
result = pipeline.run(data)                 # synchronous
result = await pipeline.run_async(data)     # asynchronous

Concurrent tool calls and token-by-token streaming come standard: Pipeline.stream() yields StreamingChunks as they're produced and exposes the final output on handle.result — the lower latency and streaming UX serving apps need. Components and Pipeline also get a symmetric lifecycle — warm_up to acquire, close to release — so long-running services don't leak connections, GPU memory, or file handles. External resources (and API keys) are now created at warm_up, not in __init__. ⚠️

🧱 A leaner, faster-moving framework ⚠️

Legacy Generators are gone, haystack-experimental is no longer a core dependency, and 30 components now live in independently released packages in haystack-core-integrations: Sentence Transformers, local & API Hugging Face, Whisper, spaCy/langdetect, Tika and Azure OCR, SerperDev/SearchApi, OpenAPI connectors, and the

<div align="center">

<a href="https://haystack.deepset.ai/"><img src="https://raw.githubusercontent.com/deepset-ai/haystack/main/images/banner.png" alt="Blue banner with the Haystack logo and the text ‘haystack by deepset – The Open Source AI Framework for Production Ready RAG & Agents’ surrounded by abstract icons representing search, documents, agents, pipelines, and cloud systems."></a>

| | |

| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |

| CI/CD | Tests types - Mypy Coverage badge Ruff |

| Docs | Website |

| Package | PyPI PyPI - Downloads PyPI - Python Version Conda Version GitHub License Compliance HVTrust Evidence Grade OpenSSF Best Practices|

| Meta | Discord Twitter Follow |

</div>

<div align="center">

🎉🎊✨ &nbsp; Haystack 3.0 is out! &nbsp; ✨🎊🎉

Read the announcement here!

🥳 🎈 🎆 🪅 🎇 🍾 🥂 🎁 🌈

</div>

Haystack is an open-source AI orchestration framework for building production-ready LLM applications in Python.

Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Build scalable RAG systems, multimodal applications, semantic search, question answering, and autonomous agents, all in a transparent architecture that lets you experiment, customize deeply, and deploy with confidence.

Table of Contents

• Installation
• Documentation
• Features
• Haystack Enterprise: Support & Platform
• Telemetry
• 🖖 Community
• Contributing to Haystack
• Organizations using Haystack

Installation

The simplest way to get Haystack is via pip:

pip install haystack-ai

Install nightly pre-releases to try the newest features:

pip install --pre haystack-ai

Haystack supports multiple installation methods, including Docker images. For a comprehensive guide, please refer

to the documentation.

Documentation

If you're new to the project, check out "What is Haystack?" then go

through the "Get Started Guide" and build your first LLM application

in a matter of minutes. Keep learning with the tutorials. For more advanced

use cases, or just to get some inspiration, you can browse our Haystack recipes in the

Cookbook.

At any given point, hit the documentation to learn more about Haystack, what it can do for you, and the technology behind.

Features

Agents built for production

Extend agent behavior with lifecycle hooks (before_llm, before_tool, on_exit, …) for guardrails and custom logic, and track step_count, token_usage, and tool calls out of the box for monitoring and cost control. Get started fast with ready-made agents from Agent Pack (e.g., a deep research agent, or an advanced RAG agent) or give your own agents progressive skill discovery via SkillToolset, so skill descriptions only enter context when needed.

Built for context engineering

Design flexible systems with explicit control over how information is retrieved, ranked, filtered, combined, structured, and routed before it reaches the model. Define pipelines and agent workflows where retrieval, memory, tools, and generation are transparent and traceable.

Native Async Support

One Pipeline runs synchronously or asynchronously and streams token by token. Agent can run concurrent tool calls.

Modular and customizable

Use built-in components for retrieval, indexing, tool calling, memory, and evaluation, or create your own. Add loops, branches, and conditional logic to precisely control how context moves through your pipelines and agent workflows.

Model- and vendor-agnostic

Integrate with OpenAI, Mistral, Anthropic, Cohere, Hugging Face, Google, Azure OpenAI, AWS Bedrock, local models, and many others. Swap models or infrastructure components without rewriting your system.

Extensible ecosystem

Build and share custom components through a consistent interface that makes it easy for the community and third parties to extend Haystack and contribute to an open ecosystem.

[!TIP]

Would you like to deploy and serve Haystack pipelines as REST APIs or MCP servers? Hayhooks provides a simple way for you to wrap pipelines and agents with custom logic and expose them through HTTP endpoints or MCP. It also supports OpenAI-compatible chat completion endpoints and works with chat UIs like open-webui.

Haystack Enterprise: Support & Platform

Get expert support from the Haystack team, build faster with enterprise-grade templates, and scale securely with deployment guides for cloud and on-prem environments with Haystack Enterprise Starter. Read more about it in the announcement post.

👉 Get Haystack Enterprise Starter

Need a managed production setup for Haystack? The Haystack Enterprise Platform helps you build, test, deploy and operate Haystack pipelines with built-in observability, collaboration, governance, and access controls. It’s available as a managed cloud service or as a self-hosted solution.

👉 Learn more about Haystack Enterprise Platform or try it free

Telemetry

Haystack collects anonymous usage statistics of pipeline components. We receive an event every time these components are initialized. This way, we know which components are most relevant to our community.

Read more about telemetry in Haystack or how you can opt out in Haystack docs.

🖖 Community

If you have a feature request or a bug report, feel free to open an issue in GitHub. We regularly check these, so you can expect a quick response. If you'd like to discuss a topic or get more general advice on how to make Haystack work for your project, you can start a thread in Github Discussions or our Discord channel. We also check 𝕏 (Twitter) and Stack Overflow.

Contributing to Haystack

We are very open to the community's contributions - be it a quick fix of a typo, or a completely new feature! You don't need to be a Haystack expert to provide meaningful improvements. To learn how to get started, check out our Contributor Guidelines first.

There are several ways you can contribute to Haystack:

• Contribute to the main Haystack project
• Contribute an integration on haystack-core-integrations
• Contribute to the documentation in haystack/docs-website

[!TIP]

👉 Check out the full list of issues that are open to contributions

Organizations using Haystack

Haystack is used by thousands of teams building production AI systems across industries, including:

• Technology & AI Infrastructure: Apple, Meta, Databricks, NVIDIA, Intel
• Public Sector AI Initiatives: European Commission, German Federal Ministry of Research, Technology, and Space (BMFTR), PD, Baden-Württemberg State
• Enterprise & Industrial AI Applications: Airbus, Lufthansa Industry Solutions, Infineon, LEGO, Comcast, Accenture, TELUS Agriculture & Consumer Goods
• Knowledge & Content Platforms: Netflix, ZEIT Online, Rakuten, Oxford University Press, Manz, YPulse

Are you also using Haystack? Open a PR or tell us your story