txtai

This release adds support for new ANN backends along with a bonanza of bug fixes from 7 new contributors!

See below for full details on the new features, improvements and bug fixes.

New Features

---

• Feature request: Support zvec as vector backend (#1040, #1130) @morgan-coded
• Add milvus-lite embedded dense ANN backend (#1149) @morgan-coded
• Allow disabling API routes (#1165)

Improvements

---

• Add tokenbatch size to HFTrainer pipeline (#1133)
• Only resolve transform functions when env variable set (#1155)
• Ensure explain / batchexplain don't return numpy values (#1157)
• Set minimum milvus-lite version (#1162) @morgan-coded
• Add safeopen option for retrieve task (#1166)

Bug Fixes

---

• Raise SQLError on unterminated bracket, function and similar clauses (#1125) @Sanjays2402
• Torch 2.13 introduced build errors (#1134)
• Fix FileTask/UrlTask crash on non-string elements (#1135) @chuenchen309
• Fix Graph.filter() crash on ids that don't exist in the graph (#1137) @chuenchen309
• fix: Task.filteredrun crashes/corrupts output for single-action tasks with merge disabled (#1138) @chuenchen309
• Fix Aggregate IndexError on a SQL query that matched nothing across shards (#1139) @chuenchen309
• Fix HTMLToMarkdown crash on a meta description tag with no content attribute (#1140) @chuenchen309
• Fix TemplateTask.match KeyError when a rule key is missing from element (#1141) @chuenchen309
• Fix Terms KeyError on a query with no similar() clause (#1142) @chuenchen309
• Fix Terms.search AttributeError on an empty terms index (#1143) @chuenchen309
• Fix msgpack Accept header crashing streaming API endpoints (#1144) @chuenchen309
• Fix console printing a traceback for a query with no results (#1145) @chuenchen309
• Fix Cluster.shard TypeError on tuple documents and empty-string ids (#1146) @chuenchen309
• Raise SQLError on empty select expressions and a dangling AS (#1147) @chuenchen309
• Fix LIMIT bind-parameter crashing content SQL search (#1148) @chuenchen309
• The authorization header doesn't work on Swagger UI (#1150, #1151) @link89
• Fix Segmentation dropping paragraph/section splits on CRLF line endings (#1152) @lntutor
• Validate resolver target classes (#1153)
• Fix IVFSparse returning unsorted sparse ANN search results (#1154) @winklemad
• Fix build error with Milvus on Windows (#1156)
• HNSW search crashes with RuntimeError when limit exceeds indexed element count (#1158) @AmirF194
• Fix HNSW search RuntimeError when limit exceeds indexed element count (#1159) @AmirF194
• Fix Scoring.delete() crashing on ids that were never indexed (#1163) @AmirF194

<p align="center">

<img src="https://raw.githubusercontent.com/neuml/txtai/master/logo.png"/>

</p>

<p align="center">

<b>All-in-one AI framework</b>

</p>

<p align="center">

<a href="https://github.com/neuml/txtai/releases">

<img src="https://img.shields.io/github/release/neuml/txtai.svg?style=flat&color=success" alt="Version"/>

</a>

<a href="https://github.com/neuml/txtai">

<img src="https://img.shields.io/github/last-commit/neuml/txtai.svg?style=flat&color=blue" alt="GitHub last commit"/>

</a>

<a href="https://github.com/neuml/txtai/issues">

<img src="https://img.shields.io/github/issues/neuml/txtai.svg?style=flat&color=success" alt="GitHub issues"/>

</a>

<a href="https://join.slack.com/t/txtai/shared_invite/zt-37c1zfijp-Y57wMty6YOx_hyIHEQvQJA">

<img src="https://img.shields.io/badge/slack-join-blue?style=flat&logo=slack&logocolor=white" alt="Join Slack"/>

</a>

<a href="https://github.com/neuml/txtai/actions?query=workflow%3Abuild">

<img src="https://github.com/neuml/txtai/workflows/build/badge.svg" alt="Build Status"/>

</a>

<a href="https://coveralls.io/github/neuml/txtai?branch=master">

<img src="https://img.shields.io/coverallsCoverage/github/neuml/txtai" alt="Coverage Status">

</a>

</p>

txtai is an all-in-one AI framework for semantic search, LLM orchestration and language model workflows.

architecture

architecture

The key component of txtai is an embeddings database, which is a union of vector indexes (sparse and dense), graph networks and relational databases.

This foundation enables vector search and/or serves as a powerful knowledge source for large language model (LLM) applications.

Build autonomous agents, retrieval augmented generation (RAG) processes, multi-model workflows and more.

Summary of txtai features:

• 🔎 Vector search with SQL, object storage, topic modeling, graph analysis and multimodal indexing
• 📄 Create embeddings for text, documents, audio, images and video
• 💡 Pipelines powered by language models that run LLM prompts, question-answering, labeling, transcription, translation, summarization and more
• ↪️️ Workflows to join pipelines together and aggregate business logic. txtai processes can be simple microservices or multi-model workflows.
• 🤖 Agents that intelligently connect embeddings, pipelines, workflows and other agents together to autonomously solve complex problems
• ⚙️ Web and Model Context Protocol (MCP) APIs. Bindings available for JavaScript, Java, Rust and Go.
• 🔋 Batteries included with defaults to get up and running fast
• ☁️ Run local or scale out with container orchestration

txtai is built with Python 3.10+, Hugging Face Transformers, Sentence Transformers and FastAPI. txtai is open-source under an Apache 2.0 license.

[!NOTE]

NeuML is the company behind txtai and we provide AI consulting services around our stack. Schedule a meeting or send a message to learn more.

We're also building an easy and secure way to run hosted txtai applications with txtai.cloud.

Why txtai?

why

why

New vector databases, LLM frameworks and everything in between are sprouting up daily. Why build with txtai?

• Up and running in minutes with pip or Docker

# Get started in a couple lines
import txtai

embeddings = txtai.Embeddings()
embeddings.index(["Correct", "Not what we hoped"])
embeddings.search("positive", 1)
#[(0, 0.29862046241760254)]

• Built-in API makes it easy to develop applications using your programming language of choice

# app.yml
embeddings:
    path: sentence-transformers/all-MiniLM-L6-v2

CONFIG=app.yml uvicorn "txtai.api:app"
curl -X GET "http://localhost:8000/search?query=positive"

• Run local - no need to ship data off to disparate remote services
• Work with micromodels all the way up to large language models (LLMs)
• Low footprint - install additional dependencies and scale up when needed
• Learn by example - notebooks cover all available functionality

Use Cases

The following sections introduce common txtai use cases. A comprehensive set of over 70 example notebooks and applications are also available.

Semantic Search

Build semantic/similarity/vector/neural search applications.

demo

Traditional search systems use keywords to find data. Semantic search has an understanding of natural language and identifies results that have the same meaning, not necessarily the same keywords.

search

search

Get started with the following examples.

| Notebook | Description | |

|:----------|:-------------|------:|

| Introducing txtai ▶️ | Overview of the functionality provided by txtai | Open In Colab |

| Similarity search with images | Embed images and text into the same space for search | Open In Colab |

| Build a QA database | Question matching with semantic search | Open In Colab |

| Semantic Graphs | Explore topics, data connectivity and run network analysis| Open In Colab |

LLM Orchestration

Autonomous agents, retrieval augmented generation (RAG), chat with your data, pipelines and workflows that interface with large language models (LLMs).

llm

See below to learn more.

| Notebook | Description | |

|:----------|:-------------|------:|

| Prompt templates and task chains | Build model prompts and connect tasks together with workflows | Open In Colab |

| Integrate LLM frameworks | Integrate llama.cpp, LiteLLM and custom generation frameworks | Open In Colab |

| Build knowledge graphs with LLMs | Build knowledge graphs with LLM-driven entity extraction | Open In Colab |

| Parsing the stars with txtai | Explore an astronomical knowledge graph of known stars, planets, galaxies | Open In Colab |

Agents

Agents connect embeddings, pipelines, workflows and other agents together to autonomously solve complex problems.

agent

txtai agents are built on top of the smolagents framework. This supports all LLMs txtai supports (Hugging Face, llama.cpp, OpenAI / Claude / AWS Bedrock via LiteLLM). Agent prompting with agents.md and skill.md are also supported.

Check out this Agent Quickstart Example. Additional examples are listed below.

| Notebook | Description | |

|:----------|:-------------|------:|

| Granting autonomy to agents | Agents that iteratively solve problems as they see fit | Open In Colab |

| TxtAI got skills | Integrate skill.md files with your agent | Open In Colab |

| Agent Tools ▶️ | Learn about the txtai agent toolkit | Open In Colab |

| Analyzing LinkedIn Company Posts with Graphs and Agents | Exploring how to improve social media engagement with AI | Open In Colab |

Retrieval augmented generation

Retrieval augmented generation (RAG) reduces the risk of LLM hallucinations by constraining the output with a knowledge base as context. RAG is commonly used to "chat with your data".

rag

rag

Check out this RAG Quickstart Example. Additional examples are listed below.

| Notebook | Description | |

|:----------|:-------------|------:|

| Build RAG pipelines with txtai ▶️ | Guide on retrieval augmented generation including how to create citations | Open In Colab |

| RAG is more than Vector Search | Context retrieval via Web, SQL and other sources | Open In Colab |

| GraphRAG with Wikipedia and GPT OSS | Deep graph search powered RAG | Open In Colab |

| Speech to Speech RAG ▶️ | Full cycle speech to speech workflow with RAG | Open In Colab |

Language Model Workflows

Language model workflows, also known as semantic workflows, connect language models together to build intelligent applications.

![flows](https://raw.gi