Typesense Fast Search Engine

This release fixes some important bugs identified in v30.1.

Bug Fixes

• [v30.2] Fixed incorrect handling of numeric != filters in both lazy and eager filter execution, including explicit not-equals lists on range-indexed numeric fields.
• [v30.2] Fixed highlighting to correctly mark single-token phrase matches and to avoid race conditions during union searches that run alongside concurrent writes.
• [v30.2] Fixed missing synonyms, stem, and stemming_dictionary fields in curation GET responses.
• [v30.2] Fixed a number of edge cases in diversity search.
• [v30.2] Fixed union search deduplication so curated pinned hits are no longer collapsed.
• [v30.2] Fixed a bug related to reference faceting.
• [v30.2] Fixed vector search behavior so zero-match phrase queries still preserve vector search results.
• [v30.2] Fixed scoped API key handling so embedded collection parameters can supply a missing collection in multi-search requests.
• [v30.2] Fixed conversation search error handling so failed searches do not incorrectly reuse the first search collection.
• [v30.2] Fixed search cache keying so scoped API key parameters and request-specific embedded params do not collide in the cache.
• [v30.2] Fixed JSON response escaping by using nlohmann::json::dump() when serializing response payloads.
• [v30.2] Fixed /health responsiveness during heavy bulk inserts by routing health checks to the meta thread pool.
• [v30.2] Fixed HTTP/2 async import teardown on early exit.

<p align="center">

<a href="https://typesense.org">

<picture>

<source media="(prefers-color-scheme: dark)" srcset="assets/typesense_logo_dark.svg">

<img src="assets/typesense_logo.svg" alt="Typesense" width="600" />

</picture>

</a>

</p>

<p align="center">

Typesense is a fast, typo-tolerant search engine for building delightful search experiences.

</p>

<p align="center">

An Open Source Algolia Alternative & <br>

An Easier-to-Use ElasticSearch Alternative

</p>

<p align="center">

<!-- <a href="https://circleci.com/gh/typesense/typesense"><img src="https://circleci.com/gh/typesense/typesense.svg?style=shield&circle-token=1addd775339738a3d90869ddd8201110d561feaa"></a> -->

<a href="https://hub.docker.com/r/typesense/typesense/tags"><img src="https://img.shields.io/docker/pulls/typesense/typesense"></a>

<a href="https://github.com/typesense"><img src="https://img.shields.io/github/stars/typesense/typesense?label=github%20stars&style=flat"></a><br>

<a href="https://cloud.typesense.org"><img src="https://img.shields.io/badge/searches_per_month_on_typesense_cloud-10 Billion-blue"></a>

<p>

<p align="center">

<a href="https://typesense.org">Website</a> |

<a href="https://typesense.org/docs/">Documentation</a> |

<a href="https://typesense.link/roadmap">Roadmap</a> |

<a href="https://typesense.link/slack-community">Slack Community</a> |

<a href="https://threads.typesense.org/">Community Threads</a> |

<a href="https://twitter.com/typesense">Twitter</a>

</p>

<br>

<p align="center">

<img src="assets/typesense_books_demo.gif?raw=true" alt="Typesense Demo" width="459" />

</p>

✨ Here are a couple of live demos that show Typesense in action on large datasets:

• Search a 32M songs dataset from MusicBrainz: songs-search.typesense.org
• Search a 28M books dataset from OpenLibrary: books-search.typesense.org
• Search a 2M recipe dataset from RecipeNLG: recipe-search.typesense.org
• Search 1M Git commit messages from the Linux Kernel: linux-commits-search.typesense.org
• Spellchecker with type-ahead, with 333K English words: spellcheck.typesense.org
• An E-Commerce Store Browsing experience: ecommerce-store.typesense.org
• GeoSearch / Browsing experience: airbnb-geosearch.typesense.org
• Search / Browse xkcd comics by topic: xkcd-search.typesense.org
• Semantic / Hybrid search on 300K HN comments: hn-comments-search.typesense.org

🗣️ 🎥 If you prefer watching videos:

• Here's one where we introduce Typesense and show a walk-through: https://youtu.be/F4mB0x_B1AE?t=144
• Check out Typesense's recent mention during Google I/O Developer Keynote: https://youtu.be/qBkyU1TJKDg?t=2399
• Here's one where one of our community members gives an overview of Typesense and shows you an end-to-end demo: https://www.youtube.com/watch?v=kwtHOkf7Jdg

Quick Links

• Features
• Benchmarks
• Roadmap
• Who's using this
• Install
• Quick Start
• Step-by-step Walk-through
• API Documentation
• API Clients
• Search UI Components
• FAQ
• Support
• Contributing
• Getting Latest Updates
• Build from Source

Features

• Typo Tolerance: Handles typographical errors elegantly, out-of-the-box.
• Simple and Delightful: Simple to set-up, integrate with, operate and scale.
• ⚡ Blazing Fast: Built in C++. Meticulously architected from the ground-up for low-latency (<50ms) instant searches.
• Tunable Ranking: Easy to tailor your search results to perfection.
• Sorting: Dynamically sort results based on a particular field at query time (helpful for features like "Sort by Price (asc)").
• Faceting & Filtering: Drill down and refine results.
• Grouping & Distinct: Group similar results together to show more variety.
• Federated Search: Search across multiple collections (indices) in a single HTTP request.
• Geo Search: Search and sort by results around a latitude/longitude or within a bounding box.
• Vector Search: Index embeddings from your machine learning models in Typesense and do a nearest-neighbor search. Can be used to build similarity search, semantic search, visual search, recommendations, etc.
• Semantic / Hybrid Search: Automatically generate embeddings from within Typesense using built-in models like S-BERT, E-5, etc or use OpenAI, PaLM API, etc, for both queries and indexed data. This allows you to send JSON data into Typesense and build an out-of-the-box semantic search + keyword search experience.
• Conversational Search (Built-in RAG): Send questions to Typesense and have the response be a fully-formed sentence, based on the data you've indexed in Typesense. Think ChatGPT, but over your own data.
• Natural Language Search: LLM-powered intent detection & query understanding, that converts any free-form natural language phrases into structured filters, sorts and queries.
• Image Search: Search through images using text descriptions of their contents, or perform similarity searches, using the CLIP model.
• Voice Search: Capture and send query via voice recordings - Typesense will transcribe (via Whisper model) and provide search results.
• Scoped API Keys: Generate API keys that only allow access to certain records, for multi-tenant applications.
• JOINs: Connect one or more collections via common reference fields and join them during query time. This allows you to model SQL-like relationships elegantly.
• Synonyms: Define words as equivalents of each other, so searching for a word will also return results for the synonyms defined.
• Curation & Merchandizing: Boost particular records to a fixed position in the search results, to feature them.
• Raft-based Clustering: Setup a distributed cluster that is highly available.
• Seamless Version Upgrades: As new versions of Typesense come out, upgrading is as simple as swapping out the binary and restarting Typesense.
• No Runtime Dependencies: Typesense is a single binary that you can run locally or in production with a single command.

Don't see a feature on this list? Search our issue tracker if someone has already requested it and add a comment to it explaining your use-case, or open a new issue if not. We prioritize our roadmap based on user feedback, so we'd love to hear from you.

Roadmap

Here's Typesense's public roadmap: https://typesense.link/roadmap.

The first column also explains how we prioritize features, how you can influence prioritization and our release cadence.

Benchmarks

• A dataset containing 2.2 Million recipes (recipe names and ingredients):
• Took up about 900MB of RAM when indexed in Typesense
• Took 3.6mins to index all 2.2M records
• On a server with 4vCPUs, Typesense was able to handle a concurrency of 104 concurrent search queries per second, with an average search processing time of 11ms.
• A dataset containing 28 Million books (book titles, authors and categories):
• Took up about 14GB of RAM when indexed in Typesense
• Took 78mins to index all 28M records
• On a server with 4vCPUs, Typesense was able to handle a concurrency of 46 concurrent search queries per second, with an average search processing time of 28ms.
• With a dataset containing 3 Million products (Amazon product data), Typesense was able to handle a throughput of 250 concurrent search queries per second on an 8-vCPU 3-node Highly Available Typesense cluster.

We'd love to benchmark with larger datasets, if we can find large ones in the public domain. If you have any suggestions for structured datasets that are open, please let us know by opening an issue. We'd also be delighted if you're able to share benchmarks from your own large datasets. Please send us a PR!

Who's using this?

Typesense is used by a range of users across different domains and verticals.

On Typesense Cloud we serve more than 10 BILLION searches per month. Typesense's Docker images have been downloaded over 12M times.

We've recently started documenting who's using it in our Showcase.

If you'd like to be included in the list, please feel free to edit SHOWCASE.md and send us a PR.

You'll also see a list of user logos on the Typesense Cloud home page.

Install

Option 1: You can download the binary packages that we publish for

Linux (x86_64 & arm64) and Mac (x86_64).

Option 2: You can also run Typesense from our official Docker image.

Option 3: Spin up a managed cluster with Typesense Cloud:

<a href="https://cloud.typesense.org"><img src="assets/deploy_with_typesense_cloud.svg?raw=true" alt="Deploy with Typesense Cloud" height="60" /></a>

Quick Start

Here's a quick example showcasing how you can create a collection, index a document and search it on Typesense.

Let's begin by starting the Typesense server via Docker:

docker run -p 8108:8108 -v/tmp/data:/data typesense/typesense:29.0 --data-dir /data --api-key=Hu52dwsas2AdxdE

We have API Clients in a couple of languages, but let's use the Python client for this example.

Install the Python client for Typesense:

pip install typesense

We can now initialize the client and create a companies collection:

import typesense

client = typesense.Client({
  'api_key': 'Hu52dwsas2AdxdE',
  'nodes': [{
    'host': 'localhost',
    'port': '8108',
    'protocol': 'http'
  }],
  'connection_timeout_seconds': 2
})

create_response = client.collections.create({
  "name": "companies",
  "fields": [
    {"name": "company_name", "type": "string" },
    {"name": "num_employees", "type": "int32" },
    {"name": "country", "type": "string", "facet": True }
  ],
  "default_sorting_field": "num_employees"
})

Now, let's add a document to the collection we just created:

document = {
 "id": "124",
 "company_name": "Stark Industries",
 "num_employees": 5215,
 "country": "USA"
}

client.collections['companies'].documents.create(document)

Finally, let's search for the document we just indexed:

search_parameters = {
  'q'         : 'stork',
  'query_by'  : 'company_name',
  'filter_by' : 'num_employees:>100',
  'sort_by'   : 'num_employees:desc'
}

client.collections['companies'].documents.search(search_parameters)

Did you notice the typo in the query text? No big deal. Typesense handles typographic errors out-of-the-box!

Step-by-step Walk-through

A step-by-step walk-through is available on our website here.

This will guide you through the process of starting up a Typesense server, indexing data in it and querying the data set.

API Documentation

Here's our official API documentation, available on our website: https://typesense.org/api.

If you notice any issues with the documentation or walk-through, please let us know or send us a PR here: https://github.com/typesense/typesense-website.

API Clients

While you can definitely use CURL to interact with Typesense Server directly, we offer official API clients to simplify using Typesense from your language of choice. The API Clients come built-in with a smart retry strategy to ensure that API calls made via them are resilient, especially in an HA setup.

• JavaScript
• PHP
• Python
• Ruby

If we don't offer an API client in your language, you can still use any popular HTTP client library to access Typesense's APIs directly.

Here are some community-contributed clients and integrations:

• Go
• .Net
• Java
• Rust
• Dart
• Perl
• Swift
• Clojure
• python orm client
• PHP SEAL Adapter
• Elixir

We welcome community contributions to add more official client libraries and integrations. Please reach out to us at contact@typsense.org or open an issue on GitHub to collaborate with us on the architecture. 🙏

Framework Integrations

We also have the following framework integrations:

• Laravel
• Firebase
• Gatsby
• WordPress
• WooCommerce
• Symfony
• InstantSearch
• [DocSearch](https://types