Control center / Discover
Browse by category instead of drowning in search results. Every category opens a submenu of subcategories with live counts, and checking one narrows the grid to exactly that slice.
Production ready inference server for LLMs optimized for throughput and cost efficiency.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Fast serving framework for LLMs with radix attention based prefix caching for lower cost.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
High throughput and memory efficient inference and serving engine for large language models.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Toolkit for optimizing transformer models for faster and cheaper inference on various hardware.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Python bindings for llama.cpp enabling low cost local large language model inference.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Run large language models locally to reduce inference costs and latency.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Efficient C plus plus inference of LLaMA models enabling cheaper on device execution.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
KV cache management layer that speeds up LLM serving by reusing cached context.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Deep learning optimization library enabling efficient large model training and inference.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Single file executables that run LLMs locally without heavy infrastructure or API cost.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Toolkit for optimizing and deploying large language models with high performance inference.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Optimizing inference proxy that applies techniques to improve accuracy and reduce LLM cost.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Fast inference library for running quantized LLMs on consumer GPUs at lower cost.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Open source LLM gateway and optimization framework unifying inference, observability and evals.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Open standard for instrumenting LLM applications to capture traces and spans for observability.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
A drop in replacement REST API that is compatible with OpenAI for local LLM inferencing.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Run your own AI cluster at home with everyday consumer devices to serve models locally.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Structured text generation library for large language models.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
A toolkit for compressing deploying and serving large language models locally with high throughput.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
A fast LLM inference engine written in Rust for local model deployment.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.
Fast inference server for serving text embedding models in production.
Best match because its name, category, capabilities, or owner matches “inference”. Adoption and trust break close ties.