LLM Routers & Gateways
Compare 114 llm routers & gateways tools to find the right one for your needs
📂 Subcategories
🔧 Tools
Compare and find the best llm routers & gateways for your needs
TrueFoundry
An end-to-end platform for building, deploying, and monitoring generative AI applications and machine learning models.
Portkey.ai
An AI gateway and observability suite to build reliable, cost-effective, and fast AI applications.
Helicone
An open-source AI gateway and observability platform for building reliable AI applications.
Helicone
An open-source platform that provides observability, caching, and routing for large language models.
Zuplo
A developer-focused API gateway that can be programmed to add features like model routing and fallbacks for AI services.
Helicone
An open-source observability platform and AI gateway for LLM applications.
Orchestry
A platform for governing and managing Microsoft 365 environments, including Teams and SharePoint.
Helicone
An open-source platform for monitoring and debugging large language model applications.
Vercel AI Gateway
A gateway that provides a unified API for accessing various AI models, with a focus on performance and developer experience.
Portkey
An AI gateway and observability suite for building reliable, cost-efficient, and fast AI applications.
Zuplo
A developer-friendly API gateway that can be used to build custom AI gateways with features like rate limiting, authentication, and transformations.
Portkey
An AI gateway and observability platform that provides routing, fallbacks, and cost management for LLMs.
Orkes Conductor
A workflow orchestration platform based on Netflix Conductor that can be used to build and manage complex AI workflows with model fallback logic.
Portkey.ai
A comprehensive platform for building, deploying, and managing LLM applications with a focus on observability and reliability.
Portkey
An AI gateway and observability platform for building reliable and cost-effective LLM applications.
TrueFoundry
An AI gateway designed for enterprises that need production-ready AI with a focus on security, compliance, and orchestration.
Vercel AI Gateway
An AI gateway that provides observability, reliability, and management for AI applications built on Vercel.
Tyk AI Gateway
An AI gateway that provides centralized governance, security, and control over AI and LLM services.
KrakenD
An open-source, high-performance API Gateway that can be configured to act as a gateway for AI and LLM services.
Unify AI
An AI-native platform that helps go-to-market teams with tasks like prospecting and personalization, with underlying routing and fallback capabilities.
Aisera LLM Gateway
An enterprise LLM gateway focused on security, governance, and responsible AI.
Gloo Gateway
A feature-rich, Kubernetes-native API gateway with support for AI and LLM integrations.
Cloudflare AI Gateway
A gateway that provides observability, reliability, and scalability for AI applications.
Tyk
An open-source API gateway that can be used to manage and secure AI APIs with a focus on flexibility and performance.
Cloudflare AI Gateway
A gateway that provides visibility and control over AI applications, with features like caching, rate limiting, and model fallbacks.
Gravitee.io
An open-source API management platform with a focus on both synchronous and asynchronous APIs, suitable for managing AI services.
F5 NGINX Management Suite
A centralized management platform for NGINX instances that can be used to configure and monitor API gateways for AI services.
Martian
An enterprise AI platform that includes capabilities for model routing and management to ensure the reliability of AI applications.
Cloudflare AI Gateway
A gateway that provides centralized visibility and control for AI applications, with features like caching, rate limiting, and analytics.
Kong AI Gateway
An enterprise-grade AI gateway for securing, managing, and extending AI-powered applications.
Cloudflare AI Gateway
A gateway that provides insights, reliability, and control for AI applications, integrated with the Cloudflare network.
MuleSoft Anypoint Platform
A comprehensive integration platform that can be used to manage and secure AI APIs as part of a broader integration strategy.
Kong AI Gateway
An AI gateway built on the popular Kong Gateway that provides features for managing and securing AI services.
Kong AI Gateway
An enterprise-grade AI gateway for securing, managing, and observing AI traffic and LLM consumption.
Databricks AI Platform
A comprehensive platform that unifies data engineering, machine learning, and generative AI for enterprise-scale applications.
Cloudflare AI Gateway
A managed service for monitoring, controlling, and optimizing AI applications.
Kong AI Gateway
A comprehensive set of AI features built on top of the Kong Gateway for managing AI traffic.
Kong AI Gateway
An extension of the Kong API Gateway that provides features for managing, securing, and observing AI traffic.
Apigee
A comprehensive API management platform that can be used to manage, secure, and analyze AI and ML APIs.
AWS API Gateway
A fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.
DreamFactory
A platform that generates secure and documented REST APIs from any data source, which can be used to create an AI gateway.
WSO2 API Manager
An open-source API management platform that can be used to create, manage, and secure AI APIs.
Axway Amplify
An open API management platform that can be used to manage and secure AI APIs across multiple gateways, clouds, and environments.
Apigee API Management
A comprehensive API management platform from Google Cloud that can be used to manage and secure access to AI and LLM services.
WSO2 API Manager
A comprehensive, open-source platform for full lifecycle API management, including capabilities for managing AI services.
DreamFactory
An API generation platform that can be used to create and manage REST APIs for various data sources, which can then be used to power AI applications.
Axway AMPLIFY API Management
A comprehensive API management platform that helps organizations manage the full lifecycle of their APIs, including those for AI services.
IBM API Connect
A comprehensive API management platform that helps organizations create, manage, secure, and socialize APIs.
Azure API Management
A comprehensive platform for managing APIs, including those for AI and ML services, with features for security, analytics, and developer engagement.
Layer7 API Gateway
An enterprise-grade API gateway from Broadcom that provides robust security and management capabilities for APIs, including those for AI services.
IBM API Connect AI Gateway
An AI gateway feature within IBM API Connect for managing and controlling access to AI services.
Bifrost by Maxim AI
A high-performance, open-source LLM gateway built in Go for enterprise-scale deployments.
BricksLLM
A cloud-native AI gateway in Go for fine-grained access control and monitoring of LLMs.
Unify AI
An AI platform for workflow automation, intelligent agents, and cross-application AI integration.
LiteLLM
An open-source library that provides a unified interface to interact with over 100 LLM providers.
Unify.ai
A platform that provides a single API to access a wide range of large language models.
Bifrost by Maxim AI
A unified API that allows developers to access and switch between various large language models with a single line of code.
Martian
A model router that intelligently directs requests to the optimal large language model based on performance, cost, and other factors.
RouteLLM
An open-source framework for building and evaluating LLM routers that can dynamically select between different models.
Orq.ai
A platform for teams to build, manage, and collaborate on generative AI applications.
BricksLLM
An open-source, cloud-native AI gateway written in Go, designed for production workloads.
TensorZero
An open-source platform that unifies the entire LLMOps stack, from gateway to observability and optimization.
Bifrost
A high-performance, open-source LLM gateway built in Go for production-grade AI systems.
LiteLLM
An open-source Python library that simplifies access to over 100 LLM providers with a unified API.
OpenRouter
A unified API that provides access to a wide range of AI models, automatically routing requests to the most cost-effective options.
Unify AI
A platform that provides a unified API for accessing various LLMs, along with tools for fine-tuning and deployment.
BricksLLM
A cloud-native AI gateway written in Go, designed for putting large language models into production.
Martian
A platform for building, managing, and deploying AI models, with features for routing and orchestration.
Pruna AI
A platform for optimizing and deploying AI models, with features for efficient routing and serving.
ModelPilot
An AI model router that helps you choose the best model for your needs based on performance, cost, and other factors.
Orchesty
An AI orchestration platform that includes a powerful routing engine for managing and directing AI traffic.
Portkey
A purpose-built AI gateway that connects applications to over 250 LLMs through a unified API.
TrueFoundry
A Kubernetes-native AI infrastructure platform with a low-latency AI Gateway for deploying, scaling, and managing advanced AI workloads.
Helicone
An open-source AI API gateway and observability platform that provides a unified API to over 100 models.
Bifrost by Maxim AI
A high-performance, open-source LLM gateway built in Go, designed for speed and scalability.
LiteLLM
An open-source Python library that provides a unified interface to over 100 LLM providers.
OpenRouter
A unified API platform that provides access to over 400 AI models from dozens of providers through a single endpoint.
Kong AI Gateway
An enterprise-grade AI gateway for securing, governing, and controlling LLM consumption from all popular AI providers.
LLM Gateway
An open-source, unified API gateway for routing, managing, and analyzing requests to various LLM providers.
BricksLLM
A cloud-native, open-source AI gateway written in Go that serves as a proxy to multiple LLM providers.
Unify.ai
Provides a unified interface to a wide array of LLMs, focusing on simplicity and ease of integration.
Martian
A model router that helps you get the best outcomes from your LLMs by routing queries to the most suitable model.
RouteLLM
An open-source toolkit from LMSYS that enables users to direct tasks among various LLMs to enhance resource management and efficiency.
NVIDIA AI Blueprint for an LLM Router
A framework designed to intelligently direct prompts to the most appropriate model, ensuring an optimal balance between reasoning depth and computational efficiency.
Paddler
An open-source platform that allows you to run inference, deploy, and scale LLMs on your own infrastructure.
Olla
A high-performance proxy and load balancer that provides intelligent routing, automatic failover, and unified model discovery across local and remote inference backends.
nexos.ai
An AI gateway service that acts as a centralized hub for enterprise AI operations, focusing on intelligent model routing, security, and compliance.
Cequence AI Gateway
An enterprise-focused AI gateway with a strong emphasis on security, providing real-time monitoring, authentication, authorization, and guardrails.
Anyscale
A configurable AI platform that unifies tools and infrastructure to accelerate the development, deployment, and scaling of AI and Python applications.
LangChain (with load balancing)
A framework for developing applications powered by language models, with experimental features for load balancing across multiple LLM providers.
Pruna AI
An AI optimization engine designed to make machine learning models more efficient in terms of speed, size, and cost.
ModelPilot
A tool that routes LLM requests to the optimal AI model based on cost, quality, and speed.
OpenRouter
A unified interface to access and switch between hundreds of large language models with built-in fallback and routing.
LiteLLM
An open-source library that provides a unified interface to over 100 LLMs with features like fallbacks and spend tracking.
Bifrost
A high-performance LLM gateway built in Go for ultra-low latency and zero-config deployment.
LiteLLM
An open-source Python library that provides a unified interface to over 100 LLM providers.
OpenRouter
A unified API platform that provides access to a wide range of AI models with intelligent routing.
Unify AI
A platform offering a suite of AI tools for workflow automation and cross-application integration.
BricksLLM
A cloud-native AI gateway written in Go for fine-grained access control and monitoring of LLM usage.
TensorZero
An open-source platform that unifies an LLM gateway, observability, optimization, and evaluation for LLM applications.
Martian
A platform featuring a model router that dynamically routes prompts to the best LLM for optimal quality and cost.
Requesty.ai
A centralized gateway that intelligently routes LLM requests across multiple AI providers to optimize for performance, cost, and reliability.
Credal.ai
A platform for building secure, access-controlled Retrieval Augmented Generation (RAG) applications on top of enterprise data.
Vercel AI SDK
An open-source, TypeScript-first library for building AI-powered user interfaces and applications.
getit.ai
NA
One AI
NA
PromptLayer
NA
Valdi
NA
Layerup
NA
GPTCache
An open-source semantic cache for storing LLM responses to reduce costs and latency.
TrueFoundry LLM Gateway
A secure and scalable AI gateway for managing and deploying LLMs in production.
Helicone
An open-source platform for monitoring, debugging, and improving LLM applications.
OpenRouter
A unified API to access hundreds of AI models from various providers.
LiteLLM
An open-source library that simplifies access to over 100 LLMs with a unified API.