What is Higress?
Higress is a cloud-native, AI-native API gateway built on the open-source Istio and Envoy proxy, designed to provide a unified, high-performance solution for traffic management, service governance, and security in modern microservices and AI-driven architectures. Originating from over two years of Envoy Gateway practice within Alibaba, Higress integrates deeply with popular microservice ecosystems such as Dubbo, Nacos, and Sentinel, offering a cost-effective, easy-to-deploy gateway that combines traffic scheduling, service governance, and security protection into a single, cohesive platform.
Key features include seamless AI gateway capabilities that simplify the management of large language model (LLM) APIs and agent development. Higress supports protocol conversion for over 100 models, enabling unified access to diverse LLMs and intelligent fallback mechanisms. It enhances data security by filtering inputs to external models for privacy protection and filtering outputs for content safety. Semantic caching optimizes token usage and reduces latency for repetitive queries, while token tracking and rate limiting help manage usage quotas and analyze consumption patterns.
Use cases span from enterprise AI middleware to financial data integration. For instance, Ctrip leverages Higress to manage multi-model access and traffic governance, while Junrun Human Resources operates over 1,000 digital agents 24/7, saving millions in labor costs. Higress also excels in MCP (Model Context Protocol) service hosting, providing authentication, traffic scheduling, and security auditing for financial data APIs, as demonstrated by Today Investment's MCP marketplace.
Technical highlights include a WASM-based plugin system for easy extension, multi-registry integration, and support for both open-source and commercial versions. Higress offers a production-ready AI gateway that accelerates AI adoption by unifying model, tool, and agent proxy management, making it an ideal choice for building AI platforms and enabling efficient, secure, and scalable AI deployments.
Use Cases
Key Features
cloud native API gateway AI native capabilities Istio and Envoy integration unified traffic management service governance security protection LLM API management protocol conversion for 100+ models semantic caching token tracking and rate limitingWho is it for?
DevOps engineers, SREs, cloud architects, platform teams, AI/ML engineers, microservices developers, security engineers
