Arch

Arch is an advanced infrastructure tool designed to help developers create fast and personalized AI agents quickly. It serves as an intelligent gateway that integrates prompts with APIs using Large Language Models (LLMs). Arch enhances the development process by offering secure handling, robust observability, and seamless API integration, all managed outside the core business logic.

Engineered by the core contributors of Envoy Proxy, Arch leverages Envoy's proven HTTP management capabilities to handle critical tasks such as prompt processing, API function calling, and parameter extraction. It centralizes safety measures to prevent jailbreak attempts and ensures safe user interactions without requiring additional code. Additionally, Arch manages LLM calls with features like smart retries and resilient upstream connections to ensure continuous availability.

Arch supports comprehensive observability by adhering to the W3C Trace Context standard, enabling complete request tracing across applications. It provides metrics to monitor latency, token usage, and error rates, optimizing AI application performance. This makes it an ideal solution for building robust, fast, and personalized AI agents while maintaining high standards of security and observability.