Avarana Architecture

Avarana is an API gateway designed to sit between your applications and external LLM providers (OpenAI, Anthropic, Azure, Bedrock). It provides three core capabilities:

• PII Redaction: Detect and mask sensitive data before it leaves your network
• Smart Routing: Route queries to cost-appropriate models based on complexity
• Audit Logging: Log every request for compliance and debugging

The gateway is built with Python and FastAPI, designed for deployment in European infrastructure to meet data residency requirements.

The core gateway is a FastAPI application that proxies requests to LLM providers. It exposes an OpenAI-compatible API, allowing existing applications to connect with minimal code changes.

Configuration is done via YAML. You specify which providers to support, your API keys (BYOK), and policies for redaction and routing.

# Avarana Gateway Configuration
providers:
  openai:
    base_url: https://api.openai.com/v1
    api_key: ${OPENAI_API_KEY}
  anthropic:
    base_url: https://api.anthropic.com
    api_key: ${ANTHROPIC_API_KEY}

policies:
  redaction:
    enabled: true
    types: [PERSON, EMAIL, PHONE, SSN, CREDIT_CARD]
  routing:
    enabled: true
    default_model: gpt-4o
    cost_threshold: 0.4  # Route simple queries to cheaper models

The gateway can be deployed as a Docker container, a Kubernetes sidecar, or directly on a VM. For European data residency, deploy in your chosen European region.

The redaction engine uses Microsoft Presidio for named entity recognition. It scans incoming prompts for PII patterns (names, emails, phone numbers, SSNs, credit card numbers) and replaces them with placeholder tokens.

Placeholder tokens are deterministic. The same input always produces the same token. This preserves referential integrity in the prompt (if "John" appears twice, both instances become the same placeholder).

After the LLM responds, the gateway can optionally rehydrate placeholders with the original values before returning the response to your application.

Not every query needs GPT-5. The routing engine analyzes prompt complexity and routes simple queries (summarization, classification, basic Q&A) to cost-effective models like Llama 3 or Claude Haiku.

Complexity scoring is rule-based, looking at prompt length, presence of code, multi-step reasoning indicators, and domain-specific keywords. You can also define explicit routing rules in your config.

routing:
  rules:
    - name: simple-queries
      condition:
        max_tokens: 100
        no_code: true
      route_to: llama-3-70b
      
    - name: code-generation
      condition:
        has_code_markers: true
      route_to: gpt-4o
      
    - name: default
      route_to: gpt-4o

In typical enterprise workloads, 50-70% of queries can be routed to cheaper models with equivalent output quality, reducing LLM costs significantly.

Every request through the gateway is logged with metadata: timestamp, user/application ID, model used, token counts, latency, and cost. Prompts and responses can optionally be logged (redacted) for debugging.

Logs are written to a destination you control, such as a local database, S3-compatible storage, or forwarded to your SIEM. For European deployments, logs stay within your chosen jurisdiction.

{
  "timestamp": "2026-01-15T14:32:01Z",
  "request_id": "req_abc123",
  "user_id": "user_456",
  "model_requested": "gpt-4o",
  "model_routed": "llama-3-70b",
  "input_tokens": 127,
  "output_tokens": 89,
  "latency_ms": 342,
  "cost_usd": 0.0002,
  "pii_detected": ["PERSON", "EMAIL"],
  "pii_redacted": true
}

Avarana is designed for self-hosting in your infrastructure. Deployment options:

• Docker: Single container, suitable for development and small deployments
• Kubernetes: Helm chart for production deployments with autoscaling
• Managed Cloud: We run the infrastructure in your chosen European region

version: '3.8'
services:
  avarana:
    image: avarana/gateway:latest
    ports:
      - "8080:8080"
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
    volumes:
      - ./avarana.yaml:/app/config.yaml
      - ./logs:/app/logs