Performance & Reliability

Latency Benchmarks

All measurements are from production traffic on NVIDIA A10G GPUs running Mistral 7B v0.2 in 4-bit GPTQ quantization. These are real numbers from live deployments, not synthetic benchmarks. Latency varies by prompt length, output length, and concurrent load.

Throughput

Throughput depends on your plan tier, model size, and the number of hives allocated. Each hive processes requests in parallel across its GPU allocation. Adding hives scales throughput linearly.

Uptime SLA

Alveare commits to 99.95% monthly uptime for Professional and Scale plans, and 99.9% for Starter plans. Enterprise plans can negotiate 99.99% SLAs. If we miss the target, you receive service credits automatically -- no support ticket required.

Uptime History (Last 12 Months)

Live uptime data available at status.alveare.ai

Self-Healing Architecture

Alveare's supervision tree architecture is borrowed from Erlang/OTP, the system that powers telecom infrastructure running at 99.999% uptime. Every specialist process runs under a supervisor. If a specialist crashes -- due to OOM, malformed input, GPU error, or any runtime failure -- the supervisor restarts it automatically in under 100 milliseconds.

Health Monitoring

Beyond crash recovery, the health monitor continuously tracks output quality metrics for each specialist. It detects degradation before it affects your users and takes corrective action automatically.

• Latency drift detection. If P50 latency exceeds the configured threshold for 5 consecutive minutes, the monitor triggers a model reload or specialist restart.
• Output format validation. For specialists with structured output (JSON, labels), the monitor validates that 95%+ of outputs match the expected schema. Below that threshold, it triggers an alert.
• Error rate tracking. If more than 2% of requests return errors in a 5-minute window, the supervisor restarts the specialist. If restarts do not resolve the issue, the circuit breaker activates and alerts the operations team.
• Token efficiency monitoring. A sudden increase in average output tokens may indicate prompt injection or model degradation. The monitor flags anomalies that exceed 2 standard deviations from the rolling average.

Auto-Scaling

Alveare's orchestration layer monitors request queue depth, GPU utilization, and latency metrics to scale your hive capacity automatically. When demand increases, additional GPU instances are provisioned from the spot fleet. When demand decreases, excess capacity is released to keep costs low.

Scaling Triggers

• Queue depth. When the request queue exceeds 50 pending requests for more than 30 seconds, a new instance is provisioned. Target queue depth: under 10 requests.
• GPU utilization. When average GPU utilization exceeds 85% for 5 minutes, scaling is triggered. Target utilization: 60-80%.
• Latency threshold. When P95 latency exceeds the plan-specific threshold (500ms for Starter, 400ms for Professional/Scale), scaling is triggered regardless of GPU utilization.
• Predictive scaling. For customers with regular traffic patterns (e.g., business hours spike), the orchestrator pre-provisions capacity 15 minutes before the expected spike based on 7-day rolling traffic patterns.

Scale-up time is typically 45-90 seconds (instance provisioning + model loading + warmup). During scale-up, existing instances continue serving traffic. Scale-down happens gradually with a 10-minute cooldown to prevent oscillation.

Spot Termination Handling

Alveare runs inference on GPU spot instances to deliver 60-80% lower infrastructure costs compared to on-demand pricing. Spot instances can be reclaimed by AWS with a 2-minute warning. Alveare handles this transparently with zero dropped requests and zero downtime.

The standby pool ensures there is always a warm instance ready to accept traffic. The default standby ratio is 1 standby per 2 active instances. For customers requiring the highest availability, the ratio can be increased to 1:1. Standby instances are also spot instances, so the cost overhead is minimal.

Caching Layer

Alveare includes a built-in response cache that stores results for identical requests. The cache sits between the API gateway and the inference layer, returning cached responses in under 10ms without consuming GPU compute.

Cache TTL is configurable per specialist (default: 1 hour, range: 0 to 24 hours). The cache key is a SHA-256 hash of the specialist name, prompt text, and all generation parameters. Changing any parameter produces a cache miss, ensuring you always get fresh results when parameters change. Cached responses do not count against your monthly request allocation.

Global Infrastructure

Alveare currently operates in AWS us-east-1 (N. Virginia) with eu-west-1 (Ireland) planned for Q2 2026. Enterprise customers can request deployment in any AWS region with GPU instance availability, including ap-northeast-1 (Tokyo) and ap-southeast-1 (Singapore).

For customers outside the US, latency from Europe to us-east-1 is typically 80-120ms round trip. The eu-west-1 region will reduce this to 10-30ms for European users. The API gateway uses anycast routing to direct traffic to the nearest edge location for TLS termination, minimizing connection overhead.