02/work/ultrabalancer
ultrabalancer
A small, inspectable data path built to stay fast when connection counts stop being friendly.
- role
- Creator & maintainer
- period
- 2025 — present
- project
- view the source
I built the Rust data path, upstream selection, connection handling, health state, configuration, metrics, admin API, packaging, and documentation. The goal was not only a fast benchmark, but a load balancer someone could actually install and operate.
the problem
A fast forwarding loop is easy to demonstrate in isolation. A useful load balancer also needs health checks, predictable connection behavior, dynamic backends, metrics, packaging, and failure handling.
I wanted the operational surface to remain understandable instead of hiding the hot path behind a large control plane.
constraints
- Keep shared state and allocation off the request path where practical.
- Make unhealthy upstreams disappear from selection quickly without causing every worker to contend on one lock.
- Expose enough metrics and control to debug the process without turning it into a platform.
- Ship installable builds across Linux, macOS, and Windows.
architecture
- listeneraccept and classify connections
- runtimeTokio tasks and pooled connections
- selectorround robin, least connections, hash, random, weighted
- healthactive checks and backend state
- upstreamforward request and stream response
- ops/metrics, /health, admin API
stack · Rust · Tokio · HTTP/2 · Prometheus · Cross-platform releases
decisions
- Separate selection from health mutation
The request path reads a compact backend view. Health checks update that view out of band, reducing the amount of coordination each forwarded request needs.
- Make algorithms boring to swap
Round robin, least connections, IP hash, random, and weighted selection share a small interface. The hot path does not need to know why a backend won.
- Operations are part of performance
A binary that is fast but opaque is expensive during an incident. Health, metrics, dynamic backend management, packages, and documentation shipped as product features.
- Publish the workload with the number
Throughput without connection count, duration, hardware, and latency is marketing. The repository reports its benchmark as 10,000 concurrent connections over 30 seconds and keeps the result hardware-specific.
results
The public repository documents the implemented features, workload shape, platform packages, and current 500K+ project claim. These are project-reported results, not an independent benchmark.
failures & lessons
- My first article overclaimed
An early write-up described a different C/C++ design and a one-million-RPS result that the current Rust repository did not reproduce. I replaced it with the implemented architecture and the benchmark the project actually publishes.
- Tail latency beats the headline
Average latency can look excellent while a small group of requests waits behind connection churn or a sick upstream. P99 and failure behavior deserve equal space beside throughput.
- The benchmark is not production
A synthetic forwarding test proves a narrow data-path property. TLS, real payloads, upstream variance, observability, and failure recovery still need workload-specific tests.