AISBF Logo AISBF

AI Service Broker Framework — AI Should Be Free

AISBF architecture

AISBF Network Architecture Diagram

Dark technical architecture diagram for AISBF production, clustered deployments, provider routing, Redis/MySQL state, CoderAI broker workers, and RunPod endpoints.

Start FreeMore tutorials
Production boundary: public TLS edge + private AISBF service network Client appsOpenAI SDK / curl TLS edgenginx / ingress AISBF node AFastAPI :17765 AISBF node Bsame config DB/cache MySQLusers, tokens, routes Rediscache + broker state Dashboardproviders/routes/users Route layerprovider → rotation → autoselect Provider handlersOpenAI, Google, Anthropic,Ollama, OAuth providers CoderAI brokerregistration + websocketcluster-safe worker reachabilityextension-dependent RunPod GPUOpenAI-compatible /v1 Local Ollamaprivate LAN models Hosted APIsOpenAI etc. HTTPSbroker state dashboard/frontendAISBF backenddatabasemessage/cache/brokersecurity edge

Reading the graph

North-south traffic

Clients enter through HTTPS at the reverse proxy, which forwards to one or more AISBF FastAPI nodes on port 17765. Proxy headers preserve scheme, host, client IP, and optional subpath.

Control plane

The dashboard writes providers, rotations, autoselects, users, tokens, tiers, and settings into durable configuration storage. In clusters, that durable layer should be shared.

Data plane

Provider handlers translate OpenAI-compatible requests to the target backend: hosted APIs, Ollama/local models, RunPod OpenAI-compatible endpoints, or extension-backed broker workers.

Reliability layer

Rotations and autoselect routes sit above providers. They absorb provider churn, rate limits, and policy choices so applications can call stable model names.

Sequence: one chat request

1. App calls /api/user/chat/completions with a user token.
2. AISBF authenticates token and loads that user's providers, rotations, and autoselects.
3. If model is a provider model, AISBF calls that provider directly.
4. If model is a rotation, AISBF picks a weighted available provider/model and applies cooldown rules.
5. If model is autoselect, AISBF classifies/describes the request and chooses a candidate or fallback.
6. Provider handler streams or returns normalized OpenAI-style output.
7. Analytics, rate-limit state, context/cache metadata, and usage records are updated.

When to add cluster complexity

Stay single-node until uptime, traffic, or operational separation demands more. The clean scaling path is: one node + SQLite; one node + MySQL/Redis; two AISBF nodes behind a load balancer; then brokered GPU workers and externalized OAuth/credential storage if needed.