AISBF
AI Service Broker Framework — AI Should Be Free
AISBFAI Service Broker Framework — AI Should Be Free
A production pipeline turns roughly ten million Italian court documents into structured, page-cited records — without a single page leaving its own infrastructure. CoderAI is the orchestration layer underneath: the GPU-pod control plane that runs a dozen model roles at once and keeps live cost, load and health in view.
The pipeline reads scanned judgments, decrees and legislative texts and emits clean records: who sued whom, on what grounds, with what outcome, each field cited back to the page it was read from. At any moment a dozen models are running — an OCR ensemble (chandra-ocr, surya-ocr, olmOCR-2), a vision model for layout, bge-m3 for dense and sparse retrieval, a rules engine and a fine-tuned extraction model. They retry, disagree and feed each other. The documents are privileged, so none of them can be shipped to a third-party API.
The reflex for a hard document task is to call a single frontier model and let it do everything. For legal records that is a bad trade: the data leaves your infrastructure, you pay frontier prices on every page, and the thing deciding what a document says is a black box you cannot inspect. So this pipeline does not. The work is split across small, self-hosted specialists, each trained for one stage. Extraction is the clearest case — one bounded task against a fixed schema, repeated millions of times — so it runs on a QLoRA adapter (4-bit base, a single A40) distilled from the pipeline’s own agreed output.
The payoff is the point of the whole design: the same quality you would get from one big general model, with full control over every stage, the data under your own roof the whole way, and a cost that drops by orders of magnitude on the high-volume stages.
So each pipeline stage is drawn as a live decision tree — a document enters at the top and the diagram shows every branch it can take and where it ends up, counters moving in real time. A stalled stage shows up as a branch that should be thin going fat, not as a stack trace an hour later. CoderAI’s control plane watches the other axis: the GPU fleet and the spend against budget.
Runs a dozen model roles across local and rented GPUs, with live $/hr against a daily budget and per-pod KV-cache, VRAM, temperature and health.
Every model runs on your own hardware. Privileged documents never leave your infrastructure and never reach a third-party API.
Serve, swap and gate small fine-tuned models per stage instead of paying for one large general model.
The full technical write-up covers the six stages end to end, the gold-set construction, the gate, and the real cost numbers.