CoderAI embeddings and reranking

One endpoint, inferred backend

POST /v1/embeddings keeps the OpenAI request shape. CoderAI works out what kind of model it is dealing with from the model itself, so adding a new embedder is a registry entry, not a code path.

Family	Serves
sentence-transformers / transformers	general text embeddings
BGE-M3	native multi-vector — `dense`, `sparse` lexical weights and `colbert` per-token vectors, in a single pass
CLIP / vision (DINOv2, ViT)	image embeddings
GME-Qwen2-VL	text and images in one shared space, loaded natively
llama.cpp GGUF	GGUF embedders, and vision GGUF with an `mmproj`
dinov2.cpp	GGUF DINOv2 through a small `dinov2-embed` subprocess (Vulkan or CPU)
GeoCLIP	`geoclip` (image) and `geoclip-location` (`"lat,lon"`) in one shared 512-d space
VPR	`vpr` / EigenPlaces (2048-d) and `salad` / DINOv2-SALAD (8448-d, isolated venv) for visual place recognition

Request options worth knowing

Field	Effect
`image`	An http(s) URL, a data URI, a local path or bare base64 — one or many. Image vectors are returned after the text ones.
`embedding_types`	Any of `dense`, `sparse`, `colbert`. The response gains `sparse_embedding` (`{indices, values}`) and `colbert_embedding` alongside `embedding`.
`dimensions`	Truncate to N dimensions.
`quantization`	TurboQuant: `turbo8` down to `turbo2`. With `encoding_format: "float"` you get lossy reconstructed vectors; with `base64` you get packed bytes plus a quantization metadata block.
`encoding_format`	`float` or `base64`.

{
  "model": "bge-m3",
  "input": ["contratto di locazione"],
  "embedding_types": ["dense", "sparse"]
}

Geolocation and place recognition

Two of these families exist for a specific job: working out where a photograph was taken.

GeoCLIP

Image ↔ coordinates

The image encoder and the location encoder project into the same 512-dimensional space, so a photo can be compared directly against candidate GPS coordinates. Post an image to geoclip, and "lat,lon" to geoclip-location. Posting the wrong input type to the wrong id is a clear error, not a silent bad vector.

EigenPlaces & SALAD

Same building, different photo

Visual place recognition matches two photographs of the same place taken from different angles, in different light, years apart — the problem where general image embeddings do poorly. Useful for matching a listing photo against street-level imagery.

Reranking

Retrieval finds fifty candidates cheaply; a cross-encoder then reads the query and each document together and orders them properly. POST /v1/rerank is that second stage.

{
  "model": "bge-reranker-v2-m3",
  "query": "termination clause",
  "documents": ["...", "...", "..."],
  "top_n": 5,
  "return_documents": true
}

Each result carries an index and a relevance_score between 0 and 1. Rerankers need no extra dependency — they run on native Transformers — and are registered in the embedding model category with capabilities: ["reranking"], so they share the same loading, caching, eviction and thermal gating as everything else.

Operational notes

Embedding models are loaded on demand, cached, and evicted under VRAM pressure like any other model.
Per-model admission control — embed_max_concurrency, embed_max_backlog, embed_min_interval_ms — keeps a bulk indexing job from starving interactive traffic.
Backends needing an incompatible dependency stack (DINOv2-SALAD, dinov2.cpp) run as self-healing subprocesses over a small JSON protocol rather than polluting the main environment.

AISBF