Virtual models

• Combo (Fallback Chain) — Models tried sequentially. If one fails, the next in order is used.
• Round-Robin — Models selected in rotating order across requests. No fallback.

/admin/virtual-models

/api/v2/virtual-models

​

Management UI

​

List

​

Create

​

Edit

​

Delete

​

CRUD API

​

List virtual models

GET /api/v2/virtual-models?active_only=false

{
  "items": [
    {
      "id": "uuid",
      "name": "my-fallback-group",
      "type": "combo",
      "underlying_model_ids": ["uuid-1", "uuid-2"],
      "underlying_models": [
        {"id": "uuid-1", "name": "gpt-4o"},
        {"id": "uuid-2", "name": "claude-sonnet-4"}
      ],
      "is_active": true,
      "created_at": "2026-07-22T18:46:19"
    }
  ]
}

​

Create virtual model

POST /api/v2/virtual-models

{
  "name": "my-fallback-group",
  "type": "combo",
  "underlying_model_ids": ["uuid-1", "uuid-2"],
  "is_active": true
}

• type must be "combo" or "round_robin"
• All underlying_model_ids must reference existing real models (non-deleted, non-virtual)
• Name must be unique among virtual models

​

Update virtual model

PUT /api/v2/virtual-models/{virtual_model_id}

{
  "name": "renamed-group",
  "type": "round_robin",
  "underlying_model_ids": ["uuid-3", "uuid-4"],
  "is_active": false
}

​

Delete virtual model

DELETE /api/v2/virtual-models/{virtual_model_id}

{"status": "deleted"}

​

Exclusion from real model APIs

​

Runtime behavior via OpenAI proxy

​

Model resolution

1. The virtual model is loaded from the DB
2. underlying_model_ids are resolved to real model rows (active, non-deleted, non-virtual)
3. Order is preserved from the stored array

​

Combo (fallback chain)

Request → try model[0] → succeed? return
                        → fail?  try model[1] → succeed? return
                                               → fail?  try model[2] → ...
                                                                      → all fail → 502

• Each model in order is attempted
• On failure, log the error and proceed to next
• If all models fail, returns 502 with errors array and last_error
• Same behavior for both streaming and non-streaming requests

​

Round-robin

Request 1 → pick model[idx % N], increment idx → use that model
Request 2 → pick model[(idx+1) % N], increment idx → use that model

• In-memory rotating index per virtual model ID (not persisted across restarts)
• Protected by asyncio.Lock for thread safety
• No fallback — if the selected model fails, the request fails
• Round-robin virtual models with a single underlying model are functionally equivalent to a direct model reference

​

OpenAI format

GET /api/v2/openai/v1/models

{
  "id": "my-fallback-group",
  "object": "model",
  "created": 1234567890,
  "owned_by": "pinter"
}

​

Constraints and limitations

• No nesting: Virtual models cannot reference other virtual models. Validation rejects underlying_model_ids pointing to combo or round-robin models.
• No pricing fields: Virtual models inherit no pricing from underlying models. Cost is tracked per real model at runtime.
• Round-robin index is in-memory: Resets on server restart. Stateless and ephemeral.
• All requests must be authenticated: Virtual models are not available via unauthenticated endpoints.
• Streaming fallback sequential: Each candidate’s stream is tried in order; the first to produce a stream wins. No model-parallel fan-out.

​

Use cases