Model Providers

Overview

Contexa supports multiple LLM runtimes simultaneously. At startup, the DynamicModelRegistry populates its descriptor map through two mechanisms:

1. LlmRuntimeCatalog chat bindings — every LlmRuntimeBinding returned from LlmRuntimeCatalog.getChatBindings() is registered. The catalog is built from Spring AI ChatModel beans created by the Contexa runtime (for example via contexa.llm.chat.ollama.*) and by Spring AI autoconfigurations (Anthropic, OpenAI, and others).
2. TieredLLMProperties layers — the primary and optional backup models configured for spring.ai.security.layer1 and spring.ai.security.layer2 are registered with their tier. The registry uses resolveCanonicalModelId(...) so the same model can be found by its runtime ID, bean name, or any alias.

The registry infers the provider name from the model ID when no binding is available. Recognised substrings include llama, qwen, gemma, mistral, phi, exaone, codellama, and deepseek (→ ollama), claude (→ anthropic), gpt/o1/davinci (→ openai), gemini/vertex (→ gemini), and bedrock (→ bedrock). Anything else is labelled spring.

LlmRuntimeCatalog

The interface that exposes discovered chat and embedding runtimes. The registry uses it to find bindings and resolve live ChatModel instances on demand.

LlmRuntimeBinding

An immutable record of a single runtime binding. Bindings are created by Contexa's LLM autoconfiguration and consumed by both the catalog and the registry.

Helper Methods

ModelDescriptor

Describes a model's identity, capabilities, default options, and current status.

ModelCapabilities (nested class)

ModelOptions (nested class)

Helper Method

DynamicModelRegistry

Central registry that manages descriptors and caches ChatModel instances. It registers models from the catalog and from tier configuration during @PostConstruct initialization.

Constructor Dependencies

Public API

ModelSelectionStrategy

Interface for model selection logic. The DynamicModelSelectionStrategy is the default implementation.

DynamicModelSelectionStrategy

The default selection strategy uses an explicit-model-first fallback chain. It is constructed with a DynamicModelRegistry, the TieredLLMProperties, and the primary ChatModel bean.

1. Explicit model request — ExecutionContext.preferredModel is tried first. When absent, the strategy looks for metadata keys in this order: requestedModelId, preferredModel, runtimeModelId, modelId.
2. Tier resolution — When no explicit model resolves, the strategy picks a tier in this order: analysisLevel.getDefaultTier(), then securityTaskType.getDefaultTier(), then ExecutionContext.tier. Values above 1 are normalized to configured layer 2.
3. Primary/backup lookup — The strategy reads TieredLLMProperties.getModelNameForTier(...) for the primary model and getBackupModelNameForTier(...) for the backup.
4. Primary ChatModel fallback — If every tier candidate fails, the auto-configured primary ChatModel bean is returned as the last fallback.

Resolution Metadata

The strategy records the outcome on ExecutionContext.metadata. Downstream code and advisors can inspect these keys:

Additional Method

Configuration

Tiered Model Hierarchy

spring:
  ai:
    security:
      layer1:
        model: qwen2.5:7b
        backup:
          model: qwen2.5:7b
      layer2:
        model: gpt-4o-mini
        backup:
          model: llama3.1:8b
      tiered:
        layer1:
          timeout-ms: 30000
        layer2:
          timeout-ms: 60000

contexa:
  llm:
    chat:
      ollama:
        base-url: http://localhost:11434
        model: qwen2.5:7b

Related