AI Engine Configuration

Quick Start: Minimal Config

The minimum configuration to run the OSS AI path with the built-in Ollama chat runtime and the tiered security models:

contexa:
  llm:
    chat:
      ollama:
        base-url: http://localhost:11434
        model: qwen2.5:7b
spring:
  ai:
    security:
      layer1:
        model: qwen2.5:7b
      layer2:
        model: gpt-4o-mini
    vectorstore:
      pgvector:
        index-type: HNSW
        distance-type: COSINE_DISTANCE
        dimensions: 1024

This enables the Ollama chat runtime through contexa.llm.chat.ollama.*, assigns Tier 1 and Tier 2 models, and configures the project-declared pgvector properties. If you want a dedicated embedding runtime, add contexa.llm.embedding.ollama.* as shown below.

Configuration Architecture

The AI runtime is configured through seven property groups. Contexa owns the runtime selection, advisor, RAG, and streaming properties, while Spring AI owns the provider integrations and pgvector base configuration.

LLM Model Configuration

The tiered model system is split between runtime selection in contexa.llm.* and tier assignment in spring.ai.security.*. The OSS core ships with Ollama chat runtime wiring and can also use Spring AI OpenAI or Anthropic providers if those beans are present.

contexa:
  llm:
    selection:
      chat:
        mode: DYNAMIC_PRIORITY
        priority: ollama,anthropic,openai
      embedding:
        mode: DYNAMIC_PRIORITY
        priority: ollama,openai
    chat:
      ollama:
        base-url: http://localhost:11434
        model: qwen2.5:7b
    embedding:
      ollama:
        dedicated-runtime-enabled: true
        base-url: http://localhost:11435
        model: mxbai-embed-large
spring:
  ai:
    security:
      layer1:
        model: qwen2.5:7b
        backup:
          model: llama3.2:latest
      layer2:
        model: gpt-4o-mini
        backup:
          model: deepseek-r1:14b
      tiered:
        layer1:
          timeout-ms: 30000
        layer2:
          timeout-ms: 60000

Recommended selection (OpenAI + Anthropic with Ollama failover)

The platform can chain managed cloud providers with the local Ollama runtime so that chat calls degrade gracefully when an API key is missing or a cloud endpoint is unreachable. The example below uses OpenAI as the primary chat provider, Anthropic as the second-tier failover, and the local Ollama runtime as the offline / no-key fallback. Embedding always goes through OpenAI.

contexa:
  llm:
    selection:
      chat:
        mode: dynamic-priority
        priority: openai,anthropic,ollama
      embedding:
        mode: dynamic-priority
        priority: openai

Key meaning

• mode: dynamic-priority — the orchestrator walks the comma-separated priority list in order and selects the first provider whose ChatModel / EmbeddingModel bean is present and reachable. The runtime form dynamic-priority is the kebab-case spelling of the Mode.DYNAMIC_PRIORITY enum; both forms bind to the same value through Spring's relaxed binding.
• priority: openai,anthropic,ollama — chat calls try OpenAI first; if the OpenAI client is missing the key or returns an unrecoverable error, the orchestrator falls back to Anthropic; if Anthropic is also unavailable, the call lands on the local Ollama runtime. List order is the failover order, so cheaper / more reliable providers should appear earlier.
• priority: openai (embedding) — Anthropic does not ship an embedding model in Spring AI, and the embedding path is dimension-pinned to 1536 in the platform's pgvector schema, which matches OpenAI's text-embedding-3-small / text-embedding-ada-002. Mixing in an Ollama embedding model (typically 768 / 1024 dimensions) would write vectors of an incompatible width into the same column and break similarity search. Keep embeddings on a single 1536-dimension provider.

Pair the selection block with the Spring AI provider properties

The contexa.llm.selection.* block decides which provider is called, but the actual API keys and model names live under the standard Spring AI property tree:

spring:
  ai:
    retry:
      max-attempts: 1
    anthropic:
      api-key: ${ANTHROPIC_API_KEY:disabled}
      chat:
        options:
          model: claude-3-sonnet-20240229
    openai:
      api-key: ${OPENAI_API_KEY:disabled}
      base-url: https://api.openai.com
      chat:
        options:
          model: gpt-4o-mini
          temperature: 0.3

• API key placeholders — the literal disabled default keeps the auto-configuration from registering a real client when the operator has not supplied an environment variable. Setting ANTHROPIC_API_KEY / OPENAI_API_KEY activates the corresponding provider; leaving them as disabled intentionally skips that branch and lets the next provider in the priority list take over.
• spring.ai.retry.max-attempts: 1 — with two providers chained via dynamic-priority, retrying the same failing provider inside Spring AI is wasted latency. 1 keeps each call single-shot and lets the CONTEXA fail-over selector pick the next entry.
• Model identifiers — gpt-4o-mini is recommended for Tier 1 (low-latency Layer 1 contextual decisions) and claude-3-sonnet-20240229 for Tier 2 (forensic Layer 2 reasoning). Swap with newer SKUs as they become available.
• temperature: 0.3 on the OpenAI side is intentional: security-decision prompts benefit from low temperature; raising it widens drift in BLOCK / ESCALATE boundaries.

OpenAI and Anthropic provider credentials are configured through Spring AI's own provider properties such as spring.ai.openai.* and spring.ai.anthropic.*; Contexa only selects between the available provider beans.

See also: LLM & Models Reference

RAG Configuration

These properties drive the retrieval path used by the pipeline and AI Lab. The OSS code binds contexa.rag.* to ContexaRagProperties, and binds pgvector settings to spring.ai.vectorstore.pgvector.*.

Vector Store (pgvector)

contexa:
  rag:
    defaults:
      similarity-threshold: 0.7
      top-k: 10
    risk:
      similarity-threshold: 0.8
      top-k: 50
    lab:
      batch-size: 50
      validation-enabled: true
      enrichment-enabled: true
spring:
  ai:
    vectorstore:
      pgvector:
        dimensions: 1024
        index-type: HNSW
        distance-type: COSINE_DISTANCE
        document:
          chunk-size: 1000
          chunk-overlap: 200

See also: Pipeline & RAG Reference

Streaming Configuration

contexa:
  streaming:
    timeout: 5m
    max-retries: 3
    retry-delay: 1s
    retry-multiplier: 1.5
    marker-buffer-size: 100
    sentence-buffering-enabled: true

See also: Streaming Reference

Advisor Configuration

contexa:
  advisor:
    chain-profile: STANDARD
    security:
      enabled: true
      order: 50
      require-authentication: false
    soar:
      approval:
        enabled: true
        order: 100
        timeout: 300

See also: Advisor System Reference

Tuning Scenarios

Common configuration adjustments for specific situations:

Slow Responses

If inference is slow, reduce the model footprint and widen the Layer 1 pipeline budget:

contexa:
  llm:
    chat:
      ollama:
        model: qwen2.5:7b
spring:
  ai:
    security:
      layer1:
        model: qwen2.5:7b
      tiered:
        layer1:
          timeout:
            total-ms: 5400000
            llm-ms: 3600000
            rag-ms: 1200000

RAG Results Are Inaccurate

If retrieved context is not relevant enough, tighten the default threshold and reduce the retrieval window:

contexa:
  rag:
    defaults:
      similarity-threshold: 0.85
      top-k: 3
spring:
  ai:
    security:
      tiered:
        layer1:
          rag:
            similarity-threshold: 0.7
          vector-search-limit: 6

Token Usage / Cost Optimization

Reduce prompt size by lowering the Layer 1 prompt and truncation budgets:

spring:
  ai:
    security:
      tiered:
        layer1:
          prompt:
            max-rag-documents: 2
            max-similar-events: 1
          vector-search-limit: 3
        truncation:
          layer1:
            payload: 100
            rag-document: 150

Production Deployment

Recommended settings for a production deployment with separate chat and embedding runtimes:

contexa:
  llm:
    selection:
      chat:
        mode: DYNAMIC_PRIORITY
        priority: ollama,anthropic,openai
      embedding:
        mode: DYNAMIC_PRIORITY
        priority: ollama,openai
    embedding:
      ollama:
        dedicated-runtime-enabled: true
        base-url: http://localhost:11435
  advisor:
    security:
      enabled: true
      require-authentication: true
spring:
  ai:
    security:
      layer1:
        model: qwen2.5:7b
      layer2:
        model: gpt-4o-mini
      tiered:
        vector-cache:
          max-size: 50000
          expire-minutes: 10
          enabled: true
          record-stats: true
        security:
          trusted-proxy-validation-enabled: true

Complete Property Reference

contexa:
  llm:
    bindings:
      chat:
        primary-ollama:
          bean-name: contexaOllamaChatModel
          provider: ollama
          model-id: qwen2.5:7b
          aliases: [default, fast]
          enabled: true
          primary: true
      embedding:
        primary-ollama:
          bean-name: contexaSharedOllamaEmbeddingModel
          provider: ollama
          model-id: mxbai-embed-large
          enabled: true
          primary: true