Model providers and routing

Shirube talks to models through a small ModelProvider interface (chat(request)). You are not locked to OpenAI. The agent loop stays the same.

Default: OpenAI

.apiKey() / OPENAI_API_KEY constructs OpenAIProvider. Optional .baseURL() for proxies or Azure-compatible gateways.

.apiKey(process.env.OPENAI_API_KEY!)
.model("gpt-4.1")
.baseURL(process.env.OPENAI_BASE_URL)

Complexity routing

If you omit .model(), Shirube labels the prompt and picks from a catalog:

Default mode is hybrid: obviously tiny prompts skip the extra classifier call; others use gpt-4.1-mini as classifier. mode: "llm" always classifies; "heuristic" never calls the mini model.

.modelRouter({
  mode: "hybrid",
  catalog: {
    classifier: "gpt-4.1-mini",
    simple: "gpt-4.1-mini",
    moderate: "gpt-4.1",
    complex: "gpt-5",
    reasoning: "gpt-5",
  },
})

result.complexity tells you the tier, confidence, and source (heuristic | llm | hybrid). Use it in logs to see if you are overspending.

Use routing when traffic is mixed. Pin `.model()` when you need a single SLA or when the classifier cost is not worth it (batch jobs, already-known difficulty).

Claude and Gemini

import { AnthropicProvider, GeminiProvider } from "shirube-ai";

.provider(new AnthropicProvider({ apiKey: process.env.ANTHROPIC_API_KEY }))
.model("claude-sonnet-4-20250514")

.provider(new GeminiProvider({ apiKey: process.env.GEMINI_API_KEY }))
.model("gemini-2.0-flash")

They use fetch against the public APIs. You still write one agent.

Fallback chain

import { OpenAIProvider, AnthropicProvider, GeminiProvider } from "shirube-ai";

.provider(new OpenAIProvider({ apiKey: process.env.OPENAI_API_KEY! }))
.fallback(
  new AnthropicProvider({ apiKey: process.env.ANTHROPIC_API_KEY }),
  new GeminiProvider({ apiKey: process.env.GEMINI_API_KEY }),
)

Each provider is tried in order until one succeeds. Use this for uptime, not for “ask three models and vote” (that would be a custom plugin).

Bring your own

Implement ModelProvider (name + chat) for Bedrock, vLLM, or a mock in tests. Tests in this repo use a scripted provider — you can do the same.