Multiple Providers

Multiple Providers Guide

Provider Flexibility Cloud & Local

Learn how to effectively work with multiple AI providers, switch between them, and choose the right provider for your use case.

Available Providers

Ollama (Local)

Status: ✅ Fully Supported

Best For: Development, privacy, offline use

Models: Gemma, Llama, Phi, Mixtral, CodeLlama

OpenRouter (Cloud)

Status: ✅ Fully Supported

Best For: Production, latest models, scalability

Providers: OpenAI, Anthropic, Google, Meta, and more

Native Providers

Status: 🚧 Coming Soon

Planned: OpenAI, Anthropic, Google

Timeline: Next major release

Provider Comparison

Feature	Ollama	OpenRouter	Native (Coming)
Cost	Free	Pay-per-use	Varies
Setup	Installation	API key	API key
Privacy	Complete	Provider-dependent	Provider-dependent
Offline	✅ Yes	❌ No	❌ No
Model Selection	50+ models	200+ models	Provider-specific
Latency	Low (local)	Network-dependent	Network-dependent

Switching Between Providers

Basic Provider Switching

import { AgentForceAgent } from '@agentforce/adk';

const config = { name: "FlexibleAgent" };

// Local development with Ollama
const devAgent = new AgentForceAgent(config)
  .useLLM("ollama", "gemma3:12b")
  .systemPrompt("Development mode assistant")
  .prompt("Test prompt for development");

// Production with OpenRouter
const prodAgent = new AgentForceAgent(config)
  .useLLM("openrouter", "openai/gpt-4")
  .systemPrompt("Production assistant")
  .prompt("Production query");

const devResponse = await devAgent.output("text");
const prodResponse = await prodAgent.output("text");

Environment-Based Provider Selection

function createAgent(name: string) {
  const isProd = process.env.NODE_ENV === 'production';
  const config = { name };

  return new AgentForceAgent(config)
    .useLLM(
      isProd ? "openrouter" : "ollama",
      isProd ? "openai/gpt-4" : "gemma3:12b"
    );
}

// Usage
const agent = createAgent("AdaptiveAgent")
  .systemPrompt("I adapt to the environment")
  .prompt("What environment am I running in?");

Dynamic Provider Configuration

interface ProviderConfig {
  provider: string;
  model: string;
  description: string;
}

const providers: Record<string, ProviderConfig> = {
  fast: {
    provider: "ollama",
    model: "phi4-mini:latest",
    description: "Fast local inference"
  },
  balanced: {
    provider: "ollama",
    model: "gemma3:12b",
    description: "Balanced performance and capability"
  },
  powerful: {
    provider: "openrouter",
    model: "openai/gpt-4",
    description: "Maximum capability"
  },
  economical: {
    provider: "openrouter",
    model: "moonshotai/kimi-k2:free",
    description: "Free cloud model"
  }
};

function createAgentWithProfile(profile: keyof typeof providers) {
  const config = providers[profile];

  return new AgentForceAgent({
    name: `${profile}Agent`
  })
    .useLLM(config.provider, config.model)
    .systemPrompt(`You are running in ${profile} mode: ${config.description}`);
}

// Usage
const fastAgent = createAgentWithProfile("fast");
const powerfulAgent = createAgentWithProfile("powerful");

Provider-Specific Configurations

Ollama Optimizations

Model Selection

// Choose model based on hardware
function getOllamaModel() {
  const totalRAM = process.memoryUsage().rss / 1024 / 1024; // Rough estimate

  if (totalRAM < 8000) {
    return "phi4-mini:latest";      // 3.8B params
  } else if (totalRAM < 16000) {
    return "gemma3:12b";           // 12B params
  } else {
    return "llama3.1:70b";         // 70B params
  }
}

const agent = new AgentForceAgent(config)
  .useLLM("ollama", getOllamaModel());

Specialized Models

// Task-specific Ollama models
const codeAgent = new AgentForceAgent(config)
  .useLLM("ollama", "codellama:34b")        // Code generation
  .systemPrompt("You are a code expert");

const chatAgent = new AgentForceAgent(config)
  .useLLM("ollama", "gemma3:12b")           // General chat
  .systemPrompt("You are a conversational assistant");

const mathAgent = new AgentForceAgent(config)
  .useLLM("ollama", "llama3.1:70b")         // Complex reasoning
  .systemPrompt("You are a mathematics expert");

Connection Management

// Custom Ollama endpoint
process.env.OLLAMA_HOST = "http://remote-ollama:11434";

const agent = new AgentForceAgent(config)
  .useLLM("ollama", "gemma3:12b");

// Health check for Ollama
async function checkOllamaHealth() {
  try {
    const response = await fetch(`${process.env.OLLAMA_HOST || 'http://localhost:11434'}/api/tags`);
    return response.ok;
  } catch {
    return false;
  }
}

OpenRouter Optimizations

Model Selection Strategy

// Cost-aware model selection
const models = {
  free: [
    "moonshotai/kimi-k2:free",
    "google/gemma-2-9b-it:free",
    "anthropic/claude-3-haiku:beta"
  ],
  budget: [
    "openai/gpt-3.5-turbo",
    "meta-llama/llama-3.1-8b-instruct",
    "anthropic/claude-3-haiku"
  ],
  premium: [
    "openai/gpt-4",
    "anthropic/claude-3-opus",
    "meta-llama/llama-3.1-405b-instruct"
  ]
};

function selectOpenRouterModel(budget: 'free' | 'budget' | 'premium') {
  const availableModels = models[budget];
  return availableModels[0]; // Select first available
}

const agent = new AgentForceAgent(config)
  .useLLM("openrouter", selectOpenRouterModel("budget"));

Provider-Specific Models

// Use specific providers through OpenRouter
const openaiAgent = new AgentForceAgent(config)
  .useLLM("openrouter", "openai/gpt-4")
  .systemPrompt("You are powered by OpenAI GPT-4");

const claudeAgent = new AgentForceAgent(config)
  .useLLM("openrouter", "anthropic/claude-3-sonnet")
  .systemPrompt("You are powered by Anthropic Claude");

const llamaAgent = new AgentForceAgent(config)
  .useLLM("openrouter", "meta-llama/llama-3.1-70b-instruct")
  .systemPrompt("You are powered by Meta Llama");

Error Handling

async function createRobustOpenRouterAgent() {
  const fallbackModels = [
    "openai/gpt-4",
    "anthropic/claude-3-sonnet",
    "meta-llama/llama-3.1-8b-instruct",
    "moonshotai/kimi-k2:free"  // Free fallback
  ];

  for (const model of fallbackModels) {
    try {
      const agent = new AgentForceAgent(config)
        .useLLM("openrouter", model)
        .systemPrompt("You are a robust assistant")
        .prompt("Test connection");

      await agent.run();
      return agent; // Success
    } catch (error) {
      console.warn(`Model ${model} failed, trying next...`);
      continue;
    }
  }

  throw new Error("All OpenRouter models failed");
}

Multi-Provider Strategies

Fallback Strategy

class FallbackAgent {
  private providers = [
    { provider: "openrouter", model: "openai/gpt-4" },
    { provider: "openrouter", model: "anthropic/claude-3-sonnet" },
    { provider: "ollama", model: "gemma3:12b" },
    { provider: "ollama", model: "phi4-mini:latest" }
  ];

  async createAgent(config: AgentConfig) {
    for (const { provider, model } of this.providers) {
      try {
        const agent = new AgentForceAgent(config)
          .useLLM(provider, model)
          .systemPrompt(`Powered by ${provider}/${model}`);

        // Test the configuration
        await agent.debug().run();
        return agent;
      } catch (error) {
        console.warn(`Provider ${provider}/${model} failed:`, error.message);
        continue;
      }
    }

    throw new Error("All providers failed");
  }
}

// Usage
const fallback = new FallbackAgent();
const agent = await fallback.createAgent({
  name: "ResilientAgent"
});

Load Balancing Strategy

class LoadBalancedAgent {
  private providers = [
    { provider: "ollama", model: "gemma3:12b", weight: 3 },
    { provider: "openrouter", model: "openai/gpt-4", weight: 1 }
  ];

  private requestCount = 0;

  selectProvider() {
    // Simple weighted round-robin
    this.requestCount++;
    const totalWeight = this.providers.reduce((sum, p) => sum + p.weight, 0);
    const selection = this.requestCount % totalWeight;

    let currentWeight = 0;
    for (const provider of this.providers) {
      currentWeight += provider.weight;
      if (selection < currentWeight) {
        return provider;
      }
    }

    return this.providers[0]; // Fallback
  }

  createAgent(config: AgentConfig) {
    const selected = this.selectProvider();

    return new AgentForceAgent(config)
      .useLLM(selected.provider, selected.model)
      .systemPrompt(`Load balanced: ${selected.provider}/${selected.model}`);
  }
}

// Usage
const balancer = new LoadBalancedAgent();
const agent1 = balancer.createAgent({ name: "BalancedAgent1" });
const agent2 = balancer.createAgent({ name: "BalancedAgent2" });

Capability-Based Selection

interface ProviderCapability {
  provider: string;
  model: string;
  capabilities: string[];
  cost: 'free' | 'low' | 'medium' | 'high';
  speed: 'fast' | 'medium' | 'slow';
}

const providerCapabilities: ProviderCapability[] = [
  {
    provider: "ollama",
    model: "codellama:34b",
    capabilities: ["code", "programming", "debugging"],
    cost: "free",
    speed: "medium"
  },
  {
    provider: "openrouter",
    model: "openai/gpt-4",
    capabilities: ["reasoning", "analysis", "writing", "code"],
    cost: "high",
    speed: "medium"
  },
  {
    provider: "ollama",
    model: "phi4-mini:latest",
    capabilities: ["chat", "simple-tasks"],
    cost: "free",
    speed: "fast"
  }
];

function selectProviderByCapability(
  requiredCapability: string,
  preferredCost?: 'free' | 'low' | 'medium' | 'high'
) {
  let candidates = providerCapabilities.filter(p =>
    p.capabilities.includes(requiredCapability)
  );

  if (preferredCost) {
    const costFiltered = candidates.filter(p => p.cost === preferredCost);
    if (costFiltered.length > 0) {
      candidates = costFiltered;
    }
  }

  // Return fastest option among candidates
  return candidates.sort((a, b) => {
    const speedOrder = { fast: 0, medium: 1, slow: 2 };
    return speedOrder[a.speed] - speedOrder[b.speed];
  })[0];
}

// Usage
function createCapabilityAgent(task: string, capability: string) {
  const selected = selectProviderByCapability(capability, "free");

  return new AgentForceAgent({
    name: `${capability}Agent`
  })
    .useLLM(selected.provider, selected.model)
    .systemPrompt(`You specialize in ${capability}. Model: ${selected.model}`)
    .prompt(task);
}

// Examples
const codeAgent = createCapabilityAgent("Review this function", "code");
const chatAgent = createCapabilityAgent("Hello, how are you?", "chat");

Provider Testing and Monitoring

Health Check System

class ProviderHealthChecker {
  async checkOllama(): Promise<boolean> {
    try {
      const response = await fetch('http://localhost:11434/api/tags');
      return response.ok;
    } catch {
      return false;
    }
  }

  async checkOpenRouter(): Promise<boolean> {
    try {
      const apiKey = process.env.OPENROUTER_API_KEY;
      if (!apiKey) return false;

      const response = await fetch('https://openrouter.ai/api/v1/models', {
        headers: { 'Authorization': `Bearer ${apiKey}` }
      });
      return response.ok;
    } catch {
      return false;
    }
  }

  async getAvailableProviders() {
    const results = await Promise.allSettled([
      this.checkOllama(),
      this.checkOpenRouter()
    ]);

    return {
      ollama: results[0].status === 'fulfilled' && results[0].value,
      openrouter: results[1].status === 'fulfilled' && results[1].value
    };
  }

  async createHealthyAgent(config: AgentConfig) {
    const available = await this.getAvailableProviders();

    if (available.openrouter) {
      return new AgentForceAgent(config)
        .useLLM("openrouter", "openai/gpt-4");
    } else if (available.ollama) {
      return new AgentForceAgent(config)
        .useLLM("ollama", "gemma3:12b");
    } else {
      throw new Error("No providers available");
    }
  }
}

// Usage
const healthChecker = new ProviderHealthChecker();
const agent = await healthChecker.createHealthyAgent({
  name: "HealthyAgent"
});

Performance Monitoring

class ProviderPerformanceMonitor {
  private metrics = new Map<string, {
    requests: number;
    totalTime: number;
    errors: number;
  }>();

  async measureAgent(
    provider: string,
    model: string,
    prompt: string
  ) {
    const key = `${provider}/${model}`;
    const startTime = Date.now();

    try {
      const agent = new AgentForceAgent({
        name: "BenchmarkAgent"
      })
        .useLLM(provider, model)
        .systemPrompt("You are a test assistant")
        .prompt(prompt);

      const response = await agent.output("text");

      const duration = Date.now() - startTime;
      this.recordSuccess(key, duration);

      return { success: true, response, duration };
    } catch (error) {
      this.recordError(key);
      return { success: false, error: error.message };
    }
  }

  private recordSuccess(key: string, duration: number) {
    const current = this.metrics.get(key) || { requests: 0, totalTime: 0, errors: 0 };
    this.metrics.set(key, {
      requests: current.requests + 1,
      totalTime: current.totalTime + duration,
      errors: current.errors
    });
  }

  private recordError(key: string) {
    const current = this.metrics.get(key) || { requests: 0, totalTime: 0, errors: 0 };
    this.metrics.set(key, {
      requests: current.requests + 1,
      totalTime: current.totalTime,
      errors: current.errors + 1
    });
  }

  getMetrics() {
    const results = {};
    for (const [key, metrics] of this.metrics) {
      results[key] = {
        averageTime: metrics.totalTime / (metrics.requests - metrics.errors),
        errorRate: metrics.errors / metrics.requests,
        totalRequests: metrics.requests
      };
    }
    return results;
  }
}

// Usage
const monitor = new ProviderPerformanceMonitor();

// Benchmark different providers
await monitor.measureAgent("ollama", "gemma3:12b", "Test prompt");
await monitor.measureAgent("openrouter", "openai/gpt-4", "Test prompt");

console.log(monitor.getMetrics());

Best Practices

Provider Selection Guidelines

Development

Use Ollama for fast iteration and testing

.useLLM("ollama", "phi4-mini:latest")

Production

Use OpenRouter for reliability and latest models

.useLLM("openrouter", "openai/gpt-4")

Cost Optimization

Mix providers based on task complexity

// Simple tasks: free models
// Complex tasks: premium models

Reliability

Implement fallbacks for production systems

// Primary -> Secondary -> Local fallback

Configuration Management

// centralized provider configuration
const PROVIDER_CONFIG = {
  development: {
    provider: "ollama",
    model: "phi4-mini:latest",
    debug: true
  },
  testing: {
    provider: "ollama",
    model: "gemma3:12b",
    debug: false
  },
  production: {
    provider: "openrouter",
    model: "openai/gpt-4",
    debug: false
  }
};

function createConfiguredAgent(env: keyof typeof PROVIDER_CONFIG) {
  const config = PROVIDER_CONFIG[env];

  let agent = new AgentForceAgent({
    name: `${env}Agent`
  }).useLLM(config.provider, config.model);

  if (config.debug) {
    agent = agent.debug();
  }

  return agent;
}

Next Steps

API Reference

Explore the complete compatibility API reference → API Reference

Method Chaining

Discover how to chain methods for complex operations → Method Chaining Guide

You now have the knowledge to effectively use multiple providers in AgentForce ADK, from simple switching to sophisticated load balancing and fallback strategies!