Models

ADK TypeScript provides flexible model integration, allowing you to use various Large Language Models (LLMs) with your agents. You can use Google Gemini models directly or integrate external models through wrapper classes.

Model Integration Methods

ADK TypeScript supports two primary integration approaches:

Google Gemini Models

The most straightforward way to use Google's flagship models with ADK TypeScript.

Access Methods

Available Models

• Gemini 2.0 Flash: Latest high-speed model for most use cases
• Gemini 2.5 Pro: Powerful model for complex reasoning tasks
• Gemini 1.5 Pro: Stable model with large context windows
• Live API Models: Special models supporting voice/video streaming

External Models via LiteLLM

Access models from OpenAI, Anthropic, Cohere, and 100+ other providers through the LiteLLM integration.

Supported Providers

Setup Requirements

1. Install LiteLLM: Add the LiteLLM package to your project
2. Configure API Keys: Set environment variables for your chosen providers
3. Create Wrapper: Use the LiteLLM wrapper class with your agents

Local and Open Source Models

Run models locally for privacy, cost control, or offline operation.

Local Deployment Options

Considerations for Local Models

• Tool Support: Ensure your chosen model supports function calling
• Performance: Consider hardware requirements for model size
• Reliability: Local models may have different reliability characteristics
• Model Quality: Open source models vary in capability and consistency

Model Configuration

Generation Parameters

Control how models generate responses:

• Temperature: Randomness in responses (0.0 = deterministic, 1.0 = creative)
• Max Tokens: Maximum response length
• Top-P/Top-K: Advanced sampling parameters
• Safety Settings: Content filtering and safety controls

Performance Optimization

• Model Selection: Choose appropriate model size for your use case
• Caching: Implement response caching for repeated queries
• Batching: Group requests when possible
• Rate Limiting: Respect provider rate limits

Cost Management

• Model Tier Selection: Balance cost vs capability
• Request Optimization: Minimize unnecessary model calls
• Usage Monitoring: Track costs and usage patterns
• Efficient Prompting: Design prompts for optimal token usage

Best Practices

Model Selection Strategy

1. Start Simple: Begin with Fast/Flash models for development
2. Test Thoroughly: Validate model performance with your specific use case
3. Consider Latency: Factor in response time requirements
4. Evaluate Quality: Test output quality across different scenarios

Environment Management

• Separate Environments: Use different models for dev/staging/production
• Configuration Management: Use environment variables for model selection
• Fallback Strategies: Implement backup models for reliability
• Monitoring: Track model performance and availability

Security Considerations

• API Key Management: Secure storage and rotation of API keys
• Network Security: Protect communications with model providers
• Data Privacy: Consider data residency and privacy requirements
• Audit Trails: Log model interactions for compliance

Troubleshooting

Common Issues

Performance Issues

• Monitor response times and adjust model selection
• Check network connectivity and latency
• Verify proper resource allocation for local models
• Consider model warming strategies for improved performance

Related Topics