Skip to main content

​
Router Performance

Picept’s Router1 has been extensively tested against leading AI models and demonstrates superior performance across all key metrics: Router1 Performance Comparison As shown in the chart above, Router1 offers significant improvements over GPT-4:
  • Quality: +7% better response quality
  • Cost: 89% more cost-effective
  • Speed: 251% faster response time
  • Latency: 35% lower latency
The quality comparison chart demonstrates Router1’s superior performance (94.3) compared to other leading models like GPT-4o (89.2), Claude models, and other providers. Picept offers flexibility in choosing between different pre-trained routers based on your specific needs:
  • Router1: Optimized for maximum quality while maintaining significant cost and speed advantages
  • Router2: Engineered for blazing-fast performance and minimal cost, with a small quality trade-off compared to Router1
This allows you to select the router that best matches your application’s requirements. For instance, use Router1 for tasks requiring highest accuracy, or Router2 for high-volume, time-sensitive applications where speed is crucial. Need something more specific? You can easily train a custom router optimized for your unique use case. Whether you have specific quality requirements, specialized domain knowledge, or unique performance constraints, Picept’s router training platform lets you create a router that perfectly aligns with your needs.

​
Training Custom Routers

​
API-Based Training

You can train custom routers tailored to your specific needs using either our API or web interface. Let’s start with the API approach: 
import requests
import time

# Prepare training payload
payload = {
    "Router_id": "test_router",  # Your custom router name 
    
    "dataset": {
        "prompts": ['prompt1', 'prompt-2', ...],  # Your list of prompts 
    },
    
    "model_list": "gpt-4o-mini[openai], gemini-1.5-flash-latest[google]", # Candidate models
    "assessment_model": "claude-3-5-sonnet-20240620[anthropic]"  # Optional quality assessment model
}

# Start training job
response = requests.post(
    "https://api.picept.ai/v1/router",
    json=payload,
    headers={ 
        "Authorization": f"Bearer {PICEPT_API_KEY}",
    }
)

# Check training status
job_id = response.json()["job_id"]
router_id = None

while True:
    status_response = requests.get(f"https://api.picept.ai/v1/router/{job_id}/status")
    status_data = status_response.json()
    status = status_data["status"]
    print(f"Job status: {status}")
    
    if status == "done":
        router_id = status_data.get("router_id")
        metrics = status_data.get("metrics", {})
        print(f"Router training completed. Router ID: {router_id}")
        print("Metrics:")
        for key, value in metrics.items():
            print(f"  {key}: {value}")
        break
    elif status == "failed":
        error_message = status_data.get("error", "Unknown error")
        print(f"Router training failed. Error: {error_message}")
        break
    
    time.sleep(5)  # Wait before next check

​
Training via Web Interface

For a more user-friendly approach, you can train routers directly through our web interface. The platform offers three ways to provide training data:
  1. Upload Files: Support for CSV and JSONL formats
  2. Chat Completions: Use your existing chat completion data
  3. Direct Input: Enter prompts and responses manually
Router Training Interface Key features of the web interface include:
  • Real-time training progress monitoring
  • Comprehensive metrics visualization
  • Easy comparison between different routers
  • Team collaboration capabilities
  • Version control for router iterations

​
Best Practices for Router Training

  1. Data Quality
    • Ensure diverse prompt examples
    • Include edge cases and common scenarios
    • Validate data format before training
  2. Model Selection
    • Choose complementary models
    • Include models with different strengths
    • Consider cost-performance trade-offs
  3. Monitoring
    • Track training progress
    • Review performance metrics
    • Test with validation data
  4. Iteration
    • Start with smaller datasets
    • Gradually expand training data
    • Monitor performance changes

​
Using Trained Routers

Once training is complete, you can use your custom router just like any other model: 
client = OpenAI(
    api_key=PICEPT_API_KEY,
    base_url='https://api.picept.ai/v1'
)

response = client.chat.completions.create(
    model="your_trained_router_id",
    messages=[{"role": "user", "content": "Your prompt here"}]
)

​
Test and Share Your Routers

Once your router training is complete, it’s immediately available for testing in the Picept Playground. This interactive environment allows you to:
  • Test your router with various prompts
  • Compare performance with other models in real-time
  • Analyze response quality, speed, and costs
  • Share results with team members
Picept Playground Interface The playground provides a collaborative environment where everyone in your organization can:
  • Access all trained routers
  • Run comparative tests
  • Share feedback and insights
  • Make data-driven decisions about which router to use for specific use cases
This seamless integration between training and testing environments helps streamline your AI development workflow and ensures your entire team can leverage the power of custom-trained routers.