Smart TTL Management

This guide explains how to use Smart TTL Management to intelligently optimize cache durations based on query patterns and data volatility, maximizing cache hit rates and reducing infrastructure costs.

Overview

Instead of using a single TTL for all cached responses, Smart TTL Management dynamically adjusts cache durations based on:

• Query Type: Different TTLs for user profiles, static content, real-time data, etc.
• Data Volatility: Automatically learns how often data changes
• Mutation Patterns: Tracks which mutations affect which queries
• Cache Control Hints: Respects @cacheControl directives from your schema

Key Benefits

• Higher Cache Hit Rates: Increase from 75% to 90%+ by optimizing TTLs
• Reduced Database Load: 15% additional reduction in database queries
• Automatic Optimization: ML-based volatility detection learns optimal TTLs
• Cost Savings: $100-200/month in reduced database costs

Configuration

use grpc_graphql_gateway::{SmartTtlConfig, SmartTtlManager};
use std::time::Duration;
use std::collections::HashMap;

let mut custom_patterns = HashMap::new();
custom_patterns.insert("specialQuery".to_string(), Duration::from_secs(7200));

let config = SmartTtlConfig {
    default_ttl: Duration::from_secs(300),              // 5 minutes
    user_profile_ttl: Duration::from_secs(900),         // 15 minutes
    static_content_ttl: Duration::from_secs(86400),     // 24 hours
    real_time_data_ttl: Duration::from_secs(5),         // 5 seconds
    aggregated_data_ttl: Duration::from_secs(1800),     // 30 minutes
    list_query_ttl: Duration::from_secs(600),           // 10 minutes
    item_query_ttl: Duration::from_secs(300),           // 5 minutes
    auto_detect_volatility: true,                       // Enable ML-based learning
    min_observations: 10,                               // Learn after 10 executions
    max_adjustment_factor: 2.0,                         // Can double or halve TTL
    custom_patterns,                                    // Custom query patterns
    respect_cache_hints: true,                          // Honor @cacheControl
};

let ttl_manager = SmartTtlManager::new(config);

Basic Usage

Calculate Optimal TTL

use grpc_graphql_gateway::SmartTtlManager;

let query = r#"
    query {
        categories {
            id
            name
        }
    }
"#;

// Calculate TTL
let ttl_result = ttl_manager.calculate_ttl(
    query,
    "categories",
    None, // No cache hint
).await;

println!("TTL: {:?}", ttl_result.ttl);
println!("Strategy: {:?}", ttl_result.strategy);
println!("Confidence: {}", ttl_result.confidence);

Query Type Detection

Smart TTL automatically detects query types and applies appropriate TTLs:

Real-Time Data (5 seconds)

query {
  liveScores { team score }      # Contains "live"
  currentPrice { symbol price }  # Contains "current"
  realtimeData { value }         # Contains "realtime"
}

Static Content (24 hours)

query {
  categories { id name }         # Contains "categories"
  tags { name }                  # Contains "tags"
  settings { key value }         # Contains "settings"
  appConfig { version }          # Contains "config"
}

User Profiles (15 minutes)

query {
  profile { name email }         # Contains "profile"
  user(id: 1) { name }          # Contains "user"
  me { id name }                # Contains "me"
  account { settings }          # Contains "account"
}

Aggregated Data (30 minutes)

query {
  statistics { count average }   # Contains "statistics"
  analytics { views clicks }     # Contains "analytics"
  aggregateData { sum }         # Contains "aggregate"
}

List Queries (10 minutes)

query {
  listUsers(limit: 10) { id }   # Contains "list"
  posts(page: 1) { title }      # Contains "page"
  itemsWithOffset(offset: 20) { id } # Contains "offset"
}

Single Item Queries (5 minutes)

query {
  getUserById(id: 1) { name }   # Contains "byid"
  getPost(id: 123) { title }    # Contains "get"
  findProduct(id: 42) { name }  # Contains "find"
}

Volatility-Based Learning

Smart TTL learns from query execution patterns:

// Record query results to track changes
let query = "query { user(id: 1) { name } }";
let result_hash = calculate_hash(&result); // Your hash function

ttl_manager.record_query_result(query, result_hash).await;

// After 10+ executions, TTL will auto-adjust based on volatility
let ttl_result = ttl_manager.calculate_ttl(query, "user", None).await;

match ttl_result.strategy {
    TtlStrategy::VolatilityBased { base_ttl, volatility_score } => {
        println!("Base TTL: {:?}", base_ttl);
        println!("Volatility: {:.2}%", volatility_score * 100.0);
        println!("Adjusted TTL: {:?}", ttl_result.ttl);
    }
    _ => {}
}

Volatility Adjustment

Cache Control Hints

Respect @cacheControl directives from your GraphQL schema:

type Query {
  # Cache for 1 hour
  products: [Product!]! @cacheControl(maxAge: 3600)
  
  # Don't cache
  liveData: LiveData! @cacheControl(maxAge: 0)
}

// Parse cache hint from schema metadata
use grpc_graphql_gateway::parse_cache_hint;

let schema_meta = "@cacheControl(maxAge: 3600)";
let hint = parse_cache_hint(schema_meta);

let ttl_result = ttl_manager.calculate_ttl(
    query,
    "products",
    hint, // Will use 3600 seconds
).await;

Mutation Tracking

Track which mutations affect which queries to invalidate caches intelligently:

// When a mutation occurs
let mutation_type = "updateUser";
let affected_queries = vec![
    "user(id: 1)".to_string(),
    "me".to_string(),
    "userProfile".to_string(),
];

ttl_manager.record_mutation(mutation_type, affected_queries).await;

// Affected queries will have shorter TTLs based on mutation frequency

Custom Pattern Matching

Define custom TTLs for specific query patterns:

let mut custom_patterns = HashMap::new();

// VIP queries get longer cache
custom_patterns.insert("premiumData".to_string(), Duration::from_secs(7200));

// Expensive queries get aggressive caching
custom_patterns.insert("complexReport".to_string(), Duration::from_secs(3600));

// Frequently updated data gets short cache
custom_patterns.insert("inventory".to_string(), Duration::from_secs(30));

let config = SmartTtlConfig {
    custom_patterns,
    ..Default::default()
};

Integration with Response Cache

Integrate with the existing response cache:

use grpc_graphql_gateway::{Gateway, CacheConfig, SmartTtlManager};
use std::sync::Arc;

// Create TTL manager
let ttl_manager = Arc::new(SmartTtlManager::new(SmartTtlConfig::default()));

// Modify cache lookup to use smart TTL
async fn cache_with_smart_ttl(
    cache: &ResponseCache,
    ttl_manager: &SmartTtlManager,
    query: &str,
    query_type: &str,
) -> Option<CachedResponse> {
    // Get optimal TTL
    let ttl_result = ttl_manager.calculate_ttl(query, query_type, None).await;
    
    // Check cache
    if let Some(cached) = cache.get(query).await {
        // Use smart TTL for freshness check
        if cached.age() < ttl_result.ttl {
            return Some(cached);
        }
    }
    
    None
}

TTL Analytics

Monitor TTL effectiveness:

let analytics = ttl_manager.get_analytics().await;

println!("Total query patterns tracked: {}", analytics.total_queries);
println!("Average volatility: {:.2}%", analytics.avg_volatility_score * 100.0);
println!("Average recommended TTL: {:?}", analytics.avg_recommended_ttl);
println!("Highly volatile queries: {}", analytics.highly_volatile_queries);
println!("Stable queries: {}", analytics.stable_queries);

Periodic Cleanup

Clean up old statistics to prevent memory growth:

use tokio::time::{interval, Duration};

// Run cleanup every hour
let mut cleanup_interval = interval(Duration::from_secs(3600));

tokio::spawn({
    let ttl_manager = Arc::clone(&ttl_manager);
    async move {
        loop {
            cleanup_interval.tick().await;
            
            // Keep stats for last 24 hours
            ttl_manager.cleanup_old_stats(Duration::from_secs(86400)).await;
        }
    }
});

Cost Impact

Before Smart TTL (Single 5-minute TTL)

After Smart TTL (Intelligent TTLs)

Best Practices

1. Start with Conservative Defaults

SmartTtlConfig {
    default_ttl: Duration::from_secs(300),  // 5 minutes
    auto_detect_volatility: false,          // Disable learning initially
    ..Default::default()
}

2. Enable Learning After Understanding Patterns

SmartTtlConfig {
    auto_detect_volatility: true,
    min_observations: 20,  // More observations = better learning
    ..Default::default()
}

3. Monitor Analytics Regularly

// Log analytics daily
let analytics = ttl_manager.get_analytics().await;
info!("Smart TTL Analytics: {:#?}", analytics);

4. Combine with Static Patterns

// Use both automatic learning AND manual patterns
let mut custom_patterns = HashMap::new();
custom_patterns.insert("criticalData".to_string(), Duration::from_secs(30));

SmartTtlConfig {
    custom_patterns,
    auto_detect_volatility: true,
    ..Default::default()
}

5. Respect Cache Hints in Production

SmartTtlConfig {
    respect_cache_hints: true,  // Always honor developer intent
    ..Default::default()
}

Example: Multi-Tier TTL Strategy

use grpc_graphql_gateway::{SmartTtlConfig, SmartTtlManager};
use std::time::Duration;

let config = SmartTtlConfig {
    // Core content (balance freshness vs load)
    default_ttl: Duration::from_secs(300),
    
    // User data (moderate freshness)
    user_profile_ttl: Duration::from_secs(900),
    
    // Reference data (cache aggressively)
    static_content_ttl: Duration::from_secs(86400),
    
    // Live data (very short cache)
    real_time_data_ttl: Duration::from_secs(5),
    
    // Reports (expensive to compute, cache longer)
    aggregated_data_ttl: Duration::from_secs(1800),
    
    // Lists (ok to be slightly stale)
    list_query_ttl: Duration::from_secs(600),
    
    // Details (fresher data)
    item_query_ttl: Duration::from_secs(300),
    
    // Learn and optimize
    auto_detect_volatility: true,
    min_observations: 15,
    max_adjustment_factor: 2.0,
    
    // Honor developer hints
    respect_cache_hints: true,
    
    // Custom overrides
    custom_patterns: {
        let mut patterns = HashMap::new();
        patterns.insert("dashboard".to_string(), Duration::from_secs(60));
        patterns.insert("search".to_string(), Duration::from_secs(300));
        patterns
    },
};

let ttl_manager = SmartTtlManager::new(config);

Monitoring

Export TTL metrics to Prometheus:

// Export analytics as metrics
let analytics = ttl_manager.get_analytics().await;

gauge!("smart_ttl_avg_volatility", analytics.avg_volatility_score);
gauge!("smart_ttl_avg_ttl_seconds", analytics.avg_recommended_ttl.as_secs() as f64);
gauge!("smart_ttl_volatile_queries", analytics.highly_volatile_queries as f64);
gauge!("smart_ttl_stable_queries", analytics.stable_queries as f64);

Related Documentation

• Response Caching
• Cost Optimization
• Performance Monitoring