Intelligent Caching

Ditana Assistant implements a sophisticated request caching system for both Wolfram|Alpha and LLM queries. This feature optimizes API usage and improves response times, especially for repeated or similar queries.

Key aspects of the caching system:

• Caches all outgoing requests to APIs.
• Implements an intelligent lifetime management for cache entries.
• Adapts cache lifetime based on the stability of responses.
• Initial cache lifetime depends on API: 1 week for LLMs, ≈ 11 minutes for Wolfram|Alpha.
• Particularly beneficial for internal queries made by Ditana Assistant, which often lack dialog history.
• Maximum cache size can be configured.

This caching mechanism significantly reduces API calls, especially for short, repetitive queries, leading to improved performance and reduced API usage.