Get faster, lower-cost AI answers by reusing earlier responses
Also listed as: Semantic caching & reuse
What is it?
Xelence plans to add semantic caching: when a new AI question means the same as one already answered, the earlier answer can be reused instead of asking the AI model again.
Why does it matter?
Many AI questions repeat in slightly different words, and sending each one to the AI model adds waiting time and running cost.
Faster AI responses for common questions and lower AI running costs.
Generative AI Services and Xelence AI Engine work together. Building AI and running it explains which does what.
Getting it into your solution
- Needs an upgrade to the Xelence release
- Needs configuration
Start thinking about
- Which AI questions do you expect staff or constituents to ask most often?
- Which answers change often enough that they should never be reused?
- How long should a reused AI answer stay valid under your policies?
Terms used on this card
- Large language model (LLM)
- The kind of AI model that understands and writes text, used behind AI assistants and agents.
- Semantic caching
- Reusing an earlier AI answer when a new question means the same thing, which makes responses faster and cuts AI running costs.
Items shown as planned are subject to change based on customer priorities for specific features and enhancements.