Faster AI answers, lower AI costs.
All IBIOS AI calls pass through a centralised AI gateway. Repeated questions are answered from cache, without a new AI model request. All providers covered from one control plane.
Semantic caching. Provider failover. Rate limiting. Cross-provider cost visibility. One gateway, all providers.
What happens when every AI call goes direct to the AI engine
Once the IBIOS AI receptionist launches (coming soon), picture a busy Monday morning when caller after caller asks it what your opening hours are. Without caching, every one of those is a separate AI request, each generating the same answer. With semantic caching, the first call generates the response and the rest are served from cache without a new AI request.
Calling several AI providers without a gateway also means managing separate credentials, rate limit responses, billing consoles, and failure modes for each one. A gateway consolidates all of that into a single layer.
The AI gateway sits between IBIOS and every AI provider. Caching, logging, rate limiting, and failover all happen at the gateway — before a token is consumed.
What does the IBIOS AI gateway do?
Three infrastructure capabilities that improve performance and reduce cost.
Semantic response caching
The gateway matches semantically equivalent queries against cached responses. A caller asking “When are you open?” gets the same cached response as one who asked “What are your opening hours?”, without a new AI model request.
Single control plane for all providers
All AI provider calls pass through the same gateway. Request logging, rate limiting, and cost visibility cover all providers from one dashboard rather than separate provider consoles.
Provider failover without code changes
If a provider is unavailable, the gateway routes to the next configured option. Provider failover is handled at the infrastructure layer — IBIOS application code does not need to handle provider-specific outage logic.
Cache hits in practice
Semantic caching catches questions that mean the same thing even when they are worded differently.
Cache miss — first caller
Caller: “What are your opening hours?” → AI engine called. Response generated. Stored in semantic cache. Token cost: applied.
Cache hit — subsequent caller
Caller: “When are you open?” → Semantic similarity match found. Cached response returned. AI engine not called. No new AI request.
AI usage metering: no surprise bills
AI costs are easy to lose sight of until the invoice arrives. The same gateway that caches answers also records every AI call, so usage is a dashboard you can read rather than a mystery.
- Every call logged with the model, input and output tokens, the feature that triggered it and the time.
- Daily, weekly and monthly usage by model and by feature, with cost estimates at current provider rates.
- Set a daily token budget or a monthly cost cap, with an alert before you reach it.
- At the cap, low-priority AI features can be throttled so the essential ones keep working.
Who does AI gateway caching matter to?
Any business running AI at volume where cost and reliability both matter.
High-volume AI receptionist deployments (coming soon)
The more calls handled, the more repeated questions arise. Every knowledge-base query served from cache is one fewer AI model request, which matters more as volume grows.
Businesses with cost-sensitive AI budgets
Semantic caching is the most direct way to reduce AI model spend without reducing capability. Repeated questions served from cache do not need a new AI model request.
Multi-provider AI deployments
Managing API keys, rate limits, and cost monitoring across several AI providers independently is operationally complex. The gateway reduces that to a single control plane.
Businesses that need provider resilience
AI providers have outages. Without a gateway, an outage in your primary provider takes down AI functionality until it is resolved. Gateway-level failover keeps the service running.
Technical specifications
Gateway
Centralised AI gateway — proxies all IBIOS AI model calls to configured AI providers from a single control plane.
Caching type
Semantic caching via embedding similarity. Configurable similarity threshold. Exact match caching also applied for identical requests.
Cache scope
Applied to knowledge-base queries, FAQ responses, and override lookups. Not applied to live-data queries (calendar, CRM, stock).
Logging
Every request logged in the gateway dashboard: provider, model, latency, token count, cache hit/miss, cost estimate.
Rate limiting
Per-tenant rate limits configurable at the gateway layer. Prevents runaway usage from a single tenant affecting service for others.
Provider failover
Gateway-level routing between providers on outage. No application code changes required. Cache continues to serve for applicable queries during outage.
Questions about AI gateway caching
What is AI gateway caching?
All IBIOS AI model calls are proxied through a centralised AI gateway rather than calling AI providers directly. The gateway sits between IBIOS and all AI providers. It logs every request, applies rate limiting, and caches responses. When the same or a semantically equivalent question is asked again, the gateway returns the cached response without making an AI request.
How does semantic caching work?
Standard caching matches only exact identical requests. Semantic caching matches requests that are asking the same question in different words. For example, “What are your opening hours?” and “When are you open?” may return the same cached response if the semantic similarity exceeds the gateway’s configured threshold. The embedding comparison happens within the gateway with no additional AI request.
Does caching affect answer quality?
No. Cached responses are the same responses that were generated for a previous identical or semantically similar query. The gateway only caches responses that were generated correctly — it does not serve cached responses for queries where the answer may have changed (such as live calendar or stock queries, which bypass caching by design).
Which questions are most likely to be answered from cache?
For a bathroom trade business, many inbound queries are repeated questions: opening hours, location, whether you offer a particular brand, how to book a consultation. These frequent, low-variability queries are the ones most likely to be served from cache. How often that happens depends on your own mix of questions, so the gateway dashboard shows cache hits and misses rather than relying on an estimate.
Does caching apply to all AI features?
Caching is applied to knowledge-base queries, FAQ responses, and response override lookups. It is not applied to queries that require live data: calendar availability, CRM lookups, supplier stock status, or any query where the answer changes between calls. These queries always generate a fresh AI request.
How does the gateway handle AI provider outages?
The AI gateway provides provider-level routing. If the primary provider is unavailable, the gateway can route to the next configured provider without IBIOS needing to handle provider failure at the application level. Cached responses continue to be served during a provider outage for applicable query types.
Can I see and cap what our AI usage is costing?
Yes. Because every AI call goes through the gateway, IBIOS meters each one: model, tokens, the feature that used it and an estimated cost. The admin dashboard shows usage by day, week and month, and you can set a daily token budget or monthly cost cap with an early-warning alert before you reach it. Metering runs in the background and adds no noticeable delay.
Which IBIOS product line includes AI gateway caching?
AI gateway caching is part of the IBIOS AI Brain product line. It is built for UK tradespeople and showroom retailers. See also: anti-hallucination AI.
IBIOS (Integrated Business Intelligence & Operations System) is made by IBIOS Limited (NI743167), Northern Ireland. Access is by waiting list; there is no free trial.
Repeated questions answered from cache.
AI gateway semantic caching. All providers, one control plane. Early access by waiting list.