Cloudflare's Clairic Achieves 98% Accuracy via Confidence Routing
Cloudflare's Clairic model reaches 98% accuracy on most questions by using confidence scores to route tasks. Learn how intelligent routing works.
Source: Towards Data ScienceCloudflare has developed a system called Clairic that achieves a 98% accuracy rate on a significant majority of queries by utilizing confidence scores to manage model routing. This approach allows a smaller, more efficient model to handle straightforward tasks while reserving more powerful resources for complex reasoning.
How does the Clairic system work?
According to a report from Towards Data Science, the Clairic system operates through a tiered routing mechanism. It utilizes a model known as System One to process incoming requests. When System One generates a response, it also produces a confidence score.
If the confidence score meets a specific threshold—in this case, a cutoff of 0.9—the system accepts the answer as correct and delivers it to the user. However, if the confidence score falls below this 0.9 mark, the system recognizes that the query may be too complex for the smaller model. In these instances, the request is automatically routed to a larger, more computationally intensive large language model (LLM) called Luna. This ensures that difficult questions receive the depth of reasoning they require while maintaining high accuracy.
What are the performance results of this method?
By implementing this confidence-based routing, Clairic has demonstrated high efficiency and reliability. The system successfully achieved a 98% accuracy rate on 72.8% of the questions it processed.
This means that for nearly three-quarters of all user queries, the smaller, faster model was able to provide highly accurate answers without needing to escalate the task to the larger Luna model. This selective escalation allows for a significant reduction in latency and computational costs for the majority of interactions, while the 98% accuracy on those specific cases ensures that the user experience remains high-quality.
Why is confidence-based routing important for AI?
In the current landscape of AI development, there is a constant tension between model size, speed, and accuracy. Larger models like Luna are highly capable but are slower and more expensive to run. Smaller models are lightning-fast and cost-effective but are more prone to errors on complex logic or specialized knowledge.
Confidence routing addresses this by creating a hybrid workflow. Instead of using a "one size fits all" approach where every query goes to the most expensive model, or a "one size fits most" approach where a small model might hallucinate on hard questions, the system uses mathematical certainty as a gatekeeper. This allows developers to optimize for both performance and reliability simultaneously.
How can this be applied to real-world workflows?
This methodology is particularly relevant for scaling AI services where cost-per-token and response latency are critical business metrics. For developers building customer-facing agents, this means they can provide instant responses for common queries (like "What are your hours?") while still providing expert-level reasoning for complex troubleshooting without manual intervention.
In a production environment, this reduces the total compute load on the infrastructure. By resolving the majority of tasks at the "edge" or with smaller models, the high-power models are only utilized when they are most needed, preventing bottlenecks and reducing the overall carbon footprint and financial cost of the AI operations.
How can you explore multi-model routing in Grafy?
While Clairic is a specific implementation by Cloudflare, the concept of utilizing different models for different needs is a core principle of modern AI workflows. At Grafy, you can experience the power of different models side by side using Grafy Chat.
By interacting with various models within a single workspace, researchers and developers can observe firsthand how different architectures handle varying levels of complexity. This allows you to manually perform the kind of "routing" logic that Clairic automates, helping you determine which model is best suited for your specific creative or technical tasks.
Frequently asked questions
What accuracy did Clairic achieve?
Clairic achieved a 98% accuracy rate on 72.8% of the questions it was asked. This was made possible by using confidence scores to decide whether a question could be answered by a smaller model or required a larger one.
What is the confidence cutoff used by System One?
The System One model uses a confidence cutoff of 0.9. When the model's confidence in its own answer is at or above this value, the answer is provided; if it is lower, the task is sent to the Luna model.
Which models are involved in the Clairic routing process?
The routing process involves two primary models: System One, which acts as the initial handler and evaluator, and Luna, which serves as the larger, more powerful LLM that handles complex queries when System One's confidence is low.
Source: Towards Data Science — Clairic Hit 98% Accuracy on 72.8% of Questions via Confidence Scores


