I spend a lot of time working with and speaking to businesses and tech companies that are shaping the future of AI in customer experience. And, guess what? They’re all saying the same thing: hybrid AI is the future of customer experience.
In this article, I’ll explain the emerging hybrid AI techniques that enable businesses to benefit from the immense power of generative AI, while maintaining certainty and consistency in areas that matter, such as critical information, business processes, and logic. I’ll also share eight factors to consider when deciding whether to go all-in with generative AI or to use a hybrid AI approach.
Some of these learnings stem from a recent conversation I had with Anne Jenkins, Vice President of Solution Architecture at Teneo, as well as the numerous conversations I’ve had recently on the VUX World podcast with companies such as Lloyds Banking Group, Toyota, Essent, Prudential, Swisscom, and others.
What is hybrid AI?
Hybrid AI is where you use a combination of AI and non-AI solutions in the same application to create the best possible experience for a given use case. It’s where you combine deterministic rules, programmatic workflows, predictive machine learning and probabilistic generative AI. The idea is that you use the absolute best components suited to the specific task or sub-task you need them to perform. You use the best of each technology to offset the weaknesses of the other.
How hybrid AI works in practice
A hybrid AI solution works by architecting a solution based on what each component does best. You use generative AI when you need natural language understanding and generation. For example, recognising customer intent from an open-ended question, or generating a human-like response. You use rule-based logic or workflows where accuracy, structure and reliability matter, such as enforcing a payment process or handling authentication.
Hybrid AI in real-world applications
To illustrate a realistic example, imagine that you use generative AI at the entry point of your application to understand customer intent. Then, you use generative AI to direct the user to the most suitable agent or capability to resolve their needs. However, let’s say that you then need to authenticate the user before you carry on. Here, you might use a deterministic, rule-based component to ensure it works the same way every time.
Let’s also imagine that the business process underpinning the conversation is high-risk, such as transferring money. You might use rules and scripted dialogue to manage this process.
Then, you might end with a generative AI capability that wraps up the conversation or transitions to the next intent. You might even use some generative AI throughout to answer contextual questions during the journey or for specific dialogue handlers, such as grounding or error recovery.
“The longer a conversation goes, the more bleached your context window becomes. That’s where traditional memory and structure come into their own.” Anne Jenkins, VP, Solution Architecture, Teneo
You see, a conversational AI solution is more than simply throwing everything at a large language model and letting the model do all the work. It requires thought and diligent design to create the most effective solution.
Use the right technology for the right job
With a hybrid AI solution, you can leverage the strengths of each technology to cover up the weaknesses of others.
For example, generative AI is brilliant at sounding natural. So it’s good to use for some elements of dialogue. However, it can hallucinate, so in high-stakes conversations, where variability isn’t acceptable, scripted responses offer certainty.
Generative AI is great at broad understanding and broad instruction following, but poor at narrow control. Rule-based machine learning solutions are the opposite. Great for narrow control, but poor at broad understanding. That means that generative AI might do a better job at understanding and orchestrating, yet determinism may fare better at a process-level to ensure consistency at scale.
Generative AI is good for light reasoning, such as following simple business logic. However, in extremely complicated scenarios, you introduce more chances of hallucination. Here, where complex business logic is needed, programming this would remove any risk of drifting.
The role of generative AI (probabilistic) and rules (deterministic)
You should think of the LLM as your hyper-intelligent front-of-house assistant. It’s great at handling unstructured input, making the experience feel fluid and stitching together disparate bits of language into something meaningful. However, it still requires company policies, systems, and processes to lean on. It can manage this in many low-risk scenarios, but when things get more risky, business rules and deterministic workflows are required.
“Sometimes you just need good old-fashioned workflow around the edges to keep things on track.” Anne Jenkins, VP, Solution Architecture, Teneo
This blend gives you the best of both: the natural experience of a human conversation, with the reliability and repeatability of enterprise systems. It’s not about replacing one approach with another. It’s about stitching them together to create something more powerful, scalable and production-ready.
What use cases benefit most from hybrid AI?
When assessing when to use a hybrid AI approach vs end-to-end generative AI, you have to consider a number of factors:
- Risk severity. How likely is generative AI to make a mistake in your use case? And what’s the impact of that mistake? If your use case is something like planning a trip abroad, then is it the end of the world if the system makes up the name of a city every once in a while? Probably not. What about if the system approves a loan for someone who has a terrible credit score and can’t afford the repayments? Higher risk = more determinism required.
- Complexity. How complex is the business logic in your use case? Generative AI has been shown to be capable of handling simple reasoning tasks. So if your business logic is fairly straightforward, you might find that generative AI managing conversation state is acceptable. If you have complex logic that can’t be easily explained in language, with many long tail considerations, then rules might perform more consistently.
- Data security. Most AI agents that are performing something useful need to collect data from customers, then either push data into a line-of-business system or retrieve data from a line-of-business system. This data needs to be handled with care, securely, and cannot be leaked, stolen, or processed irresponsibly. Depending on the type and amount of data you’re processing, you might want to use determinism to handle this component.
- Duration. How long is the conversation likely to run? Is your use case a short and quick interaction, such as checking on a delivery? Or is it long, like completing an income expenditure form for a debt collector? The longer a conversation is likely to last, the more you’ll need to manage the state of the conversation. Generative AI tends to break down after ~10 turns.
- Regulatory. Are you in a regulated industry? Are you required to provide specific information in a given use case? Do you have to prove that you’ve handled an interaction in a certain way? Typically, regulated industries benefit from some degree of determinism to satisfy legal and regulatory requirements.
- Experience. How important is it to provide a great experience to users? Of course, this should be the most important consideration in all use cases, however, sometimes you might need to sacrifice the ideal experience for one that’s actually feasible to implement. If experience trumps, generative AI can certainly help make the conversation more fluid and natural feeling.
- Cost. How often is this conversation likely to happen? Can you afford to pay for generative AI inference every single time? LLMs can be expensive at scale, particularly for high-frequency use cases. If your use case is triggered hundreds of thousands (or millions) of times per month, relying solely on generative AI could become costly or inefficient.
- Channel/Modality. Which channels are you deploying this on? And what are the UI/UX constraints of each? In an app, you can display lists, date pickers and buttons, which reduces ambiguity but requires rules. With voice AI, you must handle everything through natural conversation. Hybrid approaches allow you to optimise by channel: use structured flows where you can and gen AI to smooth over the parts that need natural conversation.
Reviewing these eight considerations will enable you to make an informed decision about the type of solution to build. Note that the kind of solution you build might change on a use case by use-case-by-use-case level. In some use cases, you might opt for an end-to-end generative AI capability. Others might require a hybrid AI approach.
Putting it into practice
Hybrid AI isn’t a fallback or workaround. It’s a deliberate design choice. For businesses serious about scaling AI in customer-facing applications, the hybrid AI model offers a reliable, controllable and cost-effective way to deploy intelligence where it matters most. The future of AI in customer experience isn’t one or the other. It’s both.
Listen to the full conversation between Anne Jenkins and myself on the VUX World Podcast. To learn more about Teneo and its hybrid AI platform and approach, you can check out the Teneo website.
