As of now, using LLMs in high-stakes enterprise customer interactions could be considered a risk. Although there are more robust ways of mitigating the hallucination issue emerging, it’s never going to be a 100% solved problem. If LLMs have direct access to private data, without an intermediary platform in between the model and the user, then this is a significant security risk.
So how do companies get the benefits of generative AI, without compromising customer experience and brand reputation?
Let’s consider the safer use cases for LLMs that are available to you right now.
We’ll counteract low-risk with some risky use cases, and tell you why those are best avoided with the currently available technology.
These insights into LLMs were shared by Amelia’s Brandon Nott, CPO and Nick Orlando, Director of Product Marketing, during a recent VUX World podcast.
AI + human = less risky
One of the best ways to mitigate risks with LLMs is to limit the data they can access, and which use cases they’re used for. For example, if the LLM has no access to PII, then you don’t have to worry about that data being mistakenly used to train a model or exposed to other users.
Another place you can use LLMs safely is information summarisation – so long as that information is safe to share! We’re talking about summarising long documents for sharing, or having calls summarised for a live agent so they have a good overview of what a customer has previously spoken about. These are safe because the information is only available internally within your organisation, and it will be filtered by a human, who should be able to spot inaccuracies.
As Brandon says, “anytime you’re using generative AI, which is then going to a human to make a judgement, you inherently have a safety net.”
Another possibility is to have an internal RAG model for employees. Imagine a chatbot that answers employee questions about holiday entitlement and other aspects of their job. You should have the information clearly defined on an internal document. So long as the company’s materials are in order, and there isn’t lots of conflicting information, then someone asking how many holidays have I used up this year should lead to an accurate response. It shouldn’t matter how complicated the company is. Even an international company should be able to have an employee-facing assistant that can accurately answer HR questions, depending on which employee it speaks to, and their local labour laws.
Going even further, it’s possible to generate code fairly safely. AI copilots can speed up the coding process. Again, the most important part of the process is to have a human check the results! So long as you’re doing that, errors would be caught before they go to a live product.
Hallucinations and prompt injections
Let’s consider the other end of the scale. What are the riskiest uses of generative AI?
According to Nick, “really the two biggest risks with generative AI are hallucinations and prompt injection attacks.”
Hallucinations are when the LLM fabricates its response, which appears to have little or no relationship with the facts. If a brand will be held accountable for the things said by AI, then hallucinations are risky. Imagine an AI assistant that invented regulations for insurance claims, for example? That’s a level of risk that no company wants.
With advancements in RAG capabilities, using guardrail-based prompt chaining and having a platform like Amelia between your user and your model, you’re able to mitigate the risk of hallucination in ways that weren’t possible even 12 months ago.
That said, the use case severity will still dictate when to use generative vs deterministic technology.
We don’t want to ignore the risks. If we get it wrong, Nick suggested, “it’s going to make people lose trust in these systems. It’s going to make people more afraid to use them, and it’s going to make people say we need to pump the brakes.”
With a prompt injection, someone attempts to trick the LLM into revealing something it wasn’t supposed to. Of course you don’t want to share a customer’s private information with a third party. It’s an ethical and legal disaster, and the bad headlines would likely get plastered all over the internet too.
This is why you need to be careful about the kinds of data the LLM can access. If you give it private data, for example healthcare or banking information, then there’s a risk someone will be able to access that data.
It’s on the practitioners to help people use LLMs to their maximum benefit, and help them navigate around the issues. We know risks exist. We have to be ever-mindful of them, so that we can use this technology well.
What does ‘good’ look like?
Amelia has considered what is the best process to use at each stage. It’s not about focusing on one technology for everything (for example using an LLM rather than an NLU). It’s about using things if they’re the best choice for that particular job, rather than aiming for ‘one size fits all’.
As Brandon says, they always ask themselves questions before they start. “You’re looking at your use cases, right? Do I need this to go the same way every single time? Do I want to add flavour or colour to a conversation? Do I want to generate code which will be reviewed by a human? Do I want to provide an answer on the fly? And as you look at these use cases, it becomes apparent pretty quickly which methods you want to use.”
There’s no on/off button for risk. It’s a matter of reviewing the use cases, how you’re solving them, and considering the best ways to mitigate the risks within.
Thanks to Nick, Brandon and Amelia for these insights! You can watch the full interview on Linkedin, YouTube, Spotify, Apple Podcasts or wherever you get your podcasts..
