Steve Siddall joins us to tell us about how Marks and Spencer is using conversational AI and the benefits it is bringing the business.
Presented by Deepgram
25555
Marks and Spencer journey with Conversational AI
Marks and Spencer, one of the UK’s leading retailers, has been using conversational AI across the web and contact centre for some years. Steven Siddall, Contact Systems Lead at Marks and Spencer, joins us to share the M&S journey.
00:00 Intro and Welcome to Deepgram
02:21 Introduction to Steve Siddall, Contact Systems Lead
07:20 The drivers for implementing conversational AI
11:14 Untapped customer demand
15:00 Proactive and personalised AI assistants
21:40 Multi-channel AI assistants
25:40 Business impact of having a chat bot
29:31 IVR automation journey
39:08 Big Bang vs incremental implementations
45:35 Implementation timelines
47:39 Business results from IVR automation
49:00 How poor traditional IVRs are
53:46 Steves advice to others on AI automation
57:37 Process and tools
1:06:40 Outro
Transcript
Kane 0:00
Hello hello hello ladies and gentlemen, boys and girls. Welcome to vux world. I’m your host, as always Kane Simms and today we’ve got a delightful Podcast coming up for you today. I’m just gonna get rid of this cable, where we’re going to be talking to Steven Siddalll who is the contact systems lead at Marks and Spencers. For those of you that don’t know, Marks and Spencers is one of the UK’s largest retailers and has been using conversational AI for quite some time in a bunch of different channels. And we’re going to be climbing into exactly how you would use what the lessons have been from Steven and his team and how you can apply the same lessons to improve your conversational AI initiatives.
But before we do that, I’d like to give a shout out to deepgram, deepgram is industry leading speech recognition technology companies up and down the world and all over the place are using deepgram proven industry proven automatic speech recognition, to power conversational AI, voice bots, and a whole manner of other transcription requirements from you know, call recordings and transcriptions to meetings and you know, transcribing meetings and assign interactions to people automatically and a whole bunch of other things can be enabled further down the pipeline. But it all starts with getting an accurate transcription, especially when you’re working with voice assistant technologies. Most organisations don’t even retrain their speech recognition models and they wonder why their bots are not performing as good as they could. And so with deepgram, you can retrain your models based on your specific industry demand based on the weather your customers speak based on your products and services. The accuracy is immense, it’s incredibly cost effective. The speed is unbelievable as well. So do check out deepgram.com/vuxworld if you do have some speech recognition or Asr requirements, that is deepgram.com/vuxworld shout out to deepgram.
Okay, so without further ado, let’s welcome today’s guest, Steven Siddall of Marks and Spencer. Steve, Welcome how’s things?
Steven 1:54
Good, thank you. Yeah,busy, busy as always, but yeah, so that’s why we like it.
Kane 2:00
No rest for the wicked as well, thanks for joining us, we appreciate you joining us. I know that you’re busy. And I know that you’ve got a whole load of stuff on your plate, especially when it comes to the conversational AI side of things. A lot of stuff going on at m&s lots of plate spinning and stuff like that. But I’m definitely excited to share with with the audience today a little bit about what you’re up to and stuff so maybe to kick it off, be good to always introduce yourself and and for those who are maybes a bit further afield, might not be aware of Marks and Spencer in the US and broadly in other parts of Europe and whatnot, might be interested in all sorts of just introduced m&s At the same time.
Steven 2:36
Yeah, sure. So just a bit about m&s So if you if you’re unfamiliar with m&s, m&s has been around for a long time in the UK, since the sort of late 1800s And, you know, where we started as a sort of a penny bizarre in lead market and, and grown from there really, and you know, we’ve got a long tradition of you know, in the UK have, you know, been synonymous with quality, innovation and we operate sort of in a number of international markets as well, but broadly, we’re in the sort of retail space you know, clothing, home food, and all that all that good stuff that everyone likes really, and there’s an interesting story on the on the website of our history about we one of the first retailers to sell avocado pears in the UK, and there’s a story I don’t know if it’s true or not, but there’s a story of a customer who, who, who tried to serve this avocado pear with custard by interesting, but I was having a look before this. There’s quite a lot of recipes now for avocado pears with custard on the internet.
Kane 3:39
Avocado pears. Are you talking about two avocados or a certain type of avocado? Interesting. Avocado with custard. I don’t know if I will manage, mind you banana and custard goes well, so maybe.
Steven 3:57
Yeah. So it’s interesting. But yeah, that’s sort of, you know, a bit about m&s Really, so a bit about myself. So I’ve worked in contact centres for about 30 years now in a variety of different sort of organisations. You know, I’ve worked embedded in the operation. I’ve worked for third party suppliers, I’ve worked for outsourced contact centre operations, but all broadly always in the field of sort of, you know, the systems that sit around contact centres to that, you know, are used to sort of make contact centres work. I’ve been with m&s About six years. And about four years ago, is sort of where we sort of started our LSR conversational AI journey, but I’ve sort of I’ve been exposed to it on and off, you know, in, in my career, you know, before it was even known as conversational AI and in the days were to be able to leverage speech you had to you had to be able to code
Kane 4:54
Voice XML and all that kind of stuff. Yeah. Interesting. So where did where did you offer Just kind of exposure or interest come from in this field then
Steven 5:04
as far as m&s goes so we’ve, we’ve sort of had a we, you know, we started so I guess with the new asked Chatbot. And about, you know, five years or so ago and, and that sort of evolved over time. And that was sort of, you know, very much in the, in the chat arena, about sort of two or three years ago, we started on our voice journey. And that was for a very specific use case to do with our store estate, where we believe there was an opportunity in the way that we set up our stores.
So historically, what happened is you ran in an M&S store, you sort of went through to one of 13, sort of centralised switchboards. And that was sort of a room in a number of stores across the UK, where someone would answer the car as you call and ask you what you wanted to do. And they would take that information. And then, broadly speaking, always just transfer you on manual transfer to the right area of m&s that might be within the store, it might be to, to the contact centre operation, or it might even be to a third party. And that was sort of looked at, you know, we thought sort of felt that there was an opportunity there to sort of go, Okay, well, could we take that switchboard operation and turn that into an automated way of where someone could actually say what it was that they wanted to do? And then we would transfer that call to the right place? And that’s sort of where we started on our own, I guess, our natural language voice journey, as it were.
Kane 6:40
Interesting. And what was the kind of motive behind that? Was it that you had lots of these switchboards dotted around all over the place? And the experience was a bit inconsistent? Like what was
Steven 6:54
There was a lot of things driving it a lot of benefits that we that we built into the business case, and, and, you know, some of it was about, okay, well, some of it was about leverage, you know, was the was the destination, we were sending out that call to that contact to the right, the best place to service it. So we always had that, that sort of customer service element in there. And, you know, consistency is obviously quite important about that, you know, anyone who sort of works in contact centres is, you know, is aware that, you know, when you try and sort of work with people, sometimes, you know, there’s a lot of processes that sit around sort of seeing what those colleagues are doing, you know, measuring the their effectiveness, you know, you’re always sort of looking at, you know, colleague transfers in a contact centre. And we felt that having, you know, a bot doing that, for us, was a way of both consistently and efficiently, efficiently, sort of getting that done rarely. And, you know, that was sort of a bit of a gateway then to, you know, whilst there are other ways then that we could then improve the customer journey by trying to automate those, those conversations a little bit more, you know, and getting the getting the customer the information that they needed a little bit earlier.
Kane 8:13
Beginner’s mistake. Interesting. So, so, let’s, let’s move numbers. We’ll come back on to that. But you mentioned that as well. You had a chat bot also. So you had an IVR journey beginning with call routing and kind of enhancing or augmenting the switchboard? Yeah, where did the Chatbot kind of situation begin?
Steven 8:36
Yeah, so that was sort of an opportunity that was identified, you know, relating to our E commerce site. So probably go back to the sort of the history of m&s, sort of, you know, it’s very much bricks and mortar retail, in the UK, you know, 600 or so stores in the UK. And that was our primary and primary sort of, point of sale. Now, you know, around sort of 10 years ago, you know, we started moving into this, you know, into the E commerce space, and, you know, got a website, you know, and you know, part of the future strategy of m&s is sort of digital first. And, you know, not to say that stores are always going to be, you know, not going to be always a part of m&s, but what we, what we felt there was that, you know, there was a lot of stuff going on the website where customers were trying to find answers to questions, and we could, you know, traditionally, if you look at the, you know, most website, you know, we’ve got help pages, we’ve got contact us, you know, contact us forms for email, we’ve got, you know, telephone numbers, if you want to ring us and stuff like that.
What we felt was the, there was a gap there that we felt that, you know, we had a number of, you know, live chat journeys that basically came from our Contact Us forms directly into a colleague. And what we then wanted to do was to sort of see what we could then do with a chat bot to see if we could then service those And those customers, get them again, you know, get them answers to questions, you know a bit sooner and really try to start to understand well, what are these customers? Actually? What are these? What are these customers trying to find out? Find them the website, you know, is there an opportunity to sort of help with the selling journey? You know, answer a question, but that’s sort of where that where that came from. It was sort of, you know, partly to sort of, can we service those contacts better? And, you know, can we leverage further opportunity in terms of, you know, what customers are looking at on the website?
Kane 10:35
Interesting, and did you find that there was a bunch of untapped demand there, when you launched that Chatbot Initially?
Steven 10:44
Yeah, I mean, it’s an interesting one, it’s sort of a, you know, when you start offering and chat on the website, you don’t know whether you’re actually generating more contacts than you would do before or, you know, you actually deflecting contacts from, you know, customers who are going to call you anyway, or customers who are going to email you anyway. So that would that’s quite interesting, and still is interesting in terms of sort of really understanding where those pain points are in the, you know, in the E commerce journey, where we can then introduce, you know, a proactive assisted offer. And, you know, a good example of that, that one of the first journeys that we did with the chatbot was where customers were, at the point in the journey where they were trying to, you know, added promo code to a particular basket. And, you know, some customers were struggling and then having to call and call or contact the contact centre, or Go in, go, go to a, you know, a live agent to try and assist them there. And that, that remains to the day to be one of the most effective automated journeys that we’ve got.
So what we do is we intervene in that now, based upon where we see behaviour to sort of say, we know, if a customer enters a promo code, and you know, it’s not, it’s refused, we offer, then that’s where we intervene with the chat journey, and sort of the What the bot does, it first tries to sort of offer us some, you know, easy to, you know, easy, you know, fault finding, you know, what, what, what might be going wrong, you know, have you tried this? Have you tried that, and then obviously, if that June, if that automated journey isn’t able to solve that for the customer, obviously, we still give the option to sort of speak to a live alive colleague in the contact centre, but that remains to the day to be one of our more successful journeys.
Kane 12:35
So, you’re essentially proactively recognising through on page behaviour, when someone might be struggling, and then you’ll just pop up the Chatbot at the right time to say, hey, is this something you’re trying to do? Is that right? Yeah, yeah,
Steven 12:49
that’s basically what we’re doing. And that’s probably more of what we want to be doing in the future. You know, if you look at what we do on the website today, it’s sort of very much like most journeys as sort of a blanket offering. So we sort of just go, okay, you know, if you need help, we’ve got a chatbot here that, you know, that might be able to help you. And, you know, on typical pages, like our help pages on our Contact Us pages, what we want to start doing is more of that style of proactively intervening in the journey based upon what it is you are doing on the website, and where we see, you know, issues occurring, and equally you know, what we know about you, if you’re logged into the website, you know, for example, you know, you’ve got a certain product in your basket, or you’ve been browsing certain pages, we’ll leverage that information, then to put you in touch with, you know, first of all, we’ll see if we can help with the Chatbot. But then, because we understand what it is that you want it that you’re actually trying to do, we can put you then in touch with a more specialist, you know, experts in that particular field. So for example, if you’ve got a, you know, if you’ve got a furniture item in your basket, for example, you’ve been browsing the furniture pages, and you may have been looking at, you know, product dimensions and stuff like that, that’s where we may leverage that information to sort of put you in touch with a furniture expert, and, and sometimes that might be in one of our stores.
Kane 14:15
Interesting, interesting. That’s a really good example of how it’s I think what most people do when they create a chatbot is they begin with that kind of use case that you alluded to there, which is the Hi, how may I help you situation? And what I think a lot of organisations fail to do is use the data that you already have. I mean, here you’re talking predominantly about web website behavioural data. It could just as well be CRM data, you know, the customers logged in, you know what the last purchase was, you know, the return to the website five days later, has it been delivered? Yes, it has, are they talking about returns are competent, and using the data that you know already to start being proactive and guiding or pre-empting that conversation, rather than always beginning from home I have a you kind of thing, you know? Yeah.
Steven 15:02
And that sort of, you know that, I guess Yeah, so we’ve very much focused on the servicing, you know, when, stuff perhaps isn’t working as well as it is we’d like it to, we feel there’s a, there’s an untapped area of more and say, more proactive style journeys in terms of you know, where we can potentially add more value by helping someone to try and find what it is that they’re looking to buy, to purchase, for example. So you know, where we’re sort of leveraging the technology, then, for more of that type of journey, you know, the value add journey, as opposed to helping when something hasn’t worked quite as well as it should have done?
Kane 15:43
yeah, definitely common coin, kind of forward in the customer journey to try and assist with the sales and the research and that kind of stuff.
Steven 15:50
Yeah. But again, you know, it’s, you know, what you can do with a, you know, you can do some simple stuff with, like, say, a blanket offering, but where the real value is in is sort of utilising that all the stuff you know, about that as that, you know, that that that person or that person? Or what’s actually going on, on the journey, you know, within the journey itself?
Kane 16:12
Yeah. What kind of so you mentioned the Chatbot, there started as, you know, a way of handling some customer support and stuff like that. What is the how you kind of measure that, because you mentioned that it’s quite difficult to determine whether the chatbot does prevent kind of website traffic from the call centre. So how are you approaching figuring out how the chat bots perform?
Steven 16:38
Yeah, so what we try and do, and there’s a lot of tools out there that can do this, you know, I guess, you know, the digital insights type piece, and, you know, there’s a lot of website analytics programmes available for that. But our vision around this, I guess, is sort of more around, okay, well, once you can sort of understand where those journeys are going wrong, do you then have the ability to intervene in those journeys, based upon what you understand where you can sort of go? Well, I’m going to intervene in this journey, but I’m not going to intervene in that journey, because it doesn’t quite meet the criteria.
So what we do is we look at sort of behavioural stuff on the website to sort of go okay, well, where our customers reach a contact journey, for example. So, you know, they’re coming into an email, or they’re emailing us. So can we trace that customer’s journey? Back up to the funnel to sort of understand, okay, so this customer here, they came into an email, they actually emailed us, where did they start? Why are they emailing us, and obviously, you can sort of understand that from, you know, the, the intent of the email, you know, the subject matter what’s in there, but then you have to combine that information with what you know about that journey, that that particular person talk, to get to that journey. And the same applies for voice, the same applies for chat. And that’s where, you know, we sort of got a good understanding of what’s going on on the help pages, but we won’t grow that level of understanding is, what did they do before they got to the help page, and then did that did that trigger of contact start outside of the website, so for example, you know, as with any sort of E commerce, retail, where’s my order is a, you know, obviously, a huge driver for contacting in operation. And, you know, that trigger for that customer to contact us, probably doesn’t happen on the website, because they bought the product two or three days ago, or some weeks ago.
So everything, everything up to that point went well, they didn’t have to contact us, but where they have to then decide to make a contact to us, because they’re waiting for the odds, you know, it’s it’s not arrived on time, then we need to be able to leverage that information so that when that customer is coming into that, where’s my order journey, so that we can sort of go, and that’s where you can start to do some really clever stuff. If you’ve identified that that person, you can sort of leverage that to change the either the offer or what you’re actually saying to the person. So for example, if we know that you have ordered something, and we can see, it’s not, you know, you know, it was meant to be delivered, you know, a certain time certain day, we see you coming into that coming into that journey, can we then leverage that information to sort of give you a more tailored start to that conversation to sort of go Hi, Steve, you know, you know, we can see you’ve ordered this product where there’s, you know, you contacting NGOs to sort of, you know, find out where this is, you know, where we can sort of start to really then, you know, so the government doesn’t have to tell us, we sort of try to inform them about what we think is most likely they’re going to be contacting us about similar stuff, you know, maybe stuff that’s going on in stores as well, you know, where a customer might have purchased something in store.
Kane 20:12
Interesting, we had a conversation with Jaya Kishore already from the CTO of yellow AI. And he was talking about this case where it’s almost like the reverse or the back end of what you’ve discussed there. So rather than somebody coming on to the website, you’ve already got some information about them, you can preempt what their query might be about, and then deliver that proactive kind of interaction, he was talking about a situation where the kind of customer is engaged with a chatbot, then decides to drop out for whatever reason, and then using that kind of data is that the right place to drop out there to get to the end of the conversation. And if not triggering, like a follow up SMS text that says, We know you got to hear you want to continue the conversation here. So I kind of got me thinking a little bit about using other modalities and other channels to your advantage. And as you kind of explained in there, someone, you know, makes an order, text, you know, movies or deliveries later, whatever the Chatbot needs to understand where to go to look for the information that says it’s too late, and what the new expected delivery date is. So I suppose there’s a question around is, is there a time when you think that other channels may be used to support that Chatbot? Because, for example, there’s no reason why that couldn’t be a proactive text message that says, hey, you know, sorry, we know your package is delayed, it’s going to be arriving within the next few days. What do you think about the usage of other channels? And that kind of omni channel automation journey?
Steven 21:42
Yeah, I mean, we, we, we definitely feel that, like a more proactive journey generation is definitely going to play a part, you know, in our strategy, sort of, you know, I mean, obviously, it’s doing a lot of the standard stuff to sort of go, and, you know, your orders not going to arrive on time and stuff like that in conjunction with our delivery partners. But then, you know, one thing that we, you know, we, we think there’s an opportunity there, but we need to do a lot of work to sort of understand it more is, you know, what a better journey, then be inviting that customer into a conversation with either a colleague or some sort of, you know, some sort of bot to sort of help them, you know, should they choose to reach out with that, as opposed to sort of leaving it very open? to sort of go well, we’ve let you know that there’s a problem, but we’ve not sort of given you, we’ve not suggested the best, the way that we might want you to get in touch with us if you see what I mean.
So leave it open to a customer that they’ll, you know, they’ll choose whatever suits them, which might not be the best way to service them from that perspective. And it also might not suit m&s as a business, you know, we might want to direct certain customers to a particular channel, it’s also to do with the time of day as well, you know, some of the stuff that we’re looking at, in terms of, not specifically, you know, with with conversational AI, but where, if you look at contact us options on the website, what we want to do is sort of offer the best, you know, the best way of contacting goes dependent on what actually actually what’s going on in the contact centre right now. So if we know, you know, wait times on voice , three, four minutes or so, but we know chats quiet, we then leverage that information to then offer, you know, the most approach, you know, offer a different contact option dependent on what it is you want to cop contact is about near for example, we still got a lot of customers who get in touch with us about where’s my order via email, obutemail, obviously, because of its nature, it tends to be you know, it’s going to be, you know, sometimes up to 24 hours before we get back to you.
So what a better journey to be able offers where’s my order intent? Would that be better sent into a real time channel because then we can actually give you the information in real time? Because quite often, by the time we get to an email saying, where’s my order, the order might have been delivered anyway.
Kane 24:27
Yeah. Yeah, of course. Yeah. Yeah. There’s something to be said maybe for automated emails using some NLU solutions. But the challenge is that they tend to drone on a little bit. So you need to do a lot of work and try to realise which part of the email is important. But yeah, that makes total sense to wrap up on the chatbot side of things. It sounds as though there’s definitely solving a real problem around the q&a and that kind of stuff. I think you mentioned that it’s starting to get into transactional use cases here and there. You’re working on making it More per octave and stuff like that. What’s the kind of impact on the business from having that chat bot in place? Like, what’s the how does how does the business determine whether it’s working?
Steven 25:10
Yeah, so we’ve got a number of measures, and these are sort of continually evolving. So interestingly enough, in the last year, we sort of moved away from the nuance chatbot to a different, different, different bar. And, and that’s, you know, that that’s got some, you know, that that’s allowed us to achieve, you know, a better sort of service rate, I guess, you know, successful service search rate. But I think probably what, what, over time, we’ve sort of evolved this, you know, so, when we started with this, we looked at it very simplistically as in, well, if you’ve not, if you’ve not escalated out of the bar, clearly problems been sorted. Whereas it’s not quite that, it’s not quite that clear cut. So you can’t just look at transfer rates to, you know, escalations to colleagues. And so what we might then do, what, what we then do is we sort of look at, you know, other things that indicate that so for example, we’ll ask a question at the end of that, you know, conversation, just as we do with a colleague, but we might then but we ask different questions.
So for example, for our chatbot, we sort of asked, Well, did we answer your question? And as opposed to a colleague post, and post interaction survey would be more around, you know, and, you know, obviously, you’ve got the traditional MPs, you know, you’ve got your brand, and you’ve got your, we have colleague MPs as well. But that’s, we use that in conjunction with Okay, well, this is this is the transfer rate out of that particular journey. But what are these other things that are saying that say, you know, what is the what proportion of customers in that journey are saying, you’ve actually answered my question, because, you know, what, sometimes what you can get is that frustration with the journey, so a customer just gives up on the conversation because they feel it’s not, it’s not working for them, and then they might appear then in another contact channel. So what we’ve started to do now is sort of look at those. Okay, well, I’ve reached out in, I’ve reached out in the chat bot, or the voice bot, for example, it appears that we’ve answered your question, because you’ve chosen to sort of hang up that journey, or you’ve you’ve gone away. But then do we see you appear in other contact channels, such as email, within a certain space of time, and then this allows us then to sort of get a more rounded view for over? You know, just because it’s not been transferred doesn’t mean that we’ve actually answered your question and started to look more about how you feel about that conversation as well. So looking at those, looking at the conversation itself, and looking for sentiment, you know, what’s the sentiment off the back of that conversation, and we look when we might be looking for frustration, in that in that conversation is, and all these factors then sort of need to be looked at in the round to sort of then judge how well that conversation is going from a customer perspective. And as well as combining it with, you know, what we’re trying to achieve from a business perspective.
Kane 28:21
Nice. Interesting. So on the voice side of things, then. So you mentioned that the that project started from looking at how to kind of augment the switchboard and how to make routing calls more effective knock on wood, if you can kind of walk us through that journey, like so you’ve identified a problem that there’s, there’s, I don’t know, maybe customers are getting either sent to the wrong places, or it’s taken a lot of resources to try and manage these calls, and a lot of kinds of time onto an average call time, etc, etc. Where did you start with, with how to approach that kind of particular part?
Steven 28:57
Yeah. So that it’s worth calling out that this was a, you know, it’s a big project in m&s, and, you know, I was part of the project, but it was a whole cross functional, you know, team that were involved in that, you know, you’ve got, you’ve got domain experts who know about customer, and, you know, you’ve got people who know about how the switchboard works, then you’ve got technical resource, you know, development teams. And, you know, initially, this was sort of, you know, the problem statement was sort of built collaboratively, you know, between the business problem, and then the, you know, the attacking the true technical people sort of look went out and look to see, okay, we think this is possible.
We’ve got a hypothesis to sort of say we can, we can replicate this, we can build this using technology and automate this routing journey based upon what it is you’re trying to do as a customer in the lifestyle and a lot of different products were looked at, you know, you Using a lot of different, you know, you sort of got your, you know, you’ve got your more managed service style of approaches, versus, you know, you sort of citizen developer type solutions, you know, and there was a lot of work there spent, you know, looking at that, and what would be the best, you know, the best thing for our particular use cases, one of the most important things for AWS as a as as, as I guess, the contact centre operations is that we needed to retain quite a lot of control and flexibility in terms of once we deliver this, we have to be able to sort of administer it ourselves and make changes to it quite often, you know, at short notice, depending on what’s happening within a particular store within a particular, you know, contact reason it is as it worked. So that was all sort of played into that. And eventually we sort of settled on a sort of a Twilio Google dialogue flow solution, and at the time, and Google dialogue flow was primarily a chat it was it historically, I think it’d been used primarily as a chat feature and using sort of what, what Twilio were doing, and, you know, the ability to sort of pick and choose from the sweetshop to sort of go, Okay, well, we’ll use dialogue flow, we’ll use speech to text from here, we’ll use text to speak from here, we sort of built that as a platform as a platform concept. And we then had, you know, you know, using our integration partners, we built interfaces, then the business users, you know, operational business users could administer the platform and give us that flexibility we still needed, which was incredibly valuable. You know, during the past, you know, that, obviously, what the world’s been through in the past couple of years or so. And then, you know, we had a very, we had a very clear plan as to how we would approach that. And that was very much form, you know, first of all, we’ll do intent capture.
So we’ll simply start asking a select group of people who were caught, sorry, a select group of stores were placed on this. And so when we just started asking the questions, tell us why are you calling? And interestingly enough, there was a lot of we did experiment a little bit with what question we’re going to ask, you know, in that journey, so we went through, one of the things we tried to experiment with with was, we asked, we started asking what what initially one of the experiments was we asked and tell us what it is you want to do today. And interested in offering insurance. This is just to do with the UK, we were getting some interesting answers off the back of that, based on British sense of humour, perhaps, you know, a quiet, you know, we saw a few swim with dolphins. And that was one of the things that we got in there, we felt that tell us why you’re calling you know why you’re calling us today, it was a better fit. And that’s, that’s still the question we use today. And then off the back of that, we simply started capturing that. And, you know, not doing anything with it first. So we just found out the colour and then that call would be sent through to, instead of sending it through to these regional switchboard hubs, we’d sent it through to sort of more contact centre type area.
They then would perform the same thing is, you know, what the, what the, what the store switchboard was doing. But also what they start to do is they’d start then to try and add value to that call. So for example, one of the one of the things that we used to get a lot of was people ringing a store, asking if a particular item was in stock in a particular colour or a particular size, and asking the store to put it aside for them, because they were going to, they were going to be down later. And that was one of the scenarios where we felt that, you know, that would be better serviced by sending that through to the contact centre, where we could then source that product from every, you know, basically, we could look for that product that the customer wanted across all our stores today, and from our E commerce side of things, and that way, we sort of then providing a better service, if you see what I mean, because we’re moving because quite a lot, that’s a lot of that time, you know, you go through to the store and it wouldn’t have it, you know, someone what would you do? And so that was where we started then to leverage a little bit of value, understand a little bit more about the core tags, because prior to prior to sort of getting the intent model in place, we didn’t really know what what all these reasons were, you know, it was very much the old five bar gate type approach and in and even even as far as you know, we didn’t actually know a lot about how many calls we were handling, within, you know, from from this room.
By starting sort of with that basic intent model, we then took all those utterances built in intent model, and off the back of that. And then we started to tune and check for accuracy on that model with the traditional sort of approach that you would take where you’d sort of go, Okay, well, let’s go and, you know, they’ve said this, we’ve matched it to this intent, did it did they say this, first of all, you know, the usual tuning activity, and then checking for accuracy to sort of events, we then started allocating those utterances to the intents that we believe they’re associated with them, checking that that was all working. And we, we sort of set ourselves a Sunday, you know, we weren’t going to start doing anything with that particular chord with that particular intent that was different, until we got it to at least sort of 80% accuracy. And today, our model is sort of that still sort of the minimum benchmark, but most of our intents are sort of in the mid 90s, in terms of accuracy of utterance to intent allocation. So we then sort of had our basic intent model, and we then rolled that across the entire store state, instead of sort of the select group of stores where we’re using off the back of that, and then we started to look for opportunities to then actually service the customers’ questions. So for example, you know, if someone was ringing up regarding storages, you know, stock availability installed, we send that through to the contact centre, because that’s a sort of a sales opportunity, value and opportunity.
If you were ringing regarding store opening hours, and we then sent you through to an automated IVR, where because we knew which store you called, we could give you the store opening hours, and we all started to automate those. And some journeys sell today are, you know, where, where we are, we’re, we’re giving you the information we think might answer your question you might have, but we feel there’s still a huge untapped resource in terms of further opportunities to service that contact, you know, you know, basically, you know, if it’s not just generic information we’re trying to give you, we actually want to try and automate the contact, if you see what I mean. So for example, if you want to track your order,, you know, we can provide an interface, and then we’re just sort of growing those journeys over time. And then leverage, you know, you know, leverage greater efficiency, and within those contact types, but again, still giving the customer the option to speak to a colleague, if, you know, if we if we’ve not been able to answer your question.
Kane 38:06
Nice, nice, that’s pretty good practice and is a really solid approach that starts with just generally capturing utterances that you don’t already have, building an intent model around that, and then looking for how to service it basically. And then, once you’ve got that up and running, you’re then just kind of looking at longtail things, or seasonal things, you know, that common goal, you know, here and there. How was the kind of process around the kind of gradual implementation because some, some organisational cultures have a kind of a culture, which is very much about the kind of big bang approach, spin once and all kinds of stuff developing it, the Big Bang launch. Other companies have that more of an agile culture whereby they’re more interested in doing a more incremental rollout.
With conversational AI, it’s very difficult to do a big bang approach, because especially if you’re starting a chord with no data whatsoever, you need to do an exercise to gather that training data via call recordings, or as you’ve done, like launching it and capturing live data. And then you need to then prioritise that dawnia. So which ones are we going to start with, which calls are we going to route first, which ones we’re going to introduce kind of some degree of self service with first and then we’ve when you’ve done that, you’ve then got the incremental improvement of the models on top of it. So it’s very difficult to have that kind of big bang approach, yet lots of organisational cultures work in that way. So I wonder whether you might share a little bit about how m&s and the culture and from your involvement in the project was that kind of incremental approach? Did it take some kind of convincing or was it already that kind of incremental?
Steven 39:42
Yeah, I mean, M&s. In terms of sort of, I guess, our our product teams, you know, they we have that in terms of an agile methodology anyway, I think, you know, even outside of, you know, conversation IR I think m&s as a as a general culture with a test and learn principle, and that’s what we try. And we try to do that I think comes from the sort of m&s as a company.
And in terms of, you know, we’ve always got customer in the firm to our minds, when we’re doing this, this type of stuff. So, there is a little bit of a fear sometimes then to, you know, this is such a big change for customers, especially, you know, with, with the nature of, you know, m&s has come to customers, you know, historically, maybe they had a, you know, the slightly older demographic, and then, you know, sort of the more, you know, like your, you know, your, you’ve put you sort of your new e commerce retailers in the market, and perhaps, a younger demographic.
So I think there’s always that care aspect in terms of wanting to go carefully with this type of stuff. Let’s do it on a smaller scale, first of all, and grow from there, once we start seeing what we’re what, what sort of response we’re getting. So one of the measures that we actually had at the start was one of the things we were worried about: What would people actually willingly interact with, with the bots? So, interestingly enough, that was one of the things that we had at the start was, what percentage of people just didn’t say anything? When we asked them the question, and then we then sort of, you know, initially we weren’t, okay, well, if they’re not saying anything, we’ll just send you right through to a colleague. Fine. And then over time, gradually we are introduced to a follow up question to sort of go, sorry, I didn’t hear that. Could you tell us why you are calling today. And then you know, we got an extra we got an incremental, you know, increase in people who then interacted with were willing to interact with the cert with the service. And then again, over time, we then go, and I think we started at sort of, like, 30% of people just didn’t say anything.
But what is interesting is over time, as people get used to it, because obviously, you can see the same, you know, we’ve got a lot of people who bring those, you know, you know, once a week, customers wise, you see more and more people then, you know, getting used to it, one of the interesting things we saw was that you ask the question, especially when you first make the change, and the first time a customer encounters this new experience, get our people hanging up. And then what we noticed was the ring came back a few minutes later, and then they say something.
And I think what it was was that people weren’t expecting it, they didn’t actually know what to say. And then they sort of go away, have a bit of a think about what they were going to say and then ring back. And then they were very articulate. They were quite articulate, quite articulate. And it’s interesting as well, some people completely opposite that, that we were getting sort of a bit of a we were getting sort of a couple of paragraphs. So you know, there was one I remember particularly where we got a story of why they worked. So I went in so I bought this jacket, but I got the jacket when we started its tag on. And I just want to know what that was, then, you know, because what we’re interested in, and that makes it really difficult for the CI, because it’s got, oh, goodness, what’s going on here? You know, there’s all sorts of things that this particular call is talking about. But people did learn how to use the technology over time, as well. So that was quite interesting. But I think that I couldn’t imagine m&s doing something like this as a big bang approach. Because it’s, it’s too, too risky. And, you know, we can move quite quickly, once we’ve got a little bit of information, it’s sort of, you know,
Steven 43:48
let’s learn a little bit, see what we find out, make a change. And we can, we can add new functionality quite quickly as we move along the journey. But we got our first intent model. After the intent capture, we had an intent model within four weeks. You know, once we started with the sort of low number of stores, we had that intent model, our it was live, it wasn’t a routine, but we were able to start measuring its quality, how good it was. And we actually started reading I think within a couple of months, and then over time, then it sort of grew and grew and grew and grew. But we sort of, you know, where we’re now where we’re at now is sort of, now we need more support from, you know, the traditional development teams, because to get the full value now we need to start integrating with API’s for better or the tracking for, you know, amending orders for certify that you know, the stuff that way you’ve got to do your it’s not just about surfacing some information. It’s actually about linking it to you as an individual.
Kane 44:58
Yeah, definitely. How long have you been doing that initial data capture piece for?
Four weeks.
Alright, so you had the four weeks of initial data capture plus four weeks of building an intent model, plus a period of time where you were just basically running the intent model, not escalating, just checking its accuracy and improving it and stuff like that, and then eventually rolled out the automated
Steven 45:21
the whole project to sort of get it to its initial business case met, the business case was six months, we moved pretty quickly in that first sort of month or two, you know, where we were where we started to route, you know, at the end of month two, and that was across the entire store estate. And one of the other interesting things is that, although the business case was only built for store calls, initially, and one of the interesting things is we then pretty quickly started to then leverage the same technology for calls to our contact centres as well. So from the commerce site, you know, all the numbers that come off the commerce site, we then started to leverage, you know, taking the same approach we use for stores. Interestingly enough, you know, totally different contact reasons that we get when someone’s trying to bring a store versus someone who’s trying to, yes, some of the same. But we had to build, you know, a slightly different intent model at first. So we had to intend models, going to store one, kind of what we refer to as a contact centre one. And then over time, we’ve merged those into a single intent model where we leverage and things we know about the call to sort of infer, you know, what is, you know, which intent it’s most likely to be dependent on where you call in from?
Kane 46:42
Interesting, interesting. So how is it performing? You mentioned, they obviously put together the business case and went through the whole process. Inevitably, it’s a living, breathing thing. Now that’s been implemented, and maybe we’ll get on to the next steps around the API access and integrations and stuff like that. But how, kind of how, what were the results essentially, on against the business case? Like?
Steven 47:05
Yeah, so if you look at this, yeah, so if you look at the store business case, I think our sort of self service, con, you know, call servicing objective was wasn’t massively ambitious, you know, we were sort of looking at sort of 10% of calls that we were looking to get in there, the primary use case was, can we accurately route the call. And you know, that was achieved, you know, at the end, really, you know, quite soon in the project, really. So we were, as I say, we will get in sort of that intent, utterance accuracy, you know, to the high 80s, by the end of sort of, you know, two or three months, and then tense that we had.
And then, you know, that was pretty much the, you know, the objectives achieved for the store piece. Now, obviously, then we, you know, the use as a living breathing thing, we then moved into the contact centre piece, which was a different thing we were trying to achieve that was sort of more focused on, you know, well, accuracy of routing, and, you know, could, you know, one of the interesting things is that is there is that, you know, traditional DTMF IVR, which everyone who works in telephony and contact centres will be familiar with, you know, we had across our line here, so, we had a number of DTMF, IVR, you know, you ring this number, you get six options, choose an option, you off the back of that, one thing that really braid brought home was the, how, how poor traditional IVR is, or actually, you know, doing what they’re trying to do, which is trying to sort of go you’re trying to do that, why is this person calling and getting them to the right place?
So we weren’t, you know, for example, we had several, you know, two or three IVRS each was, you know, three to six options, and dependent on, you know, where you call him up furniture, was it about store was it about, you know, e commerce type activity, we, when we did our model, we then went from, you know, that number of options to 160 different intents and that just sort of then gets it across as to how, you know, it’s it’s nearly impossible to build a DTMF IVR that will cater for all of those scenarios that our customers got in their head when they ring you and everyone will be used to this in terms of when you get an IVR you start you’ve got you’ve got your intent, what you’re trying to do in your head and what you’re trying to do as you’re listening to the options on the IVR and this is the best case scenario.
Some people just put one and take their chances. You’re listening to the options and you go is what I’m trying to do does it match option one better or is it match option three better. And so that’s sort of my coping mechanism, but it just brings home the fact that how, how really inefficient, they are doing this type of thing. Now, if you take, if you look at all your contact reasons, then go, how do you build an IVR? For all those contact reasons, you know, without completely overloading someone with, you know, huge messages, huge cognitive load that they’re just, you know, it’s just not gonna serve purpose.
Kane 50:29
Yeah. And that’s the value of conversational interfaces in general, to be honest, because you could argue that a website is the same thing. You’ve only got so much screen real estate, you can only have so many menu items before people kind of get overwhelmed or whatever.
Then the challenge is, how do you then deal with the breadth of language? And I think that there’s a lot of companies that are a little bit afraid of that because they think it’s a really big challenge, like, how are we going to try and understand everything that someone could possibly say, when when you think about it, though, as a business, how could you not go through that exercise? Because you will have insights now on what your customers are actually talking to you about what their actual real needs are? And you’ll have categorised and quantified those.
Whereas other companies that haven’t gone down this path, haven’t they just think that a web click on a web or on a menu item is a signifier of interest and intent. But it’s not really because you’ve aligned on the label that what the label is called, and the whether that’s translated into the mind of the user. So it’s, it’s, the challenge seems big, but it’s almost like, well, I’m bound to say this because I’m hugely biassed, but it almost seems like an inevitable challenge that has to be gone through, does it not?
Steven 51:36
Yeah, I mean, as well. And that’s just talking about getting the contact of the customer to the best place to actually answer that question. So even if you’re not trying to automate anything, that’s a huge challenge for contact centres. Anyway, being in voice has some additional challenges with cognitive load versus a website, you know, in terms of time and stuff like that. But it’s a challenge anywhere. And that’s even before you then start thinking about how do I then look at opportunities to automate these contacts, because you cannot automate a contact until you’ve understood what it is the customer is actually trying to achieve? And you cannot do that in the voice channel yet, you know, the website is slightly different. But you are, again, as you say, You’re reliant upon a person looking at all the information that’s available there, and making the right judgement, based upon what your will the information you’re giving them there on the screen and conversational interfaces is just better at getting to the heart of
Kane 52:37
the problem. Yeah, definitely. In order to do that automation, understanding the front end is one thing, understanding what someone is talking about and what their query is. And understanding and being able to classify that intent is one thing. The other thing is being able to have the infrastructure and the career capabilities to be able to facilitate that automated transaction. You kind of alluded to this earlier on around, you know, the next step in working more closely with the developers and all kinds of stuff. How much of that are you wondering whether you can shed light on the current situation at m&s, and any kind of advice you would have on companies approaching that automation stage and the importance of having your house in order when it comes to API availability and all that kind of stuff?
Steven 53:23
Yeah, I mean, it I mean, obviously, it depends on you on your organisation m&s being around for so long, we suffer from what a lot of companies suffer from as in, we’ve got a lot of disparate systems that hold all this information. And part of the, you know, there’s a lot of work within m&s going on at the moment where we, where we’re trying to bring everything together in a single place, where we can sort of this is the, this is the centralised view of all this information about customers to sort of, then well, you know, that we can then start plugging stuff in interfacing into to get that really valuable information that you need, if you’re really going to sort of going to crack automation. And you’ve got to have, you know, the, you know, those teams in place and you structure you’ve got to be set up in the right way where, you know, we’re we, as a, you know, we feel that the, as an operation is in the contact centre, we’re the best pipe, we’re here to produce the problem statements and provide the data to sort of go, this is our top reason for contact, you know, this is where we’re trying to get to, and then taking that problem to the development teams and sort of then going, Okay, well, this is this is the information we need in order to then automate this journey.
That’s where it becomes a lot more collaborative because the way that we were set up, we’ve set up our teams in m&s is we’ve sort of made a conscious decision that the people who administer our administer tune, monitor their accuracy or monitor the quality of their all. And I guess you’d call domain experts as in back there, from the customer side of things, if you see what I mean, they’ve all got a huge breadth of knowledge in terms of how m&s operates as a business, they’ve all come from the customer side of the operation. So at some point, they’ve all been involved in some shape, or form with handling customer interactions themselves. And we feel that those are the right people to be managing the experience monitoring the experience, but they’re not, they’re not developers, they don’t code and to be able to really get the full value, it’s a matter of, sort of pairing those those people up with the people who can do the integrations, you can build the web hooks and the plugins, and the integrations to where that information is held. And then build interfaces, then that they can leverage to then really start to you know, you know, use those conversational journeys and get access to that information. And then, you know, build the build, build the journey out, based upon, you know, let the techies do what the tech isn’t good at and let let letting our teams do what we’re good at, which is, you know, the customer journey itself. Hmm.
Kane 56:43
Interesting. You mentioned that before autopilot and dialogue flow. And you mentioned that when you began it was dialogue flow as presumably before they’d released CX. Yeah. Wondering whether I know we’ve only got a couple minutes left. But I’m wondering whether you can walk us through some of those processes and tools that you might use from that kind of management, maintenance ai ops kind of perspectives, or whether there’s any learners that you might have to share on that.
Steven 57:12
I mean, certainly. So we feel we, you know, we’ve still got tons of stuff we want to do, and by no means were we perfect there. So I think, because we sort of taught this isn’t to take and to teach customer facing people, the technical side, the you know, the tools, that then means that we you know, you have to invest in teaching those people about conversation, design, teaching those people about AI training, teaching those people, it’s not just about learning how to use the tool, it’s those skills that come along with those tools. And so we’re still use data flow, yes. CX is we feel as if that’s a, that’s a tool set that we will need to get us to where we’re where we’re trying to go and on our journey. And, you know, partly because of that, you know, you’ve got that ability to really closely tune those intents and journeys, you’ve got, you know, comes with quite a lot of out of the box analytics stuff to sort of really on to really understand, you know, where our record was talking about the website before, but where are where are the pain points in the journey? Where are we seeing, you know, ambiguity in conversations? Where are we? Where are people dropping out into a default in turn, what was their journey they took prior to that, and it gives us sort of that love further level of control. We’ve tried to build a by plugging into the role of data.
We’ve tried to build the capability using tools such as Power BI, to sort of provide us that sort of view of the journey. And then for us, for the team to then drill into where the journey is maybe not not going to you’re going along. But CX type products come with a lot of that nowadays, you know, comes with that. So that’s where we feel as if that will be advantageous for us. The other thing that I think we were talking about before the call was, at the moment, we don’t have a we don’t have a consistent voice talent within our journey. So we’ve got a mix of I guess, branded static messaging combined with TTS voices. And we feel as if you know, that’s another area we want to develop into so we can give a consistent voice whether it’s a static message So we’re giving you, you know, when you when you’ve left the dialogue flow part and the Twilio part of the journey, and you’ve come into the, you know, the more generic contact centre journeys, we want to make that journey a bit more consistent in that respect. Now as well, that’s one of the things that we’re quite keen on investigating now within, you know, what’s available in the market, and move, you know, to sort of do stuff around that and then looking at all are looking across all our channels, you know, our chat. And I’ve always been to sort of go, what, what is the persona we’re presenting? That’s one of the things that we didn’t think enough about at the start to sort of go, what is the identity of the m&s bar? What are we trying to portray, about this? You know, just as we try, you know, our colleagues are all, you know, we invest a lot of time about, you know, what m&s is about, you know, what are the values that are valuable to us? We think there’s an opportunity around in our automated channels to sort of, you know, do that, you know, I think a lot of a lot of companies sort of really good at that, you know, you’ve got, you’ve got your dominant you’ve got gone from Domino’s is very, very, you know, they’ve got that brand, right, within their particular conversational AI channels. That’s where we feel that, you know, we want to be doing a lot more work in that area.
Kane 1:01:26
Yeah, yeah, definitely, it could definitely help. I mean, it can help increase your improving engagement and stuff like that can help make it, you know, more memorable and debate and the level of personification that a different use case requires is quite interesting. But where I’m really keen to see things head is, you know, it’s the same as an actual live agent, you don’t get the same personality every time you call, and you don’t always get different conversations that require a different approach as well. And so at the moment, I think most organisations, they’ll do a person personality design, I would argue, most of them kind of do it a bit too high level, which is like this name, Sandra, she comes from California, and that was a doctor and all that kind of stuff.
Whereas there’s some really good frameworks out there from United, Rebecca evanhoe, and Dana debo and stuff, which really concentrated around the specific use case, which enables you to do things that have a personalised voice. But flex the persona, based on the type of use case, or if someone’s calling about a return or to make a complaint, the persona might behave slightly differently to if someone’s calling to make a sale or whatever, you know, that’s where I’m kind of really keen to see a development beyond that kind of initial persona into a persona that is able to flex to the use case. And in future, we’re probably a bit the technology exists now to do this, but I think we’re a bit behind as far as the adoption of this, but hooking up things like sentiment analysis, and even things like there’s a whole bunch of different things you can do with conversational intelligence to understand the conversation or speaking style of the caller. And pairing that with an agent that matches that conversational sequence speaking style, and can respond based on the sentiment as well. Like, I always say that this stuff is about to get more complex before it gets simpler. But definitely there’s some big value to be had in that, I think.
Steven 1:03:14
Yeah, and I think that that’s, that’s, that’s where we’re sort of we’re looking at now in terms of conversation, ai ai, in general, we’re sort of looking okay, well, we’ve, we’ve had a lot of focus around, you know, customer conversations now, how can we start to leverage that to benefit the colleagues who are, you know, talking to customers in all in all our channels, you know, very interesting stuff around, you know, what, anyone who’s in Microsoft Teams, where, you know, any, you know, obviously, predictive texting, you know, has been around for a for ages. But there’s a lot of stuff in that area at the moment in terms of where, how can we because we’re understanding what’s going on in the conversation. And that might be a live chat or live voice conversation or an email? Can we then leverage that to help the colleague better? And more efficiently answer the question, or more help them find the right answer to the customer question in the first place.
So you know, where, for example, you know, taking what we know about the conversation with the ultimate bomb, and passing that the key bits of information. And if we have to escalate, we’ve passing the key bits of information, we’ve learned about that conversation to the colleague and for example, we know it’s about where’s my order, and we know it’s about this order, but proactively offering guidance to the colleague about what they need to then do to resolve that to the result that for the for the customer and sort of really sad when I made the conversation more efficient, but also try and not make as many errors, you know, can we help up, you know, Microsoft vivre if anyone’s used that really interesting that it’s reading conversations that you’re having via email, and it’s reminding you that you’ve said you are going to do something for this person and have you done it or not. And that’s really fascinating in terms of how we then start to leverage that for our customer facing colleagues, because the end of the day, we’re always gonna have customer facing colleagues, they’re always going to be there.
Kane 1:05:26
Yeah 100% 100% And there’s such an untapped area there for the agent assists and things which reminds me of the webinar that we’re running with Cory I for those of you tuning in. We’re running a webinar with kore AI next week. It is on the sixth of May, I’ll double check . I don’t want to be lying. Fifth of may. 5th of may we’re gonna be talking about agent assist technologies and how that can help improve agent productivity and customer experience. Please do check that out. vux.world/dot/better /together if you are listening to the podcast. Steve, this has been absolutely amazing. Thank you so much for tuning in. I know we had some questions coming through here in the chat, which we haven’t had time to get to. So apologies to my apologies, Yakov for not managing to get there. But I think we were in Florida during that conversation. It was a bit difficult to break stride. But thank you so much for joining has been an absolute pleasure. Thank you so much.
Steven 1:06:22
Thanks, Kane.
Kane 1:06:24
No probs, and everybody else we are back again next week. As always. Next week, we have one the cooler weather with Korea, as I mentioned, and on Tuesday we’re talking to Brad Cleveland, who is an independent consultant working in the contact centre space. We’re going to be talking about all of the contact centre and CX trends that are occurring in 2022. So far, including, obviously conversational AI. So looking forward to seeing you again there. Without further ado, Steve has been an absolute pleasure. And we’ll see you all next week. Thanks. Bye Cheers.
