Ayesha Saleem joins us to discuss how Instacart has brought in the use 0f conversational AI, as well as some conversation design best practice.
Presented by Deepgram
Deepgram is a Speech Company whose goal is to have every voice heard and understood. We have revolutionized speech-to-text (STT) with an End-to-End Deep Learning platform. This AI architectural advantage means you don’t have to compromise on speed, accuracy, scalability, or cost to build the next big idea in voice. Our easy-to-use SDKs and APIs allow developers to quickly test and embed our STT solution into their voice products. For more information, visit:
How conversational AI is used for Instacart
Instacart is the world’s largest online grocery service. Ayesha Saleem is the Conversation Design Manager and joins us to share how Instacart is using conversational AI, the team and resources involved, the results they’ve achieved and best practice tips for designing and implementing conversational AI solutions.
00:00 Intro and welcome Deepgram
3:14 Introduction to Ayesha Saleem
6:25 Creating voice assistance and voice bots in the mortgage industry
11:10 Can chat bots check credit scores?
14:20 The current channels that Instacart is using for conversational AI
17:35 Who does the shopping and the delivering
20:05 Business models
23:20 Where the conversational AI came from for Instacart
27:06 What kind of demand Instacart customer support gets
30:20 How quick conversational AI has come into use
36:40 Agent escalations
38:47 How the current capabilities have changed since Ayesha started
41:45 Does Instacart bots have a personality?
51:19 Properly designing the conversations well
54:30 Advice on scaling conversational AI teams
1:02:20 Biggest mistakes beginners make in conversational AI
1:03:40 Outro
Transcript
Kane 0:00
Hello hello hello ladies and gentlemen, boys and girls. Welcome to vux world. I’m your host Kane Simms, and I am delighted to get into today’s conversation with Ayesha Saleem, of Instacart. Conversation design manager of Instacart. We’re going to be talking all about how Instacart is using conversational AI, some conversation, design, best practice, and a whole lot more. I’m sure, we’re gonna get into that. But first, before we do, let’s first give a shout out to our presenting sponsor. Deepgram is automatic speech recognition technology. And one of the best in the business. If you are looking for speech recognition technology to fuel and power, your voice assistants and voice bots, or even any other use cases that you might need transcription for think about call recordings and transcribing things like meetings and all that kind of stuff. Deepgram is literally one of the best in the business. It’s incredibly cost effective. It is incredibly quick, which is imperative when you’re creating conversational experiences. You don’t want to wait five minutes for the thing to be able to understand you and respond back and crucially, which is what I’ve got. I’ve been talking about this quite a lot recently. And last, the last episode of the podcast we spoke to rob Cunningham, who was the innovation manager and LNER, and he explained how he when we implemented a digital avatar in Newcastle trainstation had to train the ASR automatic speech recognition to be able to cope with different accents, different dialects and things like that. This is one of the things that not many organisations do. When they create voice capabilities. They don’t tend to retrain their automatic speech recognition to be able to understand the dialect of the people that are speaking to it. The type of jargon or colloquialisms or product types that you have, you know industry specific jargon, you can retrain your speech recognition models to increase the accuracy based on your specific use cases that then gives your NLU greater accuracy which means that your bot and your assistant has a greater chance of understanding people. That is what deep ground enables you to do and not all speech recognition providers enable you to do that. So please do check out deepgram.com/vuxworld if you would like to learn more that is deepgram.com/vuxworld thank you deepgram for presenting the VUX world and if you’re not subscribed to the VUX world yet please do so it is vux.world/subscribe, you’ll get all of the invites to events that we have like this with industry experts like Ayesha who we’re gonna get into a conversation with in just a moment, and you’ll get all of the summaries all of the articles that we publish and also summaries of insights and news that are happening within the industry on a weekly basis. That’s Vux.world/subscribe. Now, without further ado, please welcome Ayesha Saleem conversation design manager at Instacart. Ayesha welcome.
Ayesha Saleem 2:42
Hi, thank you so much. I’m so excited to be here.
Kane 2:45
No worries. Thank you for joining us. It’s an absolute pleasure. Yeah, well, well. Well, looking forward to this one. So thank you for spending the time with us. Yeah, me too. I’m excited. Cool. Cool. So it’s an interesting one in scotch, it doesn’t exist in the UK. But we do have very similar services. HelloFresh is one gussto is one. So for those in the UK, you might start to get familiar with what Instacart is, from the people in Europe, Asia, South America, stuff like that. I used to wonder if you can give us first of all, a bit of insight into what Instacart is. And tell us also a little bit about yourself as well.
Ayesha Saleem 3:21
Yeah, for sure. So let’s start with Instacart. So Instacart started as a delivery service, which obviously was so important during the pandemic, essentially, we’re able to deliver any grocery to anyone in the United States, and now Canada as well, in a very, very short amount of time. We have been expanding tremendously in the last year. So now we’re doing all sorts of deliveries, we can do flowers, we can do pet supplies, we can even do makeup, which is super clutch for the people that are so for reference. But the business model is essentially a delivery system. So I’m not sure if the UK has, like Uber Eats or Postmates or anything like that, but very similar to that, but just a broader scope and what we can deliver.
Kane 4:09
Interesting, interesting. So it’s it sounds sounds like very much like UberEATS or Deliveroo, which I thought delivery was everywhere. But apparently it’s not. I don’t think delivery exists in the US, does it? Yeah,
Ayesha Saleem 4:20
I haven’t. No, I have not heard of that one.
Kane 4:22
Right. It’s very similar. Yeah. It’s weird because I thought delivery was everywhere. And then I moved tuned in 50 miles north, and delivery just doesn’t exist. You just can’t get anything for love or money. But yeah, so that’s interesting. So we’ll, we’ll get into that and what we can kind of discuss that a bit more. But first, tell it tell us a bit about yourself. So you’ve been working in the conversational AI space for quite a while. I’m looking at your LinkedIn here you’ve got experienced clink, which is a decent company, very innovative. We’ve had Jason Mars on the show in the past which is really good. So tell me about yourself and about your experiences, how you got into interested in conversational AI?
Ayesha Saleem 5:03
Yeah, for sure. So I actually stumbled into conversational AI to be completely honest, I was interning at a company called General Electric. And I was building Alexa skills and voice assistants working with dialogue flow, even before it was called dialogue flow. I think at that point, it was api.ai. I absolutely loved it, I thought it was an amazing field, got to do a little bit on the development side, then moved over to the design side, there wasn’t really a role like conversation designer. So when I was at Slack, I was actually doing like user research, and a lot of what ended up being conversation design on different platforms and different technologies. And then I actually was hired in as Rocket Mortgage as first conversation designer, even before they knew what that was, I really helped build that role. And in that, in that field, the mortgage industry, which I think is, is very unique to the United States, as well, I was building voice assistants and chatbots, that chat bots that helped people, like qualify for a mortgage helped people through the entire mortgage journey, the mortgage journey is incredibly complicated, and to be frank, like a pain in the ass. So chatbots and voice assistants were definitely very helpful. They’re built that built that practice up at Rocket Mortgage, and I just recently left to join Instacart as a conversation design manager. I have been here since February, and I’m loving it.
Kane 6:22
nice. I can imagine mortgages being particularly difficult because it’s one of those industries where a lot of knowledge kind of just exists in people’s heads, a lot of the rules and regulations exists and people ask people’s heads and a lot of the questions that people ask can be quite kind of specific and longtail and specific to their situation and stuff like that. And a lot of that did, it doesn’t really always exist in places does it like that content and stuff doesn’t really exist all the time. It’s trapped in people’s heads on in various documents and stuff like that. So that must have been a bit of a challenge, trying to assemble. One is the design, but the design is nothing unless you’ve got the actual infrastructure to to cope with it. So I wonder whether, you know, it was rocket mortgages organised in that sense, or was part of what you had to do this to figure out how to, you know, start organising the content and how things are architected and stuff like that?
Ayesha Saleem 7:19
Yeah, that’s a great question. So I think rocket was better than a lot of traditional FinTech companies that I’ve seen, I think FinTech as a whole, because it’s so highly regulated, you know, as their, their willingness to use, like new software and new technology, and even things like getting on the cloud is a little bit like, it takes a little bit longer for them to do so. And because of that a lot of a lot of the things that you can do with technology, you just can’t do in the finance space. Rocket was a little bit better than that we didn’t have to re architect too much, I would say , there was two big challenges. The first one is that with mortgages, it’s so dependent on like, personal situations. So the beard entry for people to use chat bots, and actually get value from these things, is so high. So for example, if you want a mortgage, I can’t tell you anything without knowing what your credit score is in the United States. So I can give you like a range with, Hey, your interest rate might be like 2.5%. But I’m probably not going to do that. Because 100% Depends on your credit score, how much assets you have in the bank, which is really private information and trying to convince somebody that they need to pull credit. You know, using a chatbot is incredibly scary, especially because in the United States, and with Rocket Mortgage as well, our user base skewed a little bit higher, so early 50s. And those are people that really, at least like our experiences were that they their willingness to use, like new technologies and trust new technologies, like conversational products, was a little bit lower than, you know, like the millennials and the Gen Z generations. So that was definitely like a very interesting problem as being able to add value, but needing those personal like, you know, data points before doing so and having to explain to people that like, hey, we can help you. We just need these really scary pieces of information first. And then I would say that second thing is really similar to that is everyone’s like, mortgage journey. And everyone’s information differs so much based on so many different factors. So for example, it’s it’s tax season right now, in the United States. Everyone hopefully finished filing their taxes on Monday. What you owe in taxes and how taxes work for different home properties depends on so many things. It depends on what state you’re located in the type of property you bought, if you bought like a single family home, versus an investment property, how much you paid for that property, and then even stuff like if you have solar panels in your house, you’re gonna have a tax cut, because the government kind of incentivizes things like that. So being able to integrate with like a million different API’s to be able to actually help people and not just your people like status content, status content, static content? Sorry, I tripped over that a few times. Is is a little bit tricky. And so there’s a lot of technology that goes into building a chatbot. that’s meaningful. It’s not just like, you know, content that could like potentially help them that has to be very, very contextual.
Kane 10:26
Interesting. I mean, yeah, in, the finance industry, there’s, I mean, talking about credit scores and stuff like that you’re looking at, yeah, getting data from a whole bunch of different places, how much of what you would do in rocket mortgages, for those types of use, I’m assuming the Chatbot was able to check credit scores and stuff like that.
Ayesha Saleem 10:46
So we had it, so we were able to pull credit. But we pull credit with a third party. So that’s kind of So how it works, those people consent to give us their credit, like us pulling their credit. And then we were able to automatically pull credit and then pull that, like back into the experience to be like, Okay, can we check your credit? And we see it’s about a 750. Here’s what you qualify for.
Kane 11:10
Yeah, I’m with you. Yeah. So how much of maybe as you would speak to this from all of the different kinds of roles that you’ve that you’ve had throughout your kind of conversational AI career? How much of what a conversation design does in your experience is purely focusing on just simply what the heck’s not simply what purely focusing on what the experience is, versus all of the requirements that are needed in order to fulfil it because it’s pointless designing something that’s, you know, really fancy and flashy and checking credit and all this kind of stuff if you can’t actually deliver it. So where does your rule begin? And where does this technical spec in and future plan like, prioritisation creep in?
Ayesha Saleem 11:58
Yeah, that’s a great question. So I think there’s like stages of building out our conversation. And so and it’s also like, a give and take. So I would say that when we’re when we’re building out an entirely new experience, I like to look at ideal state, like, what could this look like, we have, you know, access to like, every single data point, and all of these API’s are open for us to use. And we have, you know, like engineers that are willing to work with us and have the time, and really like fleshing out what does that experience look like? And then obviously, you know, we all live in the real world, unfortunately, things are not as simple as you know, they are on paper. And so scaling that back and saying, like, okay, what are the biggest value adds in this experience? And what are the things that you know, we have to have in order to ship what are some of the things that are nice to have? And then what are some of the things that could be a V two, or we need a lot more, you know, like data engineers or software engineers in order to get. And so sometimes the ideal experience is not work at trips, oftentimes, the ideal experience is not work at shift. But I think it’s really important to design for that ideal experience as well. So that you kind of have like, a vision that you’re working towards.
Kane 13:05
Yeah, that’s good practice. And that’s, that’s very, very akin to the traditional kind of service design methodology, like in an ideal world, how should this thing work, ideally? And then what do we actually have today that we can actually deliver? And kind of work? But what are the must have fundamentals that without which this whole thing falls down? What are the should have things that we think were really important that will make the experience but not critical? And then what are the could have things that are kind of the bells and whistles and stuff like that? Is that the kind of the typical approach? Yeah, that was exactly right. Yeah. Nice. Nice. Interesting. So Instacart, what are the kind of current use cases and channels and stuff like that, but the Instacart is, is utilising from a conversational point of view.
Ayesha Saleem 13:54
Yeah. So we have a bunch of different things going on, I can talk about the things that are currently live, we have two versions of a chatbot for two different customer bases. So Instacart has a lot of different, like, users that use Instacart. For various reasons. We have retailers, we have shoppers, we have customers, and right now we’re live, we have a live chat bot for both shoppers and customers that are very different. So in the shopping world, it’s more like help and support. So shoppers that are you know, like checking, for example, say like, you place a grocery order with us, we’re going to have a shopper go out and actually like pick stuff out for you. And they might have questions throughout the process. For example, their credit card might decline for whatever reason, or they get your address and it’s an apartment and you haven’t left an apartment number, you just left the complex, things like that they can contact a chatbot and we’d be able to very quickly help them and either with like content or with actually like automate API automations that we have built in so that we can very quickly like answer their questions. on their customer side, there’s you want to add to your grocery order, your shopper has started shopping, and you’re realising that, hey, they’re picking up some of the wrong things or, you know, they made an exchange of, let’s say, like french fries, or sweet potato fries, and you hate sweet potatoes will be able to help with things like
Kane 15:19
Interesting. So talk us through that kind of customer journey then so that people are able to actually see the shopper going around and picking up things as you go along.
Ayesha Saleem 15:35
Yeah, we actually think that’s core to our business, because we don’t want shopping to be like a black box. So essentially, what would happen is you place an order through the app, and schedule a delivery time. And we let you know when your shopper begins shopping. So you got like a little app alert, and you’re able to see them in real time to pick things out. So for example, if you did order those potatoes, or those French fries, you can start a replacement in case something goes out of stock. Huge problem in the United States with COVID, almost everything goes out of stock, Everything good is out of stock. And if you know they’re not able to find your particular potato brand, your french fry brand. And they find a different one, they can ask you like hey, is this okay? You can approve or deny it. Or you can say, hey, it’s up to the shopper, like I don’t have the time to like, you know, watch them shop, let them pick whatever like replacement they want. Or refund me if you have a specific item, just give me the money back and keep going. Obviously, with all of those different options, people still have, like support questions. And that’s really where the Chatbot comes in.
Kane 16:40
Interesting, I didn’t realise that was the case. So it’s predominantly if on the customer side, the use case is predominantly customer support. On the cost on the shopper side. It’s, it’s for the for the moments where they’re actually doing the picking of the shop in. And also, there’s the shopper, the same person that actually does the delivery as well.
Ayesha Saleem 17:04
Yeah, so it’s shopper support its end to end shopper support. So there’s, there’s so many different things that unfortunately turn around your hours to change in their grocery store that happens all the time. With with COVID With just like grocery stores, you know, hours change, you might get there and realise that they’re not open, we would help you out with that, or customer to cancel their order or change their delivery address, the layout of the store could have changed. And you can’t really find anything. We provide our shoppers with credit cards, where we load the right amount of money on there. And every now and then suppose you you watch your shop or shopping and you realise you really need laundry detergent and you completely forgot you would be you would say you would tell them hey, can you also pick up laundry detergent, our bot would be able to add to their cart, like their credit card, so that they would be able to successfully checkout and not have like a bounce back. Because you know, the initial amount is more than the estimated amount or whatever. And then with delivery, delivery addresses are sometimes an issue. So I’ll be able to support shoppers through the entire experience up until they got paid
Kane 18:12
Interesting. I’m starting to get the grips of it as so that’s interested in that so so Instacart essentially employees, shoppers, to go into the stores and actually do the shopping and pay for the shopping in the stores. Interesting. So what it is in the UK, there’s similar services, what they do is actually the retailer itself will integrate into the platform. And so the finances Basically they’ll go through what would be the Instacart or the delivery or whatever, but then there’ll be passed on to the to the retailer. In this instance the finances go to Instacart Instacart gives it some of that finance to the shopper who then goes to the shop to pay for it.
Ayesha Saleem 18:58
That’s exactly right. We do have a retailer option as well. But primarily our business is customer to shopper
Kane 19:06
right that is really interesting that because that means that you can then you can then scale into different types of shopping very quickly, can’t you because you’re not reliant on onboard and retailers
Ayesha Saleem 19:19
Yes, that was actually the reason we did it this way. So I’m not sure if you’re familiar with Trader Joe’s at all.
Kane 19:25
I’ve heard of Trader Joe’s but I’m not familiar exactly with how it works and stuff.
Ayesha Saleem 19:29
Okay so Trader Joe’s is one of those like insane business models that’s not on the internet at all. And if they almost have like a cult like following they’re incredible. It’s it’s I would say it’s like a whole foods competitor but a lot better and they come out with their own products everything that they stock is like Trader Joe brands. So they buy directly from suppliers they stock only Trader Joe brands and absolutely delicious but they don’t have an online catalogue. They don’t even do like email marketing. They Don’t do sales, it’s just you visit their store, and you get what you need. And you leave. And it’s a very, it’s a very interesting business model. And it works very, very well for them. And one of the biggest things like it’s a huge millennial brands, huge Gen Z brand as well and with people want to shop at Trader Joe’s, and you never know what’s in stock and what’s out of stock till you get there. And it’s very, very busy all the time. So the way that Instacart was actually founded was our co founders went into Trader Joe’s and took pictures of every single product, and actually created a catalogue themselves. And people were ordering this was in San Francisco, we’re obviously you know, not a lot of people have cars, you’re kind of like holding your grocery bags and getting on the BART, which is like the public transportation. And it’s all very messy and very difficult. People were ordering through Instacart. And we would actually have our co founders go into the stores and shop for them. And we and then very quickly started scaling to different grocery stores. And so what’s really nice about it, is in a lot of areas like regionally in the United States, there’s like very popular grocery stores. So for example, like my favourite grocery store is like a mom and pop shop like down the street. I absolutely love it. I don’t shop at big, like retailers, and their, it’s very easy for them to get on Instacart, they have to do very little. And it’s very easy for a shopper that shopping at those bigger grocery stores to also just swing by my mom and pop shop, pick up what I need. I got it to me.
Kane 21:26
Interesting, that is really good. That is a business model that would work pretty well over here as well, actually, I think. Because yeah, you’re not relying on onboarding the retailers, you can just basically have someone go to wherever, wherever you want to go to and pick some stuff. But that’s a wicked. So So from from the conversational automation side of things, and most kind of like most established organisations that are trying to utilise this technology. Obviously, there’s lots of different reasons that they would, they would utilise it lots of different reasons that they would explore it. But one of the large kinds of trends is around customer support customer service, stuff like that. And it’s usually that they’re trying to, you know, deflect contact for my expensive channels, like call centres and stuff like that are they’re trying to improve the conversion rate of the website. So you know, to increase revenue or whatever Instacart law sounds as though it’s well and truly a digital native company, it was built on the internet, it is fundamentally a technology company, was the aim of this initiative, similar to what most other organisations would have an aim in in terms of trying to reduce customer support costs and things like that, or was it always kind of there from the beginning as a weird part of the customer experience for the shop run for the customer? Like, wondering one thing I know, you’ve only been there since February in terms of the history of why where it came from?
Ayesha Saleem 22:52
Yeah, so their conversation AI department at Instacart, or even the strategy at Instacart is actually very, very new. It started in November. So I’ve actually been around for a lot of it, which is funny because I’m very new as well. And it was very much created in order to really like save money in terms of customer support, and a large part due to COVID. Because obviously COVID Just like changed our business model and changed how we do things tremendously. With the rise of COVID, more and more people were placing online deliveries on Instacart saw like a huge amount of people who are not only, like relying on us for their groceries weekly or even daily. But people who wanted to shut like use our app to beat become shoppers, because the idea of you know, having that income that’s kind of dependent on their own, like time and their own workload was very attractive to a lot of people in the United States. And so our customer support times and our shopper support times just went through the roof in terms of call volumes. And from a monetary ROI standpoint, having someone on the phone and having someone you know, like available to help on a phone is a lot more expensive than a chatbot. And so that’s really how the conversational AI programme started at Instacart was an A B test where we launched a chat bot that was able to help in a couple of different use cases. And we were like how much of the call centre volume? Can we deflect into this new strategy? And how much of like, How much money will that save us and it was ROI positive almost immediately. And that was without any, like any I’m not gonna say any thought but a whole lot of thought and strategy was just a very, very simple, like MVP, AV test, you know, six months later, and we’re still seeing positive ROI with every single automation that we do. And I think that’s really changed the mentality of what we’re trying to do with conversational AI. When I got to Instacart, we were all about like automations and being able to deflect volume from call centres and I think automations are obviously great. But when you look at the print The role of conversational AI, it’s about speed, efficiency, personalization. And when you apply that to the Instacart model, and really, it’s like tech companies everywhere, or just like companies everywhere, efficiency, and optimization, and personalization, are things that everyone’s kind of attracted by and everyone needs. And so I think when we think of what we can do with conversational AI moving forward, our stuff is not only call centres and support centres, but it’s also being able to really influence like different areas of the app. And using those principles of conversational AI to just like, move our business forward. And then to like, obviously, we’re in the 21st century, but even like, just making us more modern and more, more like in line with what our users want, I guess.
Kane 25:47
Yeah, bringing it bringing it from support into call kind of a car modality, basically, you know, for shoppers to be able to find products and all that kind of stuff. They are the whole shopping journey. Hmm, interesting. So what does it kind of look like as far as so Instacart, you mentioned over COVID began obviously, growing because a lot more shoppers are wanting to use the service can’t leave the houses, things like that a lot more. When I say shoppers, I mean customers in actual shoppers want to kind of, you know, out of work furloughed, whatever it might be might have been laid off trying to trying to kind of like find jobs and stuff like that. What kind of demand does Instacart typically see from its customer support? Like, I’m presuming it’s must be a fairly, you know, very high volume part of the business.
Ayesha Saleem 26:42
Yeah, and that’s the interesting thing about customer support is it vary so much, and there’s spikes and dips. So it’s very, very hard to predict. For example, Easter, like Friday was insane, we saw three times the amount or call volumes, and we do want a typical Friday. And that’s again, like why I think conversational AI is so great is that it is scalable with those, you know, like large spikes and like depth. So it’s really, it’s really hard for me to put like a tangible number behind like our call volume and our chat volume. It’s in the 20 1000s every day, and then every now and then it just like shoots up and you look at this graph, and you’re like, Oh, I wonder what happened that day?
Kane 27:25
Yeah, it doesn’t. And what kind of use case so you’ve kind of covered some of the use cases. But in terms of the scope of you mentioned there, that when you started your account, it was like an experiment, an MVP, let’s see if we can do it, you know, presumably some of the high volume, low complexity kind of use cases, see if it sticks, and it started to stick out of this kind of scope and breadth of I know, you’re talking about putting it elsewhere in the customer journey and elsewhere throughout the different channels and stuff. But think of just about where it began on the customer support side, out of the whole breadth of use cases that that exist. What how much of that do you think is covered within the chat experience? As it stands?
Ayesha Saleem 28:07
It is a very good question. So our, like North Star for customer support is anything that an agent could do on the phone, the chat bot should be able to do. And I would say we’re about 40 to 50% in there. And like you kind of talked about in the beginning of this conversation, having the tech and the API support and all of that, for being able to do everything just doesn’t exist yet. And that’s why it’s a little bit, it takes a little bit more time. And then we also have fallback methods and our support options right now, where, for example, if you come into our chat bot, and you say like, I need help with XY and Z, and we don’t cover that yet, we default to help articles. Because we have a content team that writes help articles for shoppers, and our customers as well. And that’s a little bit more manual. It’s not a wonderful experience. But it does get the job done in the sense that you can then read an article that tells you exactly what you need to do if you need to, for example, you’re shopping and you’ve had some sort of emergency and you can no water complete the order, you would manually go through it and be able to manually go through that article and be able to help yourself out. So the Chatbot does link to all of those different things. But in terms of like being able to do things within the chat bot itself and that ecosystem, it’s about 40% I would say.
Kane 29:31
Interesting. So you began in November 2021. Was it? And you started in February 2022. And it’s now April, so that’s November, December, January, February, March, so six months, and you managed to manage into over 40% of use cases. That’s pretty quick go in that.
Ayesha Saleem 29:53
Ya know, for sure. And I will say one of the most interesting things and I think this is a pretty common UX principle the whole like I always look at percentage as confused, but it’s like 80% of the things that are used or filer 20% of the product. I probably weren’t at that very incorrectly.
Kane 30:11
80% of customers use 20% of the features.
Ayesha Saleem 30:15
Yes, much better than. And that’s that is very much true. And I think it’s so interesting. I’d like the customer support world, because you’ll look at the heavy hitters, they’re just astronomically big compared to some of the things are a little bit smaller. So like, one use case for us was like 20 25%, or call volume.
Kane 30:34
Yeah. Interesting. Interesting. So is that the was that the trick? Would you say not trick, but is that is that the thing that’s expedited a lot of this is that you were going for the really big hitter use cases? Or was there anything else that you think has contributed to moving at a relative pace with this stuff?
Ayesha Saleem 30:54
Yeah. So we started out really intentional with what we started, like where we really started with, and that’s because, obviously, this was new. And we didn’t have that stakeholder upper management, you know, full buy in quite yet, because it was such a new technology. And it was such a new initiative. So we really did start with figuring out what are our highest call? What are the reasons why people contact support the most? How much money can we save if we deflect and let’s let’s do that. And so really making that like ROI case, like that business case. Now we have, we still continue to do that. But we also look at other metrics. So one of them that is really important for CSAT customer satisfaction scores. So if we have a specific automation and the CSAT is super low, we’re getting a lot of complaints about it, we’ll look into it sooner than we might look into something else. But our two or two KPIs that we look at our self service rate, how how, what’s the rate that the user was able to go through this flow and actually completely self service? And then what’s the customer satisfaction score that they gave us after they were done with the experience?
Kane 32:05
Interesting. So how do you monitor? Is this something specific you’re looking for when you’re trying to find the self service? Rep. Do you monitor like, the final intent in a conversation? Or have you got like a bunch of intents that you’ve identified as Okay, when these ones are hit? That’s the end kind of thing? Or is it something you get from Atlanta Business System, when something’s been happening has always happened? On the back end? How do you quantify that?
Ayesha Saleem 32:31
That’s a very good question. So it is a combination of things. So the first one, like you said, are those intense? So if you hit like a thank you in 10? Or if you hit a specific area on the flow? Or you’ve given us feedback, like Did this help you? Yes, it did. No, it didn’t, then you said yes. We’re signifying that as we helped you, self service rate is complete. But obviously, not everyone gives feedback. So there is there are a large number of users that that got to like an area where we gave them the information that they would need to self service. But we have no we didn’t get any feedback on if it was actually helpful for them or not. And that’s super, super tricky. And the way that we really measure that is in a few different ways we look at or are they did they successfully do the thing that they set out to do? So if it was a shopper, did they deliver that order? Was that order delivered was that order delivered on time, and what was the rating that shopper got from that customer because similar to all of the apps we’ve talked about, you know, our customer can rate a shopper like five stars or four stars. And we look at those to kind of signify, okay, if they if they got to a point in the in the experience, where we gave them the information that they needed. And we’re able to see throughout the whole user journey, that they successfully did the thing that they wanted to do, then we’re marking that as success. Similarly, we, one of the one of the coolest things I find about Instacart is all of our data is very connected. So we’re able to really quickly kind of like see how the Chatbot and how our voice assistants affect the rest of our systems. So one of those systems is our support system, and our ticketing system. So if you’re a shopper, or let’s say you’re you’re a shopper with Instacart. And you go through the Chatbot. And we don’t help you, and you get to an agent, immediately, a ticket would be created for you even before you get to that agent or chat bot has created that ticket and assigned it to that agent. So you can very quickly work on your case. And when they’re done, close that ticket. If you don’t have the ticket created, it means that your problem was most probably solved. And so if you go through our experience, you get to a content block, or you know a piece of information that we think signifies that we’ve helped you, and you don’t go on to create a ticket with us. And that means that you don’t call us you don’t you know, like text us on a different line because we will be able to track all of that. Then we mark that as success on that contributes to our self service rate.
Kane 34:54
Interesting. That’s very sophisticated. That’s really good. I love that the whole kind where that ties together in terms of tickets raised and the tickets open when the agent gets to it, and then it can be closed there. And then which is, you know, that’s, that’s really good because half most of the time, most organisations are having any level of support, you know, tickets raised, it sits there for five minutes, then it sits there for two days, three days, someone eventually gets around to it. And this guy, like, you know, just sits in limbo, whereas just yeah, I’m in the agent with the context of the conversation, getting resolution and then closing it off in the space of one conversation is wicked. Yeah,
Ayesha Saleem 35:31
yeah. And I think it’s, it’s really nice that the handoff is very seamless, and that the customer support team and the chat bot team have the same goals, and that we’re working very closely together. And I think the user journey is very seamless because of that. Even if you were to, for example, like get frustrated with our chat bot, and, you know, like, close it and call in, we would have that information. And we would know that okay, you chose to go the phone route. And we would work to make sure that in the future, that doesn’t happen. And we’re able to help you in like the channel that you originally chose.
Kane 36:04
Interesting. Do you have that kind of agent escalation on the customer side as well? Or is that purely on the shopper? Side?
Ayesha Saleem 36:12
Yeah, that’s a good question. So we started with shopper, we’re rolling out to customers. So these are things that exist, maybe don’t exist right now, but will exist in like 24 hours, 48 hours a week or month, we’re rolling out very quickly, we’re actually rolling out something this afternoon, which was very exciting. So they might not exist to the same sophistication as they do with shopper. But our plan is to get them there as soon as possible.
Kane 36:40
Interesting, interesting. That’s wicked. So you’ve got two examples in the chat channel. And you mentioned before your experience with voice and beginning with Alexa skills and stuff, does Instacart do anything on the voice channel, as well as as a voice assistants or call centre calls and stuff like
Ayesha Saleem 36:57
that? Yeah, so we do have a call centre. And we do have something that we call shopper voice, where shoppers who are interested in being connected to a live agent can do so we’re actually exploring, like the voice assistant capabilities there. It is a little bit complicated because of Canada. So I actually didn’t know that. So a couple of weeks ago, Hugh Beck has laws with the French language. So any support that’s launched in English also has to be launched in French in French, which is delaying us a little bit, because we’ve never done anything with different languages, just because we’re primarily in the United States. And so that is something that we’re that’s an obstacle that we’re currently overcoming. And pretty soon we will have a worse channel.
Kane 37:47
Nice, nice. Yeah, translation does cause some, some complications, definitely, especially with, you know, entities and stuff like that. Whereas in one language, you know, the whole kind of use of entities in languages is entirely different, potentially. It’s quite complex. So what is the what is the sort of like, what does the makeup look like over there when you started? What was the current sort of like, capabilities as far as you know, staff skills, roles, that kind of stuff? And then what does it kind of look like now? Like, who is it that’s actually doing all of this stuff sounds as though you must have a decent team if you move and move very quickly. So you must have, you must have quite a few different skill sets and stuff like that.
Ayesha Saleem 38:36
Yeah, our team was absolutely fantastic. Um, so when I started, we were inside of Content Strategy, which is a very interesting place to be. And we had essentially started with content writers who specialised in writing content for the for support channels, that were interested in getting started with chat bots, and seeing really what they could do there. And that’s kind of how our experimentation phase started. Our team right now is very, is incredibly diverse. And skill sets. I think, one of we have a mix of technical people and a mix of people that have linguistic backgrounds, psychology, background, customer service background backgrounds, which is really cool. And then like writing content, some people that are really interested in like poetry, which I think is absolutely fantastic for like bot writing, and Persona writing, and all of that stuff. And then a lot of people that are incredibly, incredibly data literate. So like data scientists, data analysts, which is all such good skill sets to have on a team and I think that is one of my favourite things about conversational AI is that it requires so many different skill sets. It’s not just like, it has a product component and has a UX design component, a lot of front end component and it’s really cool because I really feel like it At one place in tech or like, ever, like anyone can kind of like contribute. And there’s like, there’s things that like any skill set can really like bring to the table.
Kane 40:09
It is in conversation designers come from all kinds of places don’t, you know, I know quite a few conversation designers who’ve come from like a poetry background or playwright background or something like that right in background, but then you’ve got a lot of user experience designers and stuff like that, that are kind of getting interested in it. I actually think that it’s not from the dialogue crafting side of things, but from the architecture side of things, that business analysts actually make good conversation designers, because they used to have in, you know, they used to talk to people anyway, because that’s kind of what I spend a lot of time doing is talking to different parts of the business, that they have a good understanding of different parts of the business, and also really good at understanding how to take something from one place, and one gets through various complications to get it to an end point, you know. But yeah, lots of different, lots of different skill sets is interesting. You touched on something there around, you know, personality design and things like that. What was your What was your approach to that was? Does the Instacart bots have a personality? Is it the same thing for both kind of bots? Does it differ for the shopper and the customer? What was your approach to, to the whole personality design side of things?
Ayesha Saleem 41:22
Yeah, that’s a great question. I love personality design. That’s one of my favourite parts of the job. In terms of Instacart, we’re actually currently drafting a persona. And we actually just launched a new brand identity like a couple of weeks ago. And so we were kind of waiting on that in order to really help like, define our persona, because I think it’s incredibly important to be consistent with your brand identity, like your bot persona, and your brand identity should be incredibly consistent, or that just leads to confusion. And so really waiting on that was really important. I think it’s interesting, because we have Instacart, various different kinds of users. But sometimes the users do overlap. So it’s not uncommon for a shopper to also be a customer and a customer to try, you know, to be a shopper. And just like Uber has two completely separate apps. So they have like an app for drivers and an app for customers that use that use Uber, we also have two different apps that were very distinct. And that’s something that we’re very intentionally thinking about when it comes to our personas is, do we really want to personas? Or do we want like a single persona for those people that you know, switch between the two user groups, and that’s something that we we haven’t answered yet. But I will say with my with all the personas that are crafted in the past, my like biggest I know, you didn’t ask for my advice, we’re gonna get into my, like, my absolute biggest advice is to like, base personas off like a real person, or not a real person, but like a person or two people. And that means like, take personality tests for your boss. That’s one of the things that I loved doing when I was at rocket, I actually base our Rocket Mortgage persona off of Sophie Turner, who plays Sansa Stark and Game of Thrones not sound so stark, very intentionally, Sophie Turner, partially because it required me to say that me watching Sophie Turner interviews on YouTube was part of my job. Also, mostly because her personality and her persona that she really reflected of being, you know, incredibly confident and incredibly, like welcoming and warm, was really aligned with what I think that rocket needed at the time, especially with the conversational AI experiences and having that open, friendly persona. And so I think it’s very similar that we’re going to be doing very similar things to Instacart. But being very intentional to really align with our new brand identity.
Kane 43:46
Interesting. What’s your kind of approach to doing that? If I mean, when you design a conversation, you can’t help but create a persona anyway, even if it’s subconscious kind of every single line of dialogue you write, especially, is also if it’s spoken, you’ve got to actually consciously select a voice to read that content back. And even if you don’t, you’re still present in kind of language back to a user. And people can make all kinds of assumptions about that word. Do you use correct grammar? Do you use slang? Do you you know, use emojis and stuff like that? So what are your kind of, I suppose two questions. And maybe it’s the same question, but kind of like any advice on the kind of things to consider when creating a persona personality or persona? And secondly, how do you make sure that dialogue that is written conforms to that persona? So let me just start with the first one in terms of any tips that you have around what are the kinds of things to consider what should be included in a persona good persona design?
Ayesha Saleem 44:58
That is a great question. I Think being very cognizant of who your user group is, is the first thing I think it’s impossible to design a persona without knowing who you’re designing that persona for. So just knowing who you’re talking to who your audiences, and it really does differ at rocket, our audience were most like people that were looking to buy a home or refinance a house, which were people in their late 40s, early 50s, for refinance that even went up to 60s and 70s. At Instacart, the majority of people that are customers are actually women, as millennial women. And so being very cognizant of knowing like, who your audiences I think is really, really important in the first big step of Persona design. I think a voice and tone guide for persona design is critical. And that really kind of answers that second question is how do you make sure that your content and the stuff that you’re actually presenting as in line with your persona, having guidelines and that’s not to say like, you know, there’s no creativity allowed in the in their conversational design process, but really having like guardrails that help you kind of foster that creativity while being consistent with what you want to say. And some of those guidelines like one of the biggest guidelines for me, that’s a very simple thing, but like using acronyms when you’re when you’re writing is one is acronyms. It’s not acronyms, it’s contractions, I’m sorry, using contractions and not using acronyms. I saw your face go like this. Yes, using using contractions, I see so many people, especially with like when it’s like a voice assistant, and you hear like, do not instead of don’t, it’s just sounds robotic. And so that was one of the tiny tweaks that I made when I was at rocket that really helped us. I would say, other aspects of like, like, what makes a good persona is having like, guiding principles. So there’s principles of like conversation design, like the cooperator principle, the maxim of relevance of quantity, all of that stuff. Those principles are really important. And they’re kind of almost like a checklist that you can go back and like, kind of check your content against. So one of the things I see with a lot of conversation design designers is that they over explain. And sometimes it comes off as like, they’re almost like, they’re not really putting the user in the centre of the conversation. And they’re kind of like, pushing way too much information onto the user. Like it’s not relevant. And sometimes we over explain, it makes the user feel down, it makes them think that you will, you know, like, something as simple as to sign in, please go to our sign in button, click the button and type your username and your password. Like, if someone said that to me, I would immediately be like, Do you think I’m stupid? Like, you don’t need to say all of that. And from a cognitive load perspective, right? It’s so hard to remember all of that. So having those principles in your writing, like, the principal, or the maximum, like relevance is like, we will only present things that are relevant to you at that time. And then actually checking your your content through that and saying, Okay, are we are we meeting these certain principles? And let’s move forward.
Kane 48:12
It’s interesting that one because I think people can get so caught up in like dwelling over a particular turn in a conversation to the point where you actually completely forget that this one turn in a conversation is going to be over in a split second. It’s like the interaction so fast, that like, and I’m not saying it doesn’t need to be deliberate? Because I think it does, if you’re going to have the conversation, move to the next turn. But there’s so much temptation to dwell on a turn, which then encourages that which is over elaboration, including something that could easily be an answer to a follow up question, if it’s relevant if the user decides to ask it, or like, you know, just trying to make sure that every possible thing that’s needed is included in the response. So half of what you were seeing there is actually just typical, kind of like content design best practice in an auditor, Grice’s Maxim’s, which is most of those principles, but yeah, typical content kind of design is all about that relevant, you know, the inverted pyramid, put the conclusion at the top, you know, leave the details to letter, all that kind of stuff is still just as relevant here, isn’t it?
Ayesha Saleem 49:17
Ya know, for sure. And I think the added complexity is, especially with, with chatbots, it’s, it’s the less the less amount of real estate, right? Because, you know, you have to be even more intentional with your content, because just the space you’re working with, is usually last like it’s not an entire, you know, like UI and with voice assistants. It’s the tone. And I think that’s something that people don’t realise is like, they write out their content and they don’t necessarily listen to their content, but having the pauses in the right area. I remember when I was at rocket, there was a voice, I guess like a voice IVR like, intelligent IVR is intelligent but a more intelligent IVR that was started off with Are you looking to buy a home or were you Finance. And it sounded like a yes or no question. Yeah. And so I would respond with yes or no. And in fact, it was the the prompt was waiting for one of those to like, buy a home or refinance. And so be like listening to your content, I think is also very important.
Kane 50:15
Yeah, 100%. And having it in context as well, like, a lot of the time, even when you listen back to it, if you’re not in the context of the conversation, if you haven’t gone from a few turns back and got to that point, so that you’re experiencing it in real time. I think you can also overlook things as well, you know, you’ve got to really kinda like, which is why I think testing of these things is so difficult, you know, because you need to really kinda like to properly design something, well, you can’t just just test that one response. And you kind of got to go through the whole process, which is different from the kind of unit tested and some of the kind of like technical testing, even just from experience point of view, you’ve got to go through every twist and turn to make sure that every part of the conversation is contextually relevant, you know?
Ayesha Saleem 51:00
Absolutely. Yeah, no, testing is super, super interesting with these things. And I think there’s always bias in testing as well, especially for the people that are designing because they’re so used to talking to the the experience in a certain way, that they don’t realise that there’s various different ways that people communicate. And so I think with testing, we always leave things open to like you at testing, where we have Iraq, it was like business partners, and anyone that was able and willing to help us come in and like play around with the thought, and just seeing the different ways that people interact with it. And depending on you know, like, their age demographic, where they grew up in the United States or outside of the United States, and kind of their experiences previously with bots, and that historical, you know, like, experience that they’re bringing in is really interesting. And I think like, as a perfectionist, one of the things that I had to like really come to terms with is that you can test as much as possible, but someone’s gonna say something that trips your bot up. So really like monitoring and evolving bots. And I think that’s why it’s so important to, you know, consistently like, look like analyse your data and add to the entities and add to the intents and really pick a tool that allows you to kind of make those continuous improvements is so important.
Kane 52:13
Interesting. It kind of, it’s straying into into kind of like all this process we’ve been talking about, which is, you know, crafting a personality in a persona that then gives consistency to the experience and aligns the team that’s creating the thing, then on to testing of the end to end experience and stuff like that. There’s obviously a process at play. And it sounds as though your process is very similar to what you would want to say a traditional compensation design experience, because compensation design is still quite nascent. But it sounds as though it’s all kind of familiar territory, so to speak. But that process is fine. And well, when you’ve got either a small team that’s all on the same page, or, you know, a process where you’ve been running it for a while. Whereas when you’re kind of growing, you introduce a new members into the team. Or you’re introducing new channels, which you have now two different chatbots. So you’ve got two different types of things to design for two different types of use cases. How do you approach that whole scale challenge where you know, a group of people might be working on a bunch of use cases over here, an entirely different group might be working on a bunch of use cases over here, you’ve kind of got to align the personality design, that kind of like approach to design, the way that you document and represent design artefacts, the way that you conduct testing all kind of needs to be consistent if you’re going to produce that level of consistency on the front end. So wondering what you’re kind of conscious of time. And oh, we’ve got a short while left, but any kind of advice you have on teams, as they start to develop in their maturity, and start to bring in new team members and start to scale their practice? What are some of the things that they should absolutely, fundamentally do or not do?
Ayesha Saleem 54:05
Yeah, that is a great question. Oh my God, I would say the one thing that I highly recommend are design crafts, and paired design. Both are things that we use here at Instacart. And I think they’re so so important. So for pair design, very similar, I think that started with paired programming. So that’s a concept that’s existed for a while. But having two designers that might have completely different experiences, work on a thing together. And I think I have heard some pushback from people around like, oh, it takes twice as long or like, from an optimization standpoint, you know, we can put one designer on one thing or one designer or another and get twice the amount of work done. I think like, pair design is so important because things still fat, like things go faster. And also some of those assumptions and questions are answered in the beginning. Instead of like further down the design process where it’s like, oh, we forgot about this thing. let’s iterate here. So I would say that’s a really Good away, when you steal, that’s a really good way to make sure you’re consistent. And then the second thing that we utilise our design, Chris. And I actually, I believe that I got this idea from figma. That is a traditional don’t use, they don’t use conversation design, they use traditional product design. And they do weekly design crits, where they have, you know, designers that are working on things that might not be ready to, like, be pushed to prod or be shown or even like product managers or anything like that, kind of like walk through their process and like, ask questions and really open it up to like us, like to other people to like, you know, kind of like break it or tear it apart or even like, in a nicer way, because I questioned some of the assumptions that they’re making. And I think having that feedback, like incorporating multiple feedback loops, really helps things like, stay consistent, and just be like, really great in terms of quality.
Kane 55:53
Definitely, especially with conversation design, because you’re only ever designing one half of the conversation as a conversation. Like, the more time you dwell on designing that one half, the more rework you’re going to do later on when you eventually have a conversation or partner to go through with. Which Yeah, I think that the earlier the feedback, the better, I think that what is what it takes to have that kind of approach is a real kind of a real kind of solid culture. You know, I mean, I’ve run teams in the past where I’ve tried pair programming, and on the programming side, was always quite difficult to do, because the team before when I took over, we’re very much used to just working on their own. And the engineers, I do my thing, he does his thing, we’ve got slightly different processes, but we arrived there at the end. And it almost felt as though Well, he’s gonna be watching what I’m doing. And it’s like, you know, I don’t want to feel the pressure of kind of someone caught recording while they’re looking over my shoulder. So you kind of really got to break through that whole kind of personal anxiety that people have around their work being inspected. And, you know, graphic design, people tend to just in the graphic, if you begin to design a poster, you know, you would design it to the point where it’s polished, and then you would present something you’re happy with. Whereas this whole concept of presenting something that’s an idea, and discussing the idea quickly, is quite alien to some people, I don’t know, if you’ve found that, in your experience,
Ayesha Saleem 57:21
I think you hit the nail on the head when you started it. 100% is about workplace culture. And I think having a culture at Instacart we very much have a culture of collaboration over competition. And that’s kind of shown and like everything we do in our performance reviews, in the way that we promote people and all of that stuff. And so I think having that culture is like, almost like a requirement, like a prerequisite to being able to do something like pair design. I have found like, I as the first conversation designer, and the only conversation designer at rocket for a very, very long time, I was kind of in my own silo building out conversations, design and conversation. And it’s very difficult, or at least it was very difficult for me because I kind of had to be like, a little bit cocky about it, and be like, Oh, I know, like what the user wants, like, you know, like, I’m the subject matter expert here. And I really didn’t have anyone to like bounce ideas off or even like anyone that would question like some of the assumptions I made. And it was very slow in that sense, because I would watch them wait for you know, us to get enough user feedback, and then be like, Oh, shit, like, I completely forgot about that. And, you know, evolve it that way. Or I when I started building out that team or rocket and we started having, you know, like five or six designers, and I was able to, you know, get them in earlier into the process, we were really able to, like, kind of get those kinks out very early. And I found that in the end, it really led to like less like, we monitor the process, but there was less like evolution, and there was less like, oh, we completely forgot this, let’s take it back to the drawing board. But again, I do think it depends on kinda like people’s mentality, and I 100% Respect that some people, you know, prefer to work alone, and do better alone. And I think what’s really nice about those design crits is like they’re optional. So if you feel like your work is ready to show people you can and if you feel like you need some more time by yourself, that’s okay, too. And it also kind of shows people there are, I think there are a subset of designers that feel like they cannot show work till it’s like really polished. And being on a design crit and seeing some of like, quite frankly, the trash that I bring to design to it. And I’m like, What do you guys think about this? I think kind of empowers other people feel like they can do the same?
Kane 59:31
Yeah, absolutely. I think the main, a really important thing is the fact that you can be a really good designer, and not have it figured out. You know, I think part of being a good designer is knowing when you can’t figure something out and then doing what it takes to try and figure it out. And the best thing to do most of the time is two brains are always infinitely better than one brain, or nine times out of 10 you know, two brains working on something infinitely better. And so it’s like yeah, but it’s interest in how you know the has suppose the traditional design culture is less like that. And even I suppose, you know, this isn’t from experience and this is just a complete assumption. So it could be completely wrong. But I would imagine that copywriters might also have that kind of mentality because copywriters are used to crafting words, you know, every single word needs to find its place in every sentence, which it does. And so, and you can deliberate over words for quite a long time to make the perfect sentence. And so I can imagine that the culture of copywriters coming into a role like conversation design, which is a design practice, which is all about prototyping and testing and iterating. And all that kind of stuff might be a bit of a transition.
Ayesha Saleem 1:00:43
Yeah, no, that’s definitely a really interesting, like, point. I think I’m not a copywriter. I don’t come from a copywriting background at all. But copywriters that I have seen that transition, their conversation design, obviously, I’ve only seen a few. So it’s not like inconsistent, everyone have always felt like there’s something missing in the copywriting world. And I think sometimes it’s that imposter syndrome, where they’re like, I’m in charge of crafting every word, and how do I know that this word is superior to that word? And sometimes, you know, it’s just a gut feeling, or it’s just like, you just have to go with one. And so I think when we introduce them to things like, you know, rapid prototyping, a B testing all of those things, it’s very data driven and science baked. Science backed. They’re almost like, Oh, that makes sense. And I think it’s, I haven’t seen it in a negative light just yet. And I hope it
Kane 1:01:33
Good good. Well, this is wicked. This is really, really good stuff. Maybe it’s the final question Where is briefly gone over a little bit over? If we can wrap up with a quick fire? Final question, which is, what is maybe even the the kind of mistakes or blunders that you see, either that you might have been a part of when you first started or that you notice happening with new designers? And how can they fix it? What are some of the biggest mistakes people make? And what should they do instead?
Ayesha Saleem 1:02:03
Oh, great question. I think not AB testing, I think assuming that something is gonna bring value to your users, without really testing that assumption. And I’m all about AV testing, multivariate testing, which is essentially having a control. So for example, if you have a bot, having the bot in production, or having a version of the bot, was what you think could be improved. And actually testing that against, like success metrics. And actually, studying success metrics before you launch is another big one. I think a lot of designers don’t like, they don’t know what success looks like, because they’re not used to thinking in that business way. So they’re like, Oh, this is better. And it’s like, okay, well, how is it better? How are you gonna, like, how are you gonna prove it? Or is are there ways of measuring? And are you actually going to be able to prove that, so that a b testing, and then the success metrics, I think are two things that are not necessarily a mistake? The thing is that people will definitely should be cognizant of
Kane 1:03:00
Nice one. That’s very good. Very good. Cool. Well, Ayesha, this has been an absolute pleasure. Thank you so much for joining us. It’s been an absolute pleasure. It’s been so interesting, as always, and I’ve loved loved listening to the journey, love hearing how it’s going, and hopefully, you know, we can do it again. In a few months or so when, when things have progressed still, when you’re running a team of 50 conversation designers, all that kind of stuff.
Ayesha Saleem 1:03:28
Oh my god. Yes. This was so much fun. Thank you so much for having me.
Kane 1:03:31
No worries. Thank you very much. Thank you all for joining us. You can join us tomorrow, where we’re going to be talking to Phil Jordan of Homeserve. And we’re gonna be talking about how Homserve have implemented dialogue flow in their call centre to automate some of their customer support as well. So we’ll probably be building on some of this content perhaps. And yeah, it’s gonna be interesting. So please do tune in. Thank you, Ayesha again, it’s been an absolute pleasure. And thank you all for tuning in.
