That Was Never My Intention: Morning Espressos & Intent Clashes

Intent clashes blog
Setting aside the 5.30am speaking slot (thanks a lot to the USA/Australia time difference), it was a real pleasure to speak at the Rasa Summit 2021. After more early morning espressos than is probably good for me, I had the opportunity to talk about ‘What’s next in CDD: Intent Clashes and Selective Confidence.’

Whilst my talk title is unlikely to win any prizes for literature, it hits on some really important themes that could change the very way your brand is perceived.

Watch a recording of the session captured by the team at Rasa DOWNLOAD THE SLIDES: RASA SUMMIT 2021

Imagine for a moment you’d booked a flight through a deal you saw on Flight Centre – given I mentioned espressos earlier, let’s say it’s to Italy. Unfortunately, you can no longer go. You contact Flight Centre via their chatbot and start a conversation with their support service. You want to know whether you can get a refund on the deal.

Things are going swimmingly until you write the word ‘deal’. At that point, the chatbot classifies your intent wrongly, assumes you want to learn about current deals being offered by Flight Centre, and automatically transfers you to the sales conversation pathway. You’re now presented with irrelevant questions like ‘which parts of the world are you interested in travelling to?’ when all you wanted to do was get a refund on a previous deal. That is an intent clash: a misclassification of what you as the user were looking for from the conversational experience.

The problem is, that can happen all the time. Let’s stick with Flight Centre. They have 12 different scenarios for the conversational experience. That’s 12 different conversational paths that chatbot users can travel down. If at any point a user’s intent is wrongly classified, they could find themselves moved onto another path they don’t want to be on. One moment they’re on Support Team Avenue, the next, they’re across town on Sales Boulevard.

Source: Rasa Summit 2021: What’s next in CDD: Intent Clashes and Selective Confidence - Slide 6.
Source: Rasa Summit 2021 - What’s next in CDD:
Intent Clashes and Selective Confidence - Slide 6.

Let’s be clear, intent clashes are a huge problem. If a user is given the wrong information or question based on a misunderstanding of intent, it can cause huge frustration and irreparable damage to that brand.

But – and this is the very reason I was happy to get out of bed to present at 5.30am – we’ve developed a solution for that exact problem through our Pattr platform. In fact, it’s a solution that we provide to Flight Centre, amongst other brands, including Twitter. So, here are three ways that we design conversational experiences to solve intent clashes and create a seamless user experience:

  1. Human-In-The-Loop
    No, this isn’t some kind of circus trick or yoga pose. When we design conversational experiences, we make sure that we have people involved, alongside AI, building, testing and iterating the conversational flow, both during the design phase and the product roll-out phase.
  2. Selective Intent
    By keeping people at the heart of our CDD, we are able to design conversations that select user intent based on their context. What that means is that we can selectively link or unlink intent that we don’t want to classify at specific points. Instead of a conversation where a user’s intent can be classified (or misclassified) at any point and then take them down an irrelevant pathway, selective intent means we select when to classify intents – choosing particular moments where we offer an alternative path.
  3. Confidence Thresholds
    If you’re with me so far, your own confidence must be through the roof, but that’s not the point here. The layer on top of selective intent leverages context to adjust confidence thresholds. If we’re confident of classifying a user’s intent correctly at particular points of the conversational experience, we can raise the confidence threshold to say 90% and automatically take them down a different pathway. If, however, we only have a very sneaking suspicion that we’ve understood intent correctly, we can adjust the confidence threshold to say 30% and offer a user an alternative path, without automatically taking them there. It means we’re adjusting our conversational experience to account for context and misunderstanding.

Put these three elements together, and we’re able to significantly minimize the likelihood of intent clashes for our clients, and the knock-on effect of frustrated users and brand damage. Ultimately, it’s all about creating a fast, accurate and consistent user experience through which brands can engage with their customers at scale.

Now, if you’re a business looking to better engage your customers through conversational AI, that really is worth getting out of bed at 5.30am for. If you’d like to chat to the team about how we can help you, maybe wait until a more business-appropriate hour, and then get in touch with us!

Need to know lingo

Conversational AI: a sub-domain of Artificial Intelligence that deals with speech-based or text-based AI agents that have the capability to simulate and automate conversations and verbal interactions. (IEEE)

CDD (Conversation-driven development): the process of listening to your users and using those insights to improve your AI assistant. (Rasa)

Intent clashes: where a user’s intent (what they are looking for within the conversational experience) is misunderstood or misclassified and an irrelevant response is given. (Rasa)

Selective confidence thresholds: the ability to adjust the level of certainty or confidence within conversational AI when classifying against a user’s intent.

Human-in-the-loop: the principle of maintaining a human presence within the design, build and testing of the conversational flow. (Gartner)

See Pattr in Action

Pattr brings you closer to your customers.

A conversational AI SaaS platform to power, enable, enrich and understand conversations between you and your customers, in real-time and at scale.

LLM vs Micro models Preview Image

Large language models vs micro models – the great debate

Read More
Spam/Bot comments Blog Preview Image

Here’s how Conversational AI can help you tackle spam comments on social media

Read More
8th Anniversary blog preview image

Pattr is 8 years old! Here are some of our favourite projects

Read More