How Chatbots Work (and Why They Sometimes Don't)
Domo arigato, Mr. Chatboto, for doing the jobs that no one wants to do. How do you do it? What is your secret?
We talk about chatbots a lot and what they can do for your business — but how do they actually work? You type in a question, an answer pops out. Sometimes it feels like magic.
With so many chatbots popping up on websites nowadays, you can’t help but wonder how this happened so fast. Don’t they cost a lot of time and money to make? And how can some be so eerily accurate and humanlike, and others downright “dumb”?
Let’s take a brief and uncomplicated look at the different types of chatbots on the market and how they work.
What is a chatbot?
First, what are chatbots exactly? Short for chat robot, a chatbot is a computer program that simulates human conversations. It interacts with users through instant messaging, artificially replicating the pattern of human communication.
Chatbots have been around since the 1960s, with ELIZA being the first chatbot developed by MIT professor Joseph Weizenbaum. Since then, chatbots evolved from being conversation buddies to helping businesses and people with everyday tasks and management.
Today, chatbots live in people’s homes, phones and favorite websites. How do they understand us and know what to respond? Why are some chatbots more advanced than others?
How chatbots work
In a nutshell, chatbots analyze a user’s request and give a response based on a specific decision process. The details of how this is done depends on the chatbot type.
There are three main types of chatbots : decision tree, keyword recognition-based and contextual.
These three chatbot types have the same goal: To figure out what the user needs and help to the best of its ability. Let’s look at how each type achieves this.
1
Decision tree-based
The most un-chatty chatbot of them all. Decision tree chatbots are preprogrammed to follow a sequence, which can be very simple or complex.
I created a decision tree hair salon bot named Ola on Landbot.io . It took the development work out of creating a chatbot and instead let me use building blocks to put together a cohesive conversation.
This chatbot works using pre-selected widgets with button options. It allows you to get creative with your chatbot’s text and display options, but your user is expected to choose between these options that you define.
It’s a very straightforward process, which is why many businesses use this solution to build their chatbots. They’re cheaper to build, quicker to deploy and can still be useful, entertaining and educational. Its limitations don't prohibit creativity.
2
Keyword recognition-based
These types of chatbots work similarly to decision tree chatbots but are purely keyword-dependent.
However — unlike decision tree chatbots — keyword-based chatbots can be programmed to have a more conversational approach. Since users are free to type their own questions and responses, conversations don’t have to follow a linear path. The chatbot can shift focus as long as it identifies keywords in your input.
Using Python , for example, you can define dedicated functions to handle responses such as greetings and customer inquiries. You have control over which keywords it understands and how it can respond.
The chatbot then recognizes these specific commands. If you type “Good morning,” the chatbot will identify it as a greeting and give the next appropriate response according to its conversation map.
If you say, “Yo, waddup,” you might get an error response. This isn’t a common greeting, so the chatbot likely isn’t programmed to recognize it.
This type of chatbot suffers in more complicated scenarios where more variables and knowledge are needed. If a user is asking questions that are too similar, the results may be redundant.
A hybrid of keyword-based and decision tree chatbots can help the user find their answer instead of giving up.
3
Contextual
While the previous chatbots followed rules, these chatbots make their own...well, sort of.
Contextual chatbots use artificial intelligence (AI) and its subset machine learning (ML) to have conversations and eventually learn from them. They’re more independent than their chatbot brethren, but require strategic planning and guidance.
There are two types of dialogue systems: goal-oriented (think Siri, Alexa, etc.) and general conversation ( Insomnobot-3000 , Replika ). One tries to solve problems using natural language while the other attempts to converse about a variety of topics.
There are certain parameters that need to be met for a contextual chatbot to converse at an almost human level:
Creating a conversation flow. A good conversation flow is not necessary for a chatbot to work — but it does make the exchange a lot more pleasant. For tips on how to create one, check out our post “6 steps for creating a smooth chatbot conversation flow.”
Adding proper intents. An intent is the user’s objective. If they tell the chatbot “Show me today’s weather” then their intent is to retrieve that day’s temperature and conditions. Intents are given a name, often a verb and noun, like “showWeather.” Predefining intents helps your chatbot properly respond to inputs.
Feeding the chatbot’s knowledge base. Contextual chatbots require sufficient data. Open source data frameworks are available for building general purpose chatbots, but for your chatbot to be an extension of your business, it needs your historical data.
This may include customer data, live chat transcripts, customer support documents, and any other data that you feel is significant.
Using Supervised Learning , you can train your data-beefed contextual chatbot to recognize human intent, and accept or reject intents it qualified after a chat. Over time, your chatbot will be able to make more accurate predictions and interpret natural language better.
Basically, the more data a contextual chatbot can collect and send to a structured database, the more the experience will improve.
Natural Language Processing (NLP) is what helps your AI better understand intent and context. It’s technology that helps computers interpret and act on human language by breaking it down into pieces and observing how they work together.
When a user types “hello,” NLP helps the chatbot understand that you’ve sent a greeting, and AI determines a proper reply. Unlike keyword-based chatbots, NLP provides context and meaning to text-based responses.
The use of NLP involves the following:
- Natural Language Understanding (NLU): The process of converting text into structured data that a machine understands.
- Natural Language Generation (NLG): The process of transforming structured data into natural language.
Once these actions occur, the chatbot can interact with the user on close-to human terms.
The Ultimate Guide to Chatbots in Business
Learn how chatbots work, what they can do, how to build one – and whether they will end up stealing your job.
Read moreWhy chatbots don’t work sometimes
To be frank, some chatbots really are just over-glorified flowcharts. They depend on the user to follow a strict flow that results in rigid interactions and conversational dead ends.
Language and knowledge is also too intricate for many chatbots, so nuance is lost on it. The systems we interact with today are narrow AI, which can often only do what they’ve been told.
Even NLP comes up short sometimes when detecting patterns in natural language. Language is structured, but it can also be chaotic.
Some challenges include:
- Synonyms, homonyms, slang
- Missing punctuation
- Misspellings
- Abbreviations
You have to approach a chatbot as if it’s someone in the beginning stages of learning to speak your native language. Keep it simple and clear.
It also doesn’t help that businesses and users have unrealistic expectations when it comes to chatbots. Often, both parties want a chatbot that can speak and solve problems like a human.
This isn’t possible until we have general AI, which we won’t see in our lifetime . Contextual chatbots are our next best option. Their speech and accuracy improve over time, but at your customers’ expense.
If you let a computer learn chess by playing itself, then it can make thousands of mistakes without harm. In customer service, the only way to learn is to actually interact and make mistakes with customers, which means many frustrated experiences.
Get started with your own chatbot
Chatbots work best when connected to a live chat solution. Userlike has a variety of chatbot/AI options including the HTTP API framework , which lets you connect chatbots like the OMQ bot , IBM Watson , or your own custom solution.
Once your chatbot is connected to the Userlike chat infrastructure, it has access to chat commands and can navigate visitors to specific web pages.
You can see the transcript of every interaction your chatbot has, and your bot can request and store relevant information from your site visitors. If a visitor wants to speak to a human agent, your chatbot can easily forward the chat.
Userlike offers a free 14-day trial that gives you an idea of how our messaging platform works. If it feels like a good fit, let us know and we’ll get you set up with the right plan.
