As indicated by this post, 84% consumers want to be treated as a unique individual not just a number, while 95% of companies saw 3X ROI from their personalization efforts. Now with the rise of conversational commerce, chatbots become a natural and private channel for brands to engage with users and offer personalized services in one-on-one conversations. In such a conversation, a chatbot can naturally elicit users' needs and wants and then provide personalized help or guidance based on the gathered information.

However, most chatbot personalization has been done based on users' explicit information, such as user past activities (e.g., purchasing history) and user explicit preferences (e.g., "I want a car under $5000"). While such information can help personalize a chatbot's interaction with a user, it is often inadequate in many real-world applications.

Chatbot Personalization Examples

Consider the following examples that cover a wide variety of chatbot applications, including e-commerce and healthcare. To serve a user as a unique individual, these examples show that a chatbot must know a user much more deeply beyond his/her past activities or explicit preferences.

Personalizing Product Recommendation

Sam is a new customer to a restaurant and has never ordered anything from the restaurant before. Although Sam explicitly expressed that he prefers fish to red meat, there is still a wide range of selections. What kind of dishes should the restaurant's chatbot recommend to a new customer like Sam? If the chatbot knows that Sam is family-oriented and enjoys down-to-earth food, it could then suggest the homestyle, fish banquet to Sam and his family.

Luna just got her driver's license and wants to buy her first car. Since Luna has never owned a car before, she could provide only a price constraint, e.g., under $15,000. With a number of cars that meet the price constraint, what kind of cars a chatbot on a website like carmax should recommend to Luna ? If the chatbot knows that Luna has keen artistic interests and loves stylish products, it would recommend a stylish car like Mini Cooper or VW Beetle.

Personalizing Wellness/Healthcare Nudging

Emma uses a wellness mobile app for anger management. Emma is supposed to interact with the app, document her emotional states, and practice anger management on a daily basis. How should the app's chatbot encourage Emma to stay on course when Emma does not do what she is supposed to do? If the chatbot knows that Emma is ambitious and values self-empowerment, it can encourage Emma to stick to the program for the benefits that she cares about.

Amy has diabetes but loves sweets. How should a chatbot that serves as her personal wellness assistant can help persuade Amy to reduce her sweets intake? If the chatbot knows that Amy is intellectually curious and like challenges and winning, the chatbot may suggest that Amy solves a puzzle whenever she craves sweets.

Personalized Career Advice

Phil is a senior in college. To prepare for his upcoming job interviews, Phil practices interviews with an interview chatbot. What kind of advices the chatbot should give Phil and help him succeed in his job interviews? Knowing that Phil is thoughtful and cautious but often unwilling to state his opinions, the chatbot can encourage Phil to be bolder and more decisive when expressing his thoughts during an interview.

Challenges in Learning Personality

All the examples above show that a chatbot would be able to offer personalized suggestions and advices if it knows the personality of its users. Broadly speaking, personality includes a set of characteristics that define a person as a unique individual, also known as individual differences. One's personality distinguishes a person from another from three broad aspects:

1. What a person likes to do, including the person's passions and interests.
2. What a person is good at, such as the person's talents and skills.
3. How a person deals with life's challenges, such as handling emotional and social challenges.

Now comes the question: how can a chatbot learn each user's personality? Traditionally, psychologists learn individual differences by asking people to take item-based surveys, such as a personality test. In such a survey or test, people score themselves on a set of items. A chatbot can certainly use the same approach to learn its users' personality by asking each user to take an item-based test.

However, such an approach has several drawbacks. First, it may be awkward to ask a user to take a test (e.g., "Welcome to Walmart, would you like to take a personality test?"). Second, users may not be always truthful in such a test due to social desirability biases especially in high-stakes situations (see testers cheat on the item-based self assessment). Third, the items in a test are usually come up by Psychologists in a lab, and may be out of context for some users. For example, if a person has never watched ballet, the test item asking him/her how much s/he likes ballet would be out of context. Fourth, it is difficult to assess the results objectively based on self scoring. For example, on the item "I love ballet", one person's self-score of 5 might mean a different degree of love than that of another person. Additionally, people's personality does change over time, it would be burdensome and impractical to ask users to take tests repeatedly and track their changes.

To overcome these challenges, computational approaches have been proposed. One popular approach is to ask a group of users to take an item-based survey and then collect the users' behavior data (e.g., communication text). Regression models are then used to examine which features of user behavior (e.g., words or word categories) could be used to predict survey scores. Such approaches have been used to predict Big 5 personality scores, values, and needs. However, this approach is expensive and still relies on item-based survey results to come up with computational models.

Reading Between the Lines

In the search of more efficient and objective approaches to measuring individual differences, Juji has invented a new model that combines big data analytics with Item Response Theory (also known as latent trait theory) to automatically infer individual differences from user behavior. Currently, Juji uses this approach to infer Big 5 personality scores, including all the 35 dimensions, from text. Specifically, with this model a chatbot can interact with users in a natural way without asking them to take a Big 5 personality test. It analyzes a user's conversation text on the fly to infer the user's individual differences. The chatbot can then use the insights to personalize each conversation as shown by the examples above.

If a user interacts with such a chatbot continuously (e.g., asking the chatbot questions), the chatbot can automatically learn the changes in the user's individual differences in a natural way and use such information to better help the user (e.g., offering suitable career development advice).

Juji currently focuses on inferring the Big 5 personality traits as its base model, because numerous research studies show that these traits characterize the fundamental individual differences in people and they can influence people's real-world behavior and predict important life outcomes. On top of these basic traits, Juji infers context-specific user characteristics, e.g., one's reader DNA, which can then be used to help users find products or services that match with their innate needs. For example, a chatbot would recommend self-enhancement or business books to cerebral readers, while recommend art-related books to aesthetic readers.

Reliability: How Stable Are the Inference Results

To assess the quality of the trait inference results, it is important to measure the reliability of the method. Reliability is to assess how stable the inferred scores are under consistent conditions. For example, if given a set of 100 randomly sampled words produced by a person, how do the inferred trait scores differ for each set of words? The image below shows the computed Cronbach's alpha for the inferred scores of the six facets under the Extroversion dimension in the Big 5 personality model.

The above graph shows that the model produced an acceptable reliability (> 0.8) for all the facets except for Cheerfulness, if the number of words used in the inference was over 1000. This means that the inference of those facets is reliable if it uses more than 1000 words.

Validity: How Useful Are the Inference Results

In addition to reliability, another important measure is validity, which assesses whether the inferred trait scores correspond to real-world user behavior (e.g., predicting a user's behavior or preference). Since measuring validity is often context sensitive, experiments or studies need to be carried out in a particular context. For example, Xiao et al. show that personality trait scores inferred by a Juji chatbot can predict team performance.

Chatbot Personalization by Personality

To help chatbot designers easily personalize each conversation experience, Juji provides a set of functions and "message cards", which allow a designer to personalize chatbot messages for each inferred user persona.

How to Personalize Chatbot Messages

One easy way on Juji to personalize chatbot messages by individual differences is to customize a pre-made "message card". Currently Juji provides two pre-made message cards: one allowing a chatbot designer to customize chatbot messages to match inferred shopper personas, while the other allowing a chatbot designer to customize chatbot messages to match a set of base personas.

As shown in the screenshot below, a chatbot designer can personalize an e-commerce chatbot's messages by different shopper personas. In a live conversation, a chatbot will send a custom message to a user based on the user's shopper persona inferred from the conversation.

The screenshot below lists all shopper personas that Juji automatically infers. In other words, a chatbot can send different messages or recommend different products to users with different shopper personas. For example, a chatbot may recommend high-quality, brand-name products to users with an inferred "Achiever" persona, while suggest innovative, new products to users with an inferred "Explorer" persona. To ensure accuracy, currently a chatbot would not send out a custom message if it does not have enough information (e.g., few words) to infer a user's shopper persona.

Likewise, a designer can personalize a chatbot's messages to different users based on their base personas, such as giving career advices. Again, such a chatbot just needs to carry on a natural conversation with a user, such as asking users how they are doing, and use such a conversation to infer the user's individual differences and then send custom messages based on the inference results. As shown below, Juji also automatically infers a set of base personas.

A chatbot designer can easily customize a chatbot's messages to users based on their inferred base personas. For example, a designer can customize a chatbot's message to one who has a "thinker" persona (thoughtful and cautious) versus to a user who has a "helper" persona (warm-hearted and caring).

In addition to using pre-made message cards as shown above, one can use system functions as preconditions to personalize chatbot messages by individual differences. For example, if you wish that a chatbot sends a message only to someone who is caring and warm-hearted (is-helper), you can then insert this function as a precondition of a remark so the message will be sent to only users who possess such characteristics. See below the screenshot.

When to Personalize Chatbot Messages

Now I have explained how a chatbot designer can easily customize a Juji chatbot's messages to address different user personas, the next question is when a chatbot designer should do so to get the best value?

Help Cold Start

One effective use of chatbot personalization by personality is to help cold start, such as onboarding a new user or introducing a new product. In such a situation, a chatbot often does not have other sources of information to personalize its messages. For example, a chatbot does not have the past activities of a new user or it does not have user purchasing history or reviews of a new product.

For a new user, a Juji chatbot can chat with a new user by asking the typical on-boarding questions. In addition to gathering the basic on-boarding information, this chatbot will also automatically infer the user's individual differences from the conversation and now can make recommendations without the user past activities.

For a new product, a Juji chatbot can describe the value of the product that matches the personality of users even though the users have not experienced the product yet. For example, users who desire safety would love safe, high-quality products, while users who are highly curious and open to new experiences would enjoy innovative products.

Help Longitudinal Engagement

Knowing users' individual differences also helps a chatbot engage with users longitudinally. For example, a chatbot serving as a personal wellness assistant will need to engage with a user on a daily basis. Knowing the likes and dislikes of the user, the chatbot could keep the user stay engaged as well as better persuade the user to fulfill the required tasks, such as adhere to a particular care treatment. Not only will such a chatbot improve user engagement, but it will also improve care outcomes.

Make Responsible Chatbots

With great power comes great responsibility. Now that a chatbot can deeply understand its users, how can we ensure the responsibility of such a chatbot? For example, if a chatbot detects a shopper's impulsive buying behavior, how can we make sure that the chatbot does not indulge the shopper to buy more? Similarly, if a chatbot detects a gamer's addiction to violent games, how can we make sure that the chatbot does not encourage the gamer to become more addicted? In the near future, I will blog more about how to make responsible chatbots that can truly help people but not harm them.

Further Readings

• Here is a tutorial video showing how to personalize chatbot messages based on inferred individual differences with no coding.
• Here is a more in-depth talk given at the RecSys 2020 on this topic.
• If you haven't done so, sign up on Juji to create your own chatbot and personalize its messages based on Juji inferred individual differences.
• If you are interested in reading similar posts like this one, follow our Linkedin page where we will publicize new posts.