A Simplified Approach to User Segmentation

With so much to do, one of the major mistakes that companies make when they start A/B testing activities is sticking to a small, basic audience. This does not yield the most meaningful results or sometimes, the results could be misleading.

A/B testing (also known as split testing) is a process of showing two variants of the same web page to different segments of website visitors at the same time and comparing which variant drives more conversions. One of the most critical steps for conducting A/B testing along with research and formulating a hypothesis is to create a variation based on your hypothesis, and A/B test it against the existing version (control).

A variation is another version of your current version with changes that you want to test.

User segmentation is a way of categorizing your website visitors into groups, based on specific criteria such as location, the device used to access the site, the hour of the day, or any other user behaviors. For example, sorting users by region, language, or behavior. The better teams are at segmenting their users, the more personably they can treat each group.

According to a study conducted by Mixpanel, eighty-one percent of consumers today wish companies knew them better, while ninety-four percent of marketers wish they knew their customers better. Why aren’t the two just talking it out? Because communicating with users is a science-powered art, and intuiting what users want requires tools, time, and proper segmentation.

Let us assume (we’ll call this as our hypothesis) that Peter owns a computer e-commerce store in Canada, and wants to show a ‘Friday Macbook Special’ offer page only to people who don’t yet own a Mac computer. He also thinks that people who own an iPhone/iPad might be interested in buying a mac computer.

Let us now explore how I designed a custom user segmentation tool, for selecting a specific segment of visitors, whom the user would like to show a special version of his webpage to build a better product and increase conversions.

The approach

Segregating And-Or

The problem in designing interface for rule-engines is that there is an impedance mismatch between a user and developer/maintainer for the rule engine. The most common cause of confusion when creating rules with and/or is that users generally don't know the precedence rules for the logical operators. This often creates situations where the intended results don't get generated for rules created by a user.

Consider the following rule created by a user composed using and,or logical operators:


rule-1 and rule-2 or rule-3 and rule-4 or rule-5 or rule-6 and rule-7 and rule-8

And user meant the following:


rule-1 and (rule-2 or rule-3) and (rule-4 or rule-5 or rule-6) and rule-7 and rule-8

When getting evaluated, rule engine use precedence rules of operators and understands the above expression as follows


(rule-1 and rule-2) or (rule-3 and rule-4) or rule-5 or (rule-6 and rule-7 and rule-8)

This is clearly unintended, but this causes user frustration and confusion and thereby leads to cognitive load as they need to be extra cautious while creating segmentation rules.

How does my UI module fixes the problem?

My UI module enables users to create segmentations that abide by the precedence rules of logical operators that tries to simulate creating logical groupings that explicitly state the order of evaluation.

It works as follows:

Segment definition is composed of a collection of segment chunks.

Each segment chunk can only contain conditions connected with and logical operator.

Segment chunks are connected amongst themselves with or logical operator

Considering the rule example above, each logical grouping of and conditions are packaged inside a segment chunk, and, or conditions are recorded as collection of segment chunks. This allows user to focus on building a rule set consisting of and conditions in isolation and re-use it in multiple hypothesis.

Containment operators

Containment operator such as isOneOf makes it easier to create rules for a segment having multiple possible values within one category. These operators make it possible to avoid creating multiple segment chunks by transforming implicit or condition to an and condition.

For eg. consider the following rule

All users from Hungary visiting website using iPhones via Safari, Chrome or Firefox

The atomic rule can be created as follows

Location = Hungary and device = iPhone and (browser = Safari or browser = Chrome or browser = firefox)

The same query can be transformed into an all and query using isOneOf operator

Location = Hungary and device = iPhone and browser isOneOf [Safari, Chrome, Firefox]

Containment operators only works for intra category values. For or condition on inter-category values, explicit or conditions need to be created.

Examples

Let's see how segment chunks can work across two different examples that make this point clear.

Example 1: All users from US using MacBooks or Users from UK having iOS device visiting the website on Fridays.

The above segment can be created using a single segment chunk as follows:


Day = Friday and Location isOneOf [US, UK] and OS isoneOf [macOS, iOS]

Example 2: Users from Italy using firefox on iPhone or IE on desktop only on Fridays

The above example can be constructed easily using two segment chunks as follows:


# chunk 1 
(Location = Italy and device = iPhone and browser = firefox ) 
or
# chunk 2 
(Location = Italy and browser = IE)

Now that you have an understanding of how the user segmentation works, let us go back to the original problem statement

Peter owns a computer e-commerce store in Canada, and wants to show a ‘Friday Macbook Special’ offer page only to people who don’t yet own a Mac computer. He also thinks that people who own an iPhone/iPad might be interested in buying a mac computer.

This is how the segmentation for the above hypothesis will look like along with the hypothesis and URL(s)

The creation of the second segment can be done easily by copying the conditions or operands from the previous segments like this

The final segmentation of the above use-case will look like