This post is also available in: 日本語
Introduction
There is a technique called AB test to compare the effects of sites and advertisements.
This article discusses the basics of AB testing to help you get the most out of it, and introduces some of the best tools for using AB testing.
What is the AB test?
The AB test is a keyword that is often spoken in the marketing context. It is mainly used to compare and verify the effects of the two measures. More specifically, it is used more often in the context of improving the effectiveness of a site or advertising verification.
Compare Two Groups
The AB test conceptually consists of two groups, a control group and a test group. Control groups, also known as control groups, are essentially “Continue to implement existing
measures” groups. As the name suggests, the test group uses a new test group.
What is required for the success of the AB test
Now that you have reviewed the definition of the AB test, let’s move on to what needs to be done to make the AB test successful.
Be sure to define objectives and KPIs
First, you need to decide the purpose of what you want to get from the AB test. For example, the team agrees on the purpose of the test, whether they want to improve the registration rate of the site or the length of stay. After that, we decide the KPI from the viewpoint of which indicator to look at. If you want to improve the registration rate, the click rate of the registration button, and if you want to improve the staying time, the staying time becomes KPI.
Match Prerequisites
The two groups A and B should match the assumptions as much as possible. For example, if a group of A contacts only 100 users, but a group of B contacts 10,000 users, the assumptions are different and it is difficult to validate correctly.
Whether the judge is irreversible
For example, the AB test in the production environment is not suitable for the irreversible (= Difficult to return) measure such as the full renewal of the site. The AB test is easier to verify than the conceptual upstream content. It is closer to the details such as site layout and wording. If you want to compare ideas to increase the success rate of irreversible judges like the full redesign of the site above, what can you do? In that case, the most effective methods are interviews (qualitative survey) and questionnaires (quantitative survey). You can investigate which concepts are most acceptable to users before they are released.
Ensure a significant difference in
When you do the AB test, you may feel that there is a clear difference. It is necessary to reexamine whether the result is mathematically significant or not. The t-test is a typical way of thinking to verify the significance.
The t-test is a generic term for statistical tests that take advantage of statistics following the t-distribution, assuming the null hypothesis is correct. A parametric test that assumes that the population follows a normal distribution and that the t-distribution does not depend directly on the original mean or standard deviation. It is used to test whether there is a significant difference in the mean between two pairs of specimens.
On the web, there is a free tool that you can do t-test just by entering numbers, so it might be a good idea to search on Google.
There is not always a significant difference.
The AB test is a tool that helps you decide which one is better, based on user feedback, but there are not many clear differences. Ideally, you want to see the effect clearly, but there are many things that you can’t really decide. When the AB test is performed, there is a pressure to decide between A and B, but you can avoid making a hasty decision by setting a rule such as not to decide anything that cannot be decided by force.
Introducing Useful Services for AB Testing
Let’s take a look at some of the tools you can use for AB testing.
Google Optimize For Free
When it comes to free AB testing tools, Google Optimize is the only option. Google Optimizer is a free AB testing tool from Google. In addition to intuitive editing, the integration with Google’s other tools means you can run AB tests without any complicated work.
You can read more about Google Optimization here.
Free AB test without writing code! Google Optimize Improves Your Site
Kaizen Platform
Kaizen Platform is a crowdsourced CRO service focused on CRO. Called Growth Hackers, the Kaizen platform’s network of marketers and designers offers effective creative and layout suggestions. It is a very well-supported service in that it proposes creative work for AB test.
Optimizely
Optimizely is the world’s leading AB testing tool. It’s the most used tool on the planet. Optimizely is a paid tool that allows you to drive a large number of AB testing projects simultaneously. Big companies like IBM and Microsoft also use it. In terms of functionality, it’s a forward-compatible version of the free Google Optimizer.
And finally,
In this article, I’ve organized the idea of AB testing and introduced some useful services for AB testing. Continuous PDCA is required for effective AB test operation.
This concludes this article. UTILLY has a list of services that can help with marketing. Please read the following article if you like.
10 free services you’ll definitely want to use when you start marketing
