A/B testing is a subject that has just recently become popular among business specialists. Many companies still ignore its importance, which is a shame because a/b testing can help businesses turn a profit. However, to understand this phenomenon, it is important to study it through properly.
There is a lot of good literature regarding a/b testing, a lot of which focus on testing as a process to ensure that a product meets specifications. Other books focus primarily on determining whether or not a statistical test was performed correctly, and if not, how it can be done differently. Most quality books, however, will focus on one or the other. A good-quality book will go into detail about how to use statistical methods in the design of experiments. It may explain why a certain set of specifications was chosen for a specific experiment, what statistical significance level should be used, and how to interpret results. All of this will help a reader better understand the subject and apply it to decisions within the laboratory.
How to Select a Good A/B Testing Book
However, a good a/b testing book will not stop at a comprehensive explanation of a/b testing. It should also go into detail about the different applications a product might have in a market, explaining when a/b testing is most beneficial or how it can be used to increase revenues. A good a/b testing book takes a practical approach towards the subject and focuses on solving problems regarding products and services. It will show an a/b testing best practices and framework, calculator, and an a/b test example. This book should give a step-by-step explanation of how to do a/b testing during the entire process from identifying the different elements that need to be tested to building up an a/b test case, collecting data, and analyzing it.
Making a Choice Based on Facts
If you are still hesitating to make a choice, here are criteria for selecting the best a/b testing books:
- The target audience: If you want to understand a process for your own needs, simply knowing the steps involved may suffice if you don’t care about any of the statistical interpretations of results. If you want to know all possible options available, then an in-depth explanation of a/b testing with a lot of references to research might be a better choice. If the a/b testing book is targeted at a specific audience, be it, product managers or marketers, then this could help select a book that has a more direct approach towards the reader.
- The language used: If you are not familiar with any computer languages and simply want a hands-on guide for your projects, then a step-by-step guide would suffice. However, if you plan on implementing complex algorithms or have an interest in statistical concepts, then source code examples might be needed.
- Availability of supplements: Some books come with interactive websites containing datasets for analysis as well as additional explanations to further help readers understand the material. Supplemental material such as a chapter on a specific algorithm or a dataset for readers to try out a/b testing can provide a deeper understanding of a particular concept.
- Readability: A good a/b testing book takes a practical approach and explains concepts using easy-to-understand language. It gives the reader a thorough understanding of a/b testing in simple terms without delving into unnecessary statistics and complex equations. A good a/b testing book is highly readable and encourages the reader to experiment and apply their knowledge rather than simply consume it.
Online A/B Testing with Georgi Zdravkov Georgiev
One of the best books, which may not have been touched upon above, is that of Online A/B Testing with Georgi Z. Georgiev. This book has become a bit of a bible for the software engineers and software developers as well as people involved in A/B testing. It explains why a/b testing is important and what the statistical methods are used for. It also has many very useful appendices. These include a manual of selected works by Koike and a study guide on statistics for the workplace.
The Online A/B Testing with Georgi Z. Georgiev is a book that is rather extensive and covers a lot of ground. However, it is written in a very simple way that makes it easy to read and understand. It explains in great detail how to choose the right a/b testing service for your project, how to set up the statistical analysis and data collection, and how to do the actual testing. It also explains in great detail the statistical significance of the data collected and the lean metrics used for the conversion rate optimization.
What will you find here
The book contains various appendices, which are extremely useful for a software engineer or an A/B analyst. There are eleven parts of the book, which are divided into three parts. Part one is about the basics of statistical concepts and deals with the concepts, which are needed for the understanding of a/b testing. Part two has chapters on Lean, the importance of testing and statistical utilities of a/b testing, a summary of the test preparation, and then includes a chapter on the testing management team.
Parts three and four deal with the actual implementation of a/b testing. These chapters deal with:
- planning the metrics
- the selection of a/b testing service
- and actual antifragility testing.
These chapters also deal with the use of antifragility techniques in a/b testing and what the implications are for a company. Part five explains how to use the data gathered from the tests for statistical analysis. This final chapter explains the implications for a/b testing and what a company needs to do to follow the recommendations in the Lean Analytics book. I found that this book had all the material that I needed to begin implementing a lean analytics system in my organization and had a lot of illustrations to reinforce the concepts.