Basic Concepts¶

This page explains the fundamental concepts of A/B testing and how they're implemented in BoastPress AB Testing.

What is A/B Testing?¶

A/B testing (also called split testing) is a method of comparing two or more versions of a webpage or element to determine which one performs better in terms of a specified goal, such as clicks, form completions, or purchases.

In its simplest form, an A/B test compares:

• Version A: The original version (control)
• Version B: An alternative version with one or more changes

By showing these different versions to similar visitors at the same time and measuring their performance, you can make data-driven decisions about which changes to implement permanently.

Key Components of BoastPress AB Testing¶

Tests¶

A test is the overall experiment you're running. It contains: - Multiple versions (variations) of content - Settings that determine how users are assigned to variations - Conversion tracking configuration - Performance metrics

Versions¶

Versions are the different content variations within a test:

• Control: The original version (Version A)
• Variants: Alternative versions (Version B, C, etc.)
• Default Version: The version shown when a test is paused or when a user doesn't qualify for the test

Test Modes¶

BoastPress AB Testing offers two test modes:

Impression-Based Testing¶

• Users may see different variations on each page load
• Useful for testing elements that don't require consistency across multiple pages
• Provides faster data collection

Session-Based Testing¶

• Users see the same variation throughout their session
• Session duration is configurable (default: 24 hours)
• Useful for testing elements that should remain consistent throughout a user's visit
• Provides better user experience for certain types of tests

Version Assignment Methods¶

There are two ways to assign users to test variations:

Random Assignment¶

• Users are randomly assigned to variations
• Each variation has an equal chance of being shown
• Simple to set up and suitable for most tests

Bucket-Based Targeting¶

• Users are assigned to variations based on specific criteria
• Allows for targeted testing based on user characteristics
• Useful for personalization and segmented testing

Buckets¶

Buckets are user segments defined by specific criteria: - Query parameters in the URL - Cookies in their browser - HTTP headers in their request - IP address ranges

Impressions¶

An impression occurs each time a test variation is shown to a user. Impressions are tracked for each version to measure how many times it has been displayed.

Conversions¶

A conversion is a desired action taken by a user after seeing a test variation. Conversions are tracked when a user visits the designated conversion page after seeing a test variation.

Conversion Rate¶

The conversion rate is the percentage of impressions that resulted in conversions:

Conversion Rate = (Conversions / Impressions) × 100%

This is the primary metric used to compare the performance of different variations.

Statistical Significance¶

Statistical significance indicates whether the differences in conversion rates are likely due to actual performance differences rather than random chance.

A result is considered statistically significant when the confidence level (typically 95%) is met or exceeded.

Content Delivery Methods¶

BoastPress AB Testing offers two methods for delivering test content:

Standard Mode¶

• Test content is generated when the page is loaded
• Faster initial page load
• Not compatible with caching plugins or CDNs

AJAX Mode¶

• Test content is loaded after the page has rendered
• Compatible with caching plugins and CDNs
• Slight delay in content appearance

Implementation Methods¶

There are several ways to implement tests on your WordPress site:

Shortcode¶

The primary implementation method using the [ab_test] shortcode:

[ab_test test="123"]

PHP Function¶

For direct integration in theme files:

<?php echo do_shortcode('[ab_test test="123"]'); ?>

Testing Workflow¶

The typical workflow for A/B testing with BoastPress AB Testing is:

1. Identify: Determine what element to test and form a hypothesis
2. Create: Set up the test with control and variant versions
3. Implement: Add the test to your site using shortcodes or other methods
4. Collect: Gather impression and conversion data
5. Analyze: Evaluate the results and determine statistical significance
6. Implement: Apply the winning variation permanently
7. Iterate: Start the process again with new tests

In the next section, we'll walk through creating your first test step by step.