Creating Your First Test¶

This guide will help you set up your first A/B test with BoastPress AB Testing in just a few minutes.

Overview¶

Setting up an A/B test involves these basic steps:

1. Create a new test
2. Add test variations
3. Implement the test on your site
4. Track conversions
5. Analyze results
6. Monitor results

Let's walk through each step.

Step 1: Create a New Test¶

1. In your WordPress admin, navigate to A/B Tests > Add New
2. Enter a descriptive name for your test (e.g., "Homepage Headline Test")
3. Set the conversion page (the URL where conversions will be tracked)
4. Choose your test mode:
   • Per-Impression: Users may see different variations on each page load
   • Per-Session: Users see the same variation throughout their session
5. Click Create Test
6. Choose your Content Loading option:
   • Server Side The plug-in decided what content to display within your WordPress server and deliver it natively within the HTML
   • AJAX Deliver default content with the page and use a client-side JavaScript event to gather the chosen content. This option is great if you have aggressive caching enabled.
7. Choose your Version Selection Mode option:
   • Random: Randomly select a version for each user
   • Bucketed: Assign users to a specific version based on any number of parameters. This is useful for more advanced targeting.
8. Click Create Test

Step 2: Add Test Variations¶

After creating the test, you'll be directed to the test edit page:

1. In the Versions section, click Add New Version
2. For your control version (Version A):
3. Enter a name (e.g., "Original" or "Control")
4. Add your original content in the editor
5. Check "Set as Default" if you would like this to be your default version when no other version is selected.
6. Click Add Version
7. For your variant (Version B):
8. Click Add New Version again
9. Enter a name (e.g., "Variation B")
10. Add your alternative content
11. Click Add Version

Step 3: Implement the Test on Your Site¶

1. Copy the shortcode displayed in the test edit page (e.g., [ab_test test="123"])
2. Edit the page where you want to run the test
3. Paste the shortcode at the exact location where you want the test content to appear
4. Update the page

Step 4: Activate the Test¶

1. In the test edit page, ensure that the test status is set to "Active"
2. Click Update Test

Step 5: Track Conversions¶

Conversions are tracked automatically when users who have seen a test variation visit the conversion page you specified.

Step 6: Monitor Results¶

1. Navigate to A/B Tests in your WordPress admin
2. Click on your test name to view the results
3. Monitor the impressions, conversions, and conversion rates for each variation

When to End Your Test¶

Consider ending your test when:

• You've achieved statistical significance (indicated in the results dashboard)
• You've collected at least 100-300 conversions (not just impressions) per variation
• The test has run for at least 1-2 full business cycles (typically 2-4 weeks)
• You've accounted for day-of-week and time-of-month effects
• Conversion trends have stabilized (no major fluctuations in recent days)
• You've reached your predetermined sample size (calculated based on expected effect size)

Avoid common testing mistakes: - Ending tests too early when you see "significant" results - Ignoring external factors (seasonality, marketing campaigns, etc.) - Making decisions based on too few conversions

For high-traffic sites, you may reach conclusive results sooner. For low-traffic sites, prioritize patience over rushing to a decision.

Remember: Statistical significance doesn't necessarily mean practical significance. Consider the actual business impact of implementing a winning variation.

Next Steps¶

Now that you've created your first test, you might want to:

• Explore bucket-based targeting

For more detailed information, check out the Creating Tests section.