Multivariate and A/B Testing

Engaging Networks Testing Features: Page Split Test & Split Test Email

Engaging Networks offers robust native testing features, including Page Split Tests and Split Test Emails, which are highly effective for optimizing campaigns and donor engagement. We strongly recommend leveraging these built-in options whenever possible, as we’ve seen consistently positive results from both.

How It Works

1. Create Two Variations:
   Start by creating two separate donation forms with the variations you'd like to test. For example:

   • Form A might have a standard gift array with amounts of $25, $50, $100.
   • Form B might include a higher tier with amounts of $50, $100, $200.

2. Set Up the Split Form:
   After creating the two forms, set up a "split form" in Engaging Networks. This acts as the control mechanism for the test, randomly directing visitors to either Form A or Form B.

3. Generate a Single URL:
   The split form generates one unified URL that you can use across various platforms, such as:

   • Email campaigns
   • Paid ads
   • Website buttons or CTAs
   • Social media promotions

   This single URL ensures that the traffic is evenly split between the two forms, making it easy to manage and track the performance of your test.

Our Recommendation

• Audience Size:
  To achieve statistically significant results, we recommend to test on high trafficked pages and for a minimum of three months.

• Tracking:
  Create a tracking spreadsheet to review and iterate on your testing.

Why We Recommend It

Page Split Tests are particularly valuable for organizations looking to maximize their fundraising efforts without a significant time or resource investment. By making data-driven optimizations, you can enhance user experience and boost donation rates effectively.

Source: Page Builder Split Test - Supportal Documentation

Split Test Email

Split test emails work similarly—you create two versions of an email and test them against each other. One great feature is that you can choose what percentage of your audience to test. For example, if you have a query of 100,000 supporters, you can test on 25% of them. Once the winning version is identified, you can roll it out to the remaining 75% of your audience with just one click. While there’s an automatic rollout option, we prefer to manually control it based on the test elements. Alternatively, you can test on 100% of the audience to gather insights from a larger group. When running email tests, we recommend using a minimum audience of 20,000 supporters to achieve statistically significant results. Testing on fewer than that may not provide meaningful insights.

How It Works

1. Create Two Variations:
   Develop two versions of the email you'd like to test. For example:

   • Version A could have a button CTA.
   • Version B might include a more detailed text link CTA.

2. Configure the Test Parameters:
   Within Engaging Networks, set up the split test by defining the percentage of your audience to include in the test. Options include:

   • Testing on a smaller segment, such as 25% of your audience, and rolling out the winning version to the remaining 75%.
   • Testing on 100% of the audience to gather insights across the entire group.

3. Monitor Performance:
   Once the test emails are sent, Engaging Networks tracks performance metrics like click-through rates, and conversion rates to identify the winner.

4. Roll Out the Winning Version:

   • If testing on a smaller segment, you can use Engaging Networks' one-click option to send the winning version to the remaining audience.
   • While there’s an automatic rollout option, many prefer manual control to ensure the results align with specific campaign goals.

Our Recommendations

• Audience Size:
  To achieve statistically significant results, we recommend to test on a minimum audience of 20,000 supporters. Testing on fewer supporters may not provide meaningful insights.

• Manual vs. Automatic Rollout:
  While the automatic rollout feature is convenient, manual control is beneficial when testing nuanced elements, as it allows you to review the results before proceeding.

Why We Recommend It

The Split Test Email feature is a great built-in tool for improving the effectiveness of your emails. By understanding what engages your supporters, you can craft more targeted, compelling communications that drive results. Whether you’re aiming to boost conversions, enhance engagement, or learn more about your audience, this feature ensures your campaigns are informed by real data.

Source: Engaging Networks - Split Test Email Campaigns

Split Test Conditional Emails

Split test conditional emails combine the benefits of the traditional split testing with targeted audience segmentation, allowing you to serve email variations to specific supporter "profiles". This native feature enables you to create highly tailored messaging and provides deeper insights into what works best for different segments of your audience.

How It Works

1. Create Conditional Email Variantions:
   Develop two or more versions of an email, each tailored to specific audience profiles or supporter segments. For example:

   • Version A could target recurring donors with messaging focused on impact and appreciation.
   • Version B might target one-time donors with a call to action for recurring giving.

2. Set Up the Test in Engaging Networks:
   Use the Split Test Conditional Email feature to assign each variant to its intended audience profile. The system ensures that each version is only sent to the segment it was created for.

Main Differences

• Audience-Specific Targeting:
  Each email variant is assigned to a predefined audience segment, ensuring precise targeting.

• Multiple Variants:
  Unlike regular split tests that typically compare two email versions, conditional tests allow for multiple variants to be tested simultaneously across different profiles.

Our Recommendations

• Personalize Messaging:
  Targeted emails often have higher engagement rates compared to generic message. Therefore we recommend to create customized content for the different profile segments, increasing engagement and the likelihood of achieving campaign goals like increased conversion or engagement rates.

• Track Data:
  To get the most our of this feature, it is importatnt to understand what resonates best with each audience profile to refine your email marketing strategies. Consistent tracking and reporting is essential.

Why We Recommend It

Split Test Conditional Emails are ideal for organizations that want to move beyond general A/B testing and adopt a more segmented, data-driven approach. By tailoring email variations to specific audience profiles, you can deliver highly relevant content and maximize the impact of your campaigns.

Source: Engaging Networks - Split Test Conditional Email Campaigns

Interested in learning more?

We would love to hear from you. Contact us!