Marketing Automation
December 10 2024
%20(1).jpg?width=1280&height=720&name=674f017dbe4281db5986275a_Header%20Braze%20inl%C3%A4gg%20(1)%20(1).jpg)
Those of us who work in marketing know that it is not always easy to demonstrate the impact of a single campaign. The challenge doesn't get any easier when we want to show the impact of all marketing efforts. It's often hard to separate what's truly driving your campaigns from what would have happened anyway. Braze's Global Control Group (GCG) is part of the answer. It randomly selects a portion of your customer base that receives no marketing—helping you answer the question, "What would have happened if we did nothing at all?"
With GCG, we try to see the effectiveness of all our communications on key metrics (which can be purchases, listens, list creation, etc.).
This guide will walk you through how to use the Global Control Group in Braze to accurately measure the impact of your communications, optimize journeys and campaigns, and show the rest of the company why your team deserves recognition.
How to set up and use Global Control Group
Step 1: Create a Global Control Group
The first step is (obviously) to create the control group.
1. Navigate to Global Control Group Settings
- From the Braze dashboard (far left), go to Audience > Global Control Group
- If you are using an older version of the navigation, you can find this option under Engagement > Global Message Settings > Global Control Group Settings.
2. Assign a percentage of all users to the control group
- Enter the percentage of users you want to assign to your Global Control Group. Usually this is between 1% and 10%, depending on the size of your target audience and how accurate you want the results to be. Braze will estimate the number of users each percent represents.
If you have a low user count, a higher percentage may be needed to achieve confidence in the result. The important thing is to choose a group size that is statistically significant so you know that the difference between your GCG and the rest of the users is due to your communication and not pure chance. It may sound complicated, but understanding this is vital. You can read more about choosing the size of control groups here.
- The GCG is dynamic, which means that when the total number of users changes, the GCG will be adjusted to remain proportional. So it will remain at 10% if you decide it should be, even if you conjure up 100 000 new users tomorrow.
Tip for setting the minimum size: The Global Control Group should not be smaller than 1,000 users. Braze does not recommend having less. If you are unsure of how big a size you need, it may be a good idea to involve your data team.
3. Assign Exclusion Settings
- Tags allow you to specify which campaigns or Canvases should not be included in the Global Control Group. For example, transactional messages that must reach all users should be excluded. Excluding means that also the users in the GCG will receive information with these tags.
But think carefully before you also exclude other communications. For example, it may be tempting to exclude onboarding. After all, all customers need to receive it, right? But precisely because it is an important part of the communication during the customer journey, I think you should often make sure your control group does not receive it. Because, think back to the reason why you set up a GCG - you want to get an overview of the impact of all communications. Onboarding is often an important and perhaps large part of all communication from you to your customers. If the control group starts to look more and more like the rest of the users, you won't be able to get any insights you can trust.
4. Save
- Once you have set up all the settings, click Save. From now on, all current and future campaigns and Canvases (except those with specific exceptions) will exclude these users.
Tips for managing Global Control Groups: I recommend creating new Global Control Groups on a regular basis. This will ensure that no set of users is excluded from notifications for too long a period and ensure that the GCG is not too different from the rest of the users. A good guideline is to do a new one every three months (exactly how often differs per market of course).
Step 2: Analyzing Global Control Group Reports
Once you have created a Global Control Group, you can measure the effectiveness of your campaigns by comparing the performance of all users against the control group.
1. Navigate to Control Group Reporting
- Go to Analytics > Global Control Group Report. If you are using the older navigation system, you can find this under Data.
2. Select Parameters and Run the Report
- Select the metric you want to analyze (e.g. sessions or custom events). Click Run Report to see data on how the treatment group compares to the control group.
Examples of Possible Analyses: With the Global Control Group Report you can measure, among other things:
- How your messages impact app or web sessions.
- How much increased feature usage your communications drive. E.g. creating a wish list, or following a creator.
- Whether your marketing increased the average number of purchases per user.
3. Exporting Control Group Members
- If you need to access the users within your control group, you can export them via CSV or use the API endpoint `/users/export/global_control_group`.
- Note that historical control groups are not saved, so it is important to export the necessary data before disabling a group.
Step 3: Adjust or Disable your Global Control Group
The Global Control Group is flexible and can be adjusted or deactivated as needed.
1. Disable the Global Control Group
- Navigate back to Global Control Group Settings and click Disable if you no longer need the control group. Users who were previously in the control group can now immediately receive promotions.
- You can create a new Global Control Group after disabling the previous one. Even if you assign the same percentage, Braze will select a new group of users.
Tips and Best Practices
Optimal Control Group Size
- Your control group should be no fewer than 1,000 users and no more than 10% of your total target audience. Consult your data team to ensure you have thought correctly when choosing the control group size.
Consistent experiment length
- Keep the length of your experiments consistent to avoid bias. A good starting point is to run experiments for a quarter (about three months). Shorter experiments can produce unreliable results, especially for brands whose users interact infrequently. In addition, long experiments can introduce bias, as the control group may start to differ from the rest of the users due to things like new users being different and having different behaviors. After all, an effective campaign against a specific target group can introduce thousands of new users into your database who are different from the majority of existing users. Therefore, it is important to find a balance to maintain comparable groups.
Avoid testing and segmentation with Random Bucket Numbers
- Braze uses Random Bucket Numbers to assign users to the control group. If you use Random Bucket Numbers when testing or in existing segments, I recommend that you do not use these during the period you have Global Control Group active.
Do you need any help?
I hope this has helped you set up a Global Control Group. If you have any questions about that process, other Braze questions, or have any other MA/CRM needs, please don't hesitate to contact me or any of the rest of us here at Leadfront.
Have a great time!
