Recommended
195 | Add 'free' label to paying methods | Mobile
Last updated:19 February 2025
First validation:20 Jun 2019
Last validation:29 Jan 2025
Number:195

A/B test results
Number of A/B tests25
Win ratio56 %
Loss ratio8%
Inconclusive ratio36%
Average impact KPI+ 4.82%
KPI:Transactions
Visitors:316.950
Transactions:195.325
Device:Mobile
Pagetype:Checkout
Effort to implement:Low
Industry:Giftcards, General consumer goods, Electronics, Fashion & Shoes, Furniture & Home, Pets, Baby & Children, Jewelry & Luxury, Health & Beauty, DIY & Hardware
Theme:Payment method, Labels, USP communication
Channel:On-Site
Short description
In this experiment, we tested the addition of a 'free' label to the payment methods in a mobile checkout process.
Tips for applying to your e-commerce site
Do’s
Add the word 'free' at each payment method.
Use a contrasting colour for the label or word 'free', so it attracts attention.
Don'ts
In general: don't overcomplicate the payment process with unclear charges.
Only use this if all payment methods are completely free of additional charges.
This hypothesis is grounded in the following psychological principles:
- Motivation: adding a 'free' label may enhance the perceived value of the transaction, encouraging users to complete a purchase.
- Reduce Uncertainty: the 'free' label could reduce doubt or hesitation about hidden fees, making users more confident in proceeding.
Based on this theoretical background, we believe that adding a 'free' label to payment methods for online shoppers will cause an increase in online transactions. We will know this when we see online transactions increase.
Overall, these results suggest a small visual cue like a “Free” label can boost user trust and reduce friction. It works especially well when users are unsure about payment method costs.
Possible explanations why the impact was not as hypothesized:
- Already expected: Some users already assume that payment methods are free, so the label doesn’t change perception.
- Design overload: If the interface already contains many labels or icons, the “Free” tag can be overlooked.
- Lack of trust: In rare cases, users may see the “Free” tag as suspicious or too salesy, creating friction.
General
All A/B tests in Evidoo have been analysed using Bayesian Statistics. The most important advantage of Bayesian statistics is that it is easy to understand. If there is a difference between the control and the variant, we determine the probability that there is a difference. The probability that the variation differs from the control, is indicated in a percentage.
An A/B test labeled as ">80%" (winner), indicates that the hypothesis has a high probability (>80%) of being true.
An A/B test labeled as "21 - 79%" (inconclusive), suggests the hypothesis has an intermediate chance of being true. This probability range indicates that there is still uncertainty regarding the hypothesis. Therefore it can not be clearly categorized as true or false.
An A/B test labeled as "< 20%" (loser), likely represents a hypothesis that has less than a 20% chance of being true. This suggests that the hypothesis is likely false.
If the primairy KPI is ‘transactions’, there is no impact on average order value, unless mentioned in the learnings.
All A/B tests in Evidoo are also analysed on secondary KPIs, for example 'add to carts'. If we found remarkable results on other KPI’s, check the tab 'learnings'. We also analysed different segments, for example 'new visitors' or 'returning visitors'. If we found remarkable results on specific segments, check also the tab 'learnings'.
All conducted A/B tests in Evidoo comply with:
- conducting sample size and power checks.
- performing the experiment only after an A/A test has been completed.
- implementing Sample Ratio Mismatch (SRM) checks.
- maintaining a minimum runtime of 2 weeks.
- measuring A/B tests only when they are visible to the website visitor, such as counting an A/B test in the middle of the product page only if the visitor has scrolled to that point.
All A/B tests in Evidoo have been analysed using Bayesian Statistics. The most important advantage of Bayesian statistics is that it is easy to understand. If there is a difference between the control and the variant, we determine the probability that there is a difference. The probability that the variation differs from the control, is indicated in a percentage.
An A/B test labeled as ">80%" (winner), indicates that the hypothesis has a high probability (>80%) of being true.
An A/B test labeled as "21 - 79%" (inconclusive), suggests the hypothesis has an intermediate chance of being true. This probability range indicates that there is still uncertainty regarding the hypothesis. Therefore it can not be clearly categorized as true or false.
An A/B test labeled as "< 20%" (loser), likely represents a hypothesis that has less than a 20% chance of being true. This suggests that the hypothesis is likely false.
If the primairy KPI is ‘transactions’, there is no impact on average order value, unless mentioned in the learnings.
All A/B tests in Evidoo are also analysed on secondary KPIs, for example 'add to carts'. If we found remarkable results on other KPI’s, check the tab 'learnings'. We also analysed different segments, for example 'new visitors' or 'returning visitors'. If we found remarkable results on specific segments, check also the tab 'learnings'.
All conducted A/B tests in Evidoo comply with:
- conducting sample size and power checks.
- performing the experiment only after an A/A test has been completed.
- implementing Sample Ratio Mismatch (SRM) checks.
- maintaining a minimum runtime of 2 weeks.
- measuring A/B tests only when they are visible to the website visitor, such as counting an A/B test in the middle of the product page only if the visitor has scrolled to that point.
Questions? Visit our FAQ page. Need any advice?
Contact us