Redesigning the hero section of the homepage will increase the primary conversion rate (CR1) by improving visual hierarchy and CTA clarity. We expect a 5% lift in CR1 with no negative impact on Clicks or Revenue.
CR1
5.0%
Homepage, hero section
Experiment Data
Control:No changes
Test 1:Redesigned hero section with updated visual layout and revised CTA button โ new color scheme and copy aimed at improving click-through to the primary conversion funnel
homepage_hero_ab_test
20000
Homepage
01.12.2025
01.12.2025
AB
Metadata
Alex N.
UIHomepageConversion
AB Labz
26.05.2026 16:30
Analysis Results
Conclusions
**Overall conclusion of the experiment:**
A total of 19,925 users participated in the test, which corresponds to 99.6% of the required sample size of 20,000. We tested a new design of the main block on the homepage, expecting a 5% increase in conversion rate (CR1). The results showed a significant effect in CR1 (+11.97%, p=0.0007) and a negative impact on Revenue (-1.90%, p=0.40). Despite the substantial improvement in conversion, the decline in revenue raises concerns.
**Results by metrics:**
- Key metric (CR1): an increase of 11.97% (p=0.0007) โ this is a significant effect that should be considered for implementation.
- Significant metrics (p < 0.05): CR1 demonstrated a high growth, confirming the success of the new version.
- Metrics with large effects, but p>0.05: Revenue showed a decline, but with a high p-value (0.40), which is not statistically significant.
- ๐ก๏ธ Guardrail metrics: Revenue decreased by 1.90% (p=0.40) โ this is a minor change, but it does not raise alarms as it is not statistically significant.
**Summary:**
- Decision: I recommend **repeating the test**. We have a significant effect in CR1, but the uncertainty in Revenue requires additional data to assess the robustness of the results.
- Risks: Revenue showed a negative change, albeit minor (p=0.40). This is a signal of potential long-term risks to revenue.
- Ideas: It is worth exploring how design changes affect user behavior and conversions across different segments, as well as testing different CTA options to improve revenue.