New homepage hero section

Hypothesis Data

New homepage hero section
Redesigning the hero section of the homepage will increase the primary conversion rate (CR1) by improving visual hierarchy and CTA clarity. We expect a 5% lift in CR1 with no negative impact on Clicks or Revenue.
CR1
5.0%
Homepage, hero section

Experiment Data

Control: No changes
Test 1: Redesigned hero section with updated visual layout and revised CTA button โ€” new color scheme and copy aimed at improving click-through to the primary conversion funnel
homepage_hero_ab_test
20000
Homepage
01.12.2025
01.12.2025
AB

Metadata

Alex N.
UI Homepage Conversion
AB Labz
26.05.2026 16:30

Analysis Results

Conclusions

**Overall conclusion of the experiment:** A total of 19,925 users participated in the test, which corresponds to 99.6% of the required sample size of 20,000. We tested a new design of the main block on the homepage, expecting a 5% increase in conversion rate (CR1). The results showed a significant effect in CR1 (+11.97%, p=0.0007) and a negative impact on Revenue (-1.90%, p=0.40). Despite the substantial improvement in conversion, the decline in revenue raises concerns. **Results by metrics:** - Key metric (CR1): an increase of 11.97% (p=0.0007) โ€” this is a significant effect that should be considered for implementation. - Significant metrics (p < 0.05): CR1 demonstrated a high growth, confirming the success of the new version. - Metrics with large effects, but p>0.05: Revenue showed a decline, but with a high p-value (0.40), which is not statistically significant. - ๐Ÿ›ก๏ธ Guardrail metrics: Revenue decreased by 1.90% (p=0.40) โ€” this is a minor change, but it does not raise alarms as it is not statistically significant. **Summary:** - Decision: I recommend **repeating the test**. We have a significant effect in CR1, but the uncertainty in Revenue requires additional data to assess the robustness of the results. - Risks: Revenue showed a negative change, albeit minor (p=0.40). This is a signal of potential long-term risks to revenue. - Ideas: It is worth exploring how design changes affect user behavior and conversions across different segments, as well as testing different CTA options to improve revenue.