A/B test in the moderated User Testing

Just wondering if it’s common and practical to show our participants another version of our design to get their thoughts? My concern is validating version A without observing version B wouldn’t be a useful approach.

That would be preference testing. A/B testing can only be done on a live site with 1000s of visitors.

When preference testing you show on participant version A and then version B. The next participant you show version B first and then version A to reduce bias.


This is my favourite article on this topic:

Repeat after me: Preference testing is not A/B Testing

Researchers sometimes ask participants which of two alternative designs they prefer. The data from these studies comprise opinions that have little predictive value. In contrast, multivariate A/B testing involves target users doing real tasks. The data from these studies comprise behavioural observations that predict real-world behaviour.


1 Like

Welcome to the forum @Cam_Rogers!

Yes, definitely not the same. And there are specific ways to run preference testing to avoid bias. :grinning: