Just wondering if it’s common and practical to show our participants another version of our design to get their thoughts? My concern is validating version A without observing version B wouldn’t be a useful approach.
This is my favourite article on this topic:
Repeat after me: Preference testing is not A/B Testing
Researchers sometimes ask participants which of two alternative designs they prefer. The data from these studies comprise opinions that have little predictive value. In contrast, multivariate A/B testing involves target users doing real tasks. The data from these studies comprise behavioural observations that predict real-world behaviour.
https://www.userfocus.co.uk/articles/preference-testing-is-not-A-B-testing.html
1 Like